How does that happen? Why do people stop using their land for farming? Do they just keep it to have a nice place to spend a weekend? Or is the land so cheap that they don't actually worry about paying property taxes?
I can't speak for the midwest but in New England there were once a lot of fairly small farms. If a house comes with an additional 5-50 acres of property, at $1000 an acre for rural land, it may not add a great deal to the cost of the house. If it's wetland, and therefore difficult to develop or harvest timber from, around here it might go for $500 an acre. Many people with a few tens of acres in this area are engaged in small scale timber harvesting so having the extra land isn't necessarily a financial burden.
As far as taxes, some people will place "the back 40" in to tree growth. State law here allows a landowner to develop a timber harvest plan and get a significant reduction on property tax. In unincorporated parts of the state, I've heard this amounts to $1/acre per year in total tax. I don't know how much of a tax rebate individuals get inside an incorporated town but it is very significant. A number of communities have been complaining about the state mandated tax abatement program and urging reforms because of abuse. For example, owners of waterfront property have been known to place the land into tree growth even though they couldn't possibly harvest the timber due it's close proximity to water -- environmental laws. Of course, this is some of the more expensive property as well.
When I lived outside Seattle I heard of tax abatement programs for landowners who use their property for agricultural uses. Some of the requirements were pretty minimal. We had neighbors who stabled horses or bread a horse per year specifically so they could receive abatements which were only available for land used for agriculture. Property taxes were quite high there so I can certainly see the appeal of working the system.