Not that I agree with high property tax by any means, to me it's like never being able to truly own a piece of land. But it's not the Fund Manager's fault the farmer spends all his money on more taxable assets nor anyone but the farmer who is (presumably?) investing in assets he's not getting adequate (again presumption wise) return on.
Also, your argument falls flat on the fact that someone is paying the property tax on the "swanky house" . If there is a property, there is property tax. I don't see how you can blame renters for not paying their fair share, that's ridiculous. The cost is built into the rent, and actuality in upstate NY, people owning houses pay they rent out pay more property tax because they don't get STAR credit for owner occupied housing with no children in the school district.