At my local FBO, airplanes rent for between $110 and $170 an hour wet (with fuel) depending on the type and equipment. If you're a student, expect to pay between $25 and $50 an hour for instruction, and the average student (so I'm told) requires between 50 and 60 hours of instruction before they're ready to sit for exams. Add in about $200 for your medical and another $500 or so for leaning materials, another few hundred in miscellaneous costs, and the cost just get licensed is, at the low end, around $8,000 and can easily go in excess of $13,000+.
And then you've got your license. Then what? Have you looked at the cost of airplanes recently? There's a reason pretty much nobody buys airplanes anymore. Only clubs and flight schools own airplanes. You want something newer than 40 years old and seats 4 people, it will run you in excess of $50,000. And forget anything new. A new Cessna 172 currently goes for in excess of $300,000.
So yes. It's so expensive even to just learn to fly that it is effectively priced out of all the but (what's left of) the upper middle class and the wealthy.
But there's another issue, too, that I think warrants some attention: health.
So many things that are considered "common" diagnoses now and are easily treatable, such as high blood pressure, ADHD, depression, etc. are considered disqualifying conditions by the FAA. Even though many of these conditions are easily treatable by modern medicine, they're disqualifying for even a third-class (private pilot) medical certificate.
While the costs are what is primarily keeping people away from flying right now, the archaic medical certifying process used by the FAA is not helping.