Getting a pilot's license is not all that hard (almost every one of my co-workers has a VFR license and most of them own a single-engine ship.)
The hard part is getting and *keeping* an IFR ticket, where you have to put in so many flight hours that it's really tough to do if you're not a full-time commercial pilot. Let's not even talk about the costs of owning, leasing, or even just fueling and maintaining even a low-end private jet.
It's fantastically liberating to be able to fly your own plane, but it also tends to be quite limiting. Consider the range on your, let's say, Cessna 182, for the 7-8 hours max you'd want to be in the left seat. Also consider what happens when you're grounded or diverted by VFR.
Most private pilots still go via commercial carriers when they travel. Flying yourself from Los Angeles to Maine can be fun, but it's no less greuling (and often not much faster) than the equivalent road trip.
The "use it or lose it" factor of IFR currency (FAR 61.57) in reality pretty much requires you to fly continuously, and without IFR you're stuck with mainly recreational flying in a relatively limited geographical area, only in clear skies. It doesn't suck, but it is not in reality the substitute you hold it out to be, nor do the costs end at the price of school.
I'm an active FAA-certificated instrument and multi-engine flight instructor who likes to use light single-engine aircraft for personal travel for 150 to 500 mile trips in the northeastern United States. Anything beyond 500 miles = airline. Anything less than 150 = drive.
If a Cessna 182 were even capable of such endurance, would you want to sit in there for 7-8 hours? A 4 hour VFR flight is tiring. A 4 hour hand-flown IFR flight is more exhausting than an 8 hour shift of manual labor. IFR currency isn't "user it or lose it." The FAA comes up with some off-the-wall proposed regulations, but they haven't made currency THAT hard, yet.
If your instrument currency expires, you have 6 months to simply go for a ride with a pilot friend so that he can keep an eye outside while you fly the currency requirements on instruments with a view-limiting device that keeps you from seeing outside. If you've exceeded 6 months since currency expiration, you require an Instrument Proficiency Check from a flight instructor, designated pilot examiner or the FAA. See FAR 61.57(c). This is the worst state your instrument currency can be in. There is no "lose it" when it comes to instrument currency, and flying once every 6 months to keep your instrument currency is hardly continuous. Of course, being IFR current is NOT the same as being IFR proficient. This requires a much higher level of flying activity. However, it still doesn't require continuous flying by the pilot to maintain proficiency.
With that said, I hate flying commercially because of the TSA and their ineffective "security theater." Given my love for aviation, I find this criminally sad. Have you heard of the airline pilot who exposed the flaws and obvious gaps in TSA security at SFO via Youtube? I've seen similar gaps since 2003. The TSA is a joke. The Agency does just enough to make the customers feel good and enhance the perception of security enough to allow the TSA to "grow the business."