This is simply not true. I have downgraded several times.
I unfortunately did NOT backup my system before performing an upgrade to the first Beta of Yosemite. It worked fine for a days, but I noticed some oddities, and then I saw that my system started to crash more often. Since I didn't have a backup, I put in the for Snow Leopard, and away it went. It installed fine (though it took a couple of extra steps, since the Recovery partition Yosemite creates is not compatible and had to wipe that first). I then had to upgrade from Snow Leopard to Mavericks from the Apple Store. I had a similar problem on the version before Lion (had a backup that time), and was able to simply restore the backup and was up and running in very short order (two hours tops).
I was a very heavy Windows user for more than 10 years, but now that I have switched to OS X, I will never go back.
I will admit that the Hidden Files thing and the Home / End / Page Up / Page Down still drives me nuts, 4 years after I started using it. I understand that you can press Cmd + Whatever, but I would rather it had been a single key.
The rest of the OS is great. I have two UI developers sitting across from me at work, and they use their Macs to the fullest extent possible and Retina display with the trillions of colors and subtlety makes all the difference, but to me it has been a great experience.