I don't think even that's the problem. I find that many things that work reasonably well in Windows and Mac OS X do not work properly on many Linux distributions. There may be understandable reasons for this, but in practical terms, it's a really big problem.
For example, I have a docking station for my Lenovo X201. When I put my laptop on the docking station, it should automatically switch to the external display - at the correct resolution. When I open my laptop lid, it should activate both. When I boot up while docked and lid closed, only external display should come on at the correct resolution. About an year ago (which is when I tested last), it didn't do any of these things perfectly, It kept forgetting the resolution of the external display, and I had to keep readjusting it. Opening and closing the lid was a slow and unbearable affair.
This is apart from the fact that the graphics are pretty sluggish, with occasional tearing etc. Scrolling and panning were also fairly slow. Intel drivers are correctly installed. The UI just doesn't have the polish and smoothness that Android, Windows and OS X do. The fonts are also pretty ugly by default, The buttons and layouts look squished or otherwise disproportionate. There are many many similar hiccups as the ones outlined above. As a point of comparison, I'll point out that I started using Mac OS X only recently, and have found it instantly more pleasant and intuitive to use, although I still find Windows to provide the most flexibility, especially when it comes to multi-monitor support.
Android is a testament to the fact that fluid and beautiful desktops on Linux are entirely possible, on a range of hardware. I think KDE (my favourite) and Gnome just need to stop worrying about new features, and just polish their existing experience. Alternatively, maybe the trick to finally having Linux on the Desktop, is to have Android on the Desktop.