If it's open source and *doesn't* have a GUI, it's probably fantastic. My email, programming, backups, version control etc. is all open source and I wouldn't have it any other way.
But as soon as you add a GUI and plug in a monitor, the quality drops away and things start to get iffy. What happened with KDE4, for example, was unacceptable. You can't just dump everything and expect users to accomodate that.
And stability. A lot of open source apps are fantastic but they have rough edges - little bugs and issues. The way media managers like Rhythmbox and Amarok handle an iPod, for example: sometimes I get weird errors about mounting the iPod, or it doesn't behave properly when there's no free space left, and other little issues. They may not be show stoppers, but they're enough to give you a bad impression. The quality just isn't quite there.
And you know what the worst part is? This isn't getting any better. Open source GUIs are about the same quality now as they were a decade ago. Sure they're more capable, but all the rough edges are still there and don't seem to be going away. I've been using desktop Linux since Redhat 5.2 and I can honestly say the standards and general incompleteness, relative to the competition, are about the same today as they were back then.
I still use Linux on my desktop but I'm tempted to buy a Mac next time and use it as a front-end, while keeping all the 'real' stuff on a Linux box. But I don't want to manage two computers if I can help it. Ho hum.