Maemo Linux, the OS that runs my n900, runs Xorg just fine. I don't think GNOME would be a good fit, but Hildon does use GTK, dbus, hald, pulseaudio, NetworkManager, and the evolution DB backend. And while you can't run amd64.deb, it does have a repository of ARM arch .debs. I think the best way to define what Slashdot would call linux is "capable of running wireshark". And the N900 meets that task, but does hilight your point that desktop apps are not in and of themselves, ready for mobile. I think the opposite might be true though; things that work on phones should be able to work natively on desktops without much fuss.
Maemo goes back to the n770 in 2005, so Android is hardly the first ARM phone attempt. Frankly, the reasons for success and failure here don't involve Linux technicals. It's about market position and strategies. Nokia has so many base patents on cellphones that they feel entitled to dictate the pace of phone growth. Their Maemo smartphone strategy assumed a leisurely 5 years and a patent portfolio to stop competition from racing ahead. Well that didn't stop Apple, or Google, and Nokia is happy to collect their profits in the form of patent settlements instead, while the board of directors approves a backdoor sale of Nokia to Microsoft.