Linus is of course entitled to use any DE, and to criticize any particular piece of GNU/Linux (or anything else under the sun) he pleases. I used xfce for a couple of years mainly for it's alleged, 'lightweight' aspect. Then it turned out it uses nearly as much system resources as Gnome 2 and for that matter, KDE 4x for several common tasks. To be frank, in comparison to Gnome and KDE's current iterations, XFCE feels old, kludgy and primitive (the most recent XFCE release shows some incremental improvements buttressing the same, outdated philosophy and poor underlying design principles). I don't hate XFCE, and I don't wish it didn't exist, I just think there are far better alternatives. For lightweight DE's, look instead at something like Openbox (as implemented e.g. on the Crunchbang debian distro).
Gnome 3 can best be described as a work in progress. We don't yet know how it's many shortcomings are going to be addressed. Still, I'm glad it's here, and I think it's a very necessary step for the Gnome project. If you really want to try it, skip the vm nonsense (actually, it ran very well for me in VirtualBox), and keep it updated, because the improvements are coming fast and thick.
By tossing off these kind of unreasoning and reactive quips about topics on which he has few valid insights, Linus only diminishes his own stature. Just sayin...