The portability of GTK+ is, to put it politely, utter rubbish. X11 is the only platform where it isn't a disgrace.
The portability of GTK+ is also fairly irrelevant when it comes to a desktop for Unix. As long as you can use it with X11 today and either Wayland or X11 tomorrow, it's a suitable toolkit for the development of a Unix DE.
It would be nice to see GIMP and other apps move away from GTK, but uh, GIMP, GTK, etc. But I don't think it matters much for XFCE. If anything, what I want is for my DE not to be based on a major toolkit. This breaks down when it gets to the file manager, but it's not clear that the fm even needs to be closely integrated with the desktop unless you want icons on your desktop. I don't really feel that I need a "desktop", as it were. I only use it on Windows, and then only because it's easy to get to.