As for the primitives, nobody has ever said they slow modern clients. The point is that clients don't even use the primitives any more and its the same story applies for most of the rest of X11. It has a 1980s 2D-centric, damage based view of the desktop and extensions are used to fool it into supporting surfaces and composition. But those extensions are workarounds which are design compromised by the architecture and so they are slower and less efficient than they would be.
Hence the push for Wayland. It will ultimately lead to a more lightweight and responsive desktop. Fedora Core 23 will be released soon and GNOME 3 should be feature complete for Wayland. And 24 might flip the switch and make Wayland the default. It doesn't stop people using X11 if they want so I don't see the problem.