X is optimized for programs that use a small number of colors to draw an effectively vector-based user interface on a raster display. It is very, very good at that, and provides a powerful range of tools for the job.
Most programs use color-rich bitmap-based user interfaces. Doing this with core X functionality is painfully slow and difficult (think tens of seconds to draw a 800x600 JPEG), so everyone uses protocol extensions for this. Wayland is designed around bitmap-based drawing at the core.