Xorg, which on desktop is as critical as init to keep running, is not really simple.
Never go full retard. X is not even remotely as important as init. For one thing, if X dies, who will restart it? And do we really want computers that explode when the GUI dies? I, for one, would like network services to terminate gracefully. The whole idea of TCP/IP networks, the dominant network used with Unix, is peer-to-peer. I may well run both services and clients on my machine. If X dies, the clients may die (if they're not text and running in screen) but the servers won't.
kernel, which is also as critical as init to keep running, and it is *much* *much* more complex than systemd. systemd is not at the "bottom layer" of the system, there's the whole of kernel underneath still.
So the argument is that since the kernel is complex, we should add more complexity, or that more complexity won't matter? That's an ignorant, illogical argument.
And one common myth is that systemd has these so many features and systemd is pid 1 therefore pid 1 is this huge bloated monster that does udev, logging and NTP, right? Wrong; actually, just the core bits of systemd run in pid 1 and the rest is compartmentalized in a bunch of separate daemon processes.
Systemd still has to be more complicated so that it knows how to run these other processes, which wasn't even necessary. init was never meant to manage daemons. daemons were meant to manage themselves, or be run from inetd. If you want more complexity, inetd is the place to add it. And for handling daemons which don't adequately manage themselves, there's daemontools. There was simply no need whatsoever for this to happen.
So, this "increased complexity" issue is not really as bad as it sounds, realistically.
It is bad, because PID1 is now responsible for a bunch of things which could have existed in any other daemon. And rather than roll the things which actually make sense in together, everything is getting rolled together. So now not only do we depend on a complex kernel, but we depend on a needlessly complex init system. There was no good reason to put all of this stuff into the same daemon.