The kernel can execute ring 0 instructions. Your initrd can't. The difference is that you could construct an appropriately modified hibernation image that booted an arbitrary kernel - or even an entirely separate OS. In that scenario, your kernel is effectively a new bootloader, except unlike the signed bootloaders it'll happily boot an entirely unsigned OS. That's unlikely to end well.
But, conceptually, you're right. Secure Boot doesn't magically make a system secure, but it *is* a vital part of system security - if you can't trust your kernel, any other security you attempt to build is pretty much pointless.