Due to custom compilation of hardware. The OS requires firmware for all of the interfaces and chips. While you may be able to get away with a "One size fits all" solution like Ubuntu on an AMD/Intel chip, there's a huge variety of ARM version chips out there, each with different clock speeds and (presumably) instruction sets. Not to mention all the different WiFi, Bluetooth, and GSM/Edge antennae.
On x86 platforms, we have standards for dealing with things like booting, drivers etc. That's what's needed for ARM too. It would help not only with smartphones, but also with SBCs like the Raspberry Pi, Odroid etc.
Somehow, I suspect it will be hard to get the different manufacturers to agree on such standards. Oblig. XKCD: https://xkcd.com/927/