Sure, Android fragmentation is a real issue. However, before Android, just about every phone manufacturer had its own operating system, and it was difficult to do development for.
It isn't like if Android didn't exist, everything would just run iOS. If Android didn't exist, we'd likely have a situation where every vendor has their own entirely different platform. That'd be real fragmentation in the phone industry.
Right now, Android is much like Windows. You don't know exactly what version a user will have, and what hardware and configuration they'll have, but at least there is a set of common APIs you can rely on. Thanks to majority market share, you can develop an Android app and get a massive chunk of the market, even if that app needs some code to deal with specific versions of Android.