People complained about the bulk and weight of having a removable cover and another layer of hard plastic around the battery.
And besides, what are you talking about? Extra plastic? A non-removable battery is still covered by the phone case. There's no extra layer of hard plastic, just the small tabs or whatever mechanism keeps the cover attached.
I suspect Entrope is talking about the "hard plastic around the battery". A non-removable battery is covered by a thin, easily-punctured sheet of plastic, and that is then covered by the phone case. A removable battery is covered by a more substantial layer of plastic so it's more difficult to puncture or bend the actual battery inside--because that could be bad. And that is then covered by the phone case.