If you don't continuously test, you're not doing it right. Maybe I'm biased because I practice TDD.
Let's say you're right and that it takes me 5 iterations to test and get the result right. Now I have at least 6 different resolutions to test on for android, and at least 3 different skins for the device maker. That's about 18 different devices I need to test. Each one requires launching a new emulator. If I need to make a change for any emulator (for example, the red button on Motoblur doesn't look right or the strok handling on HTC sense is different) I get to do it again.
Now let's talk data. What does the UI look like with a lot of data, with little data, with pathologic data? Each of those requires testing on each version.
Suddenly, we have a combinatorial explosion. This where I see the big problem with android fragmentation.