The problem on the ground in New Zealand seems to be something of a side effect of being New Zealand -- a small yet prosperous nation at the end of the world and the end of the supply chain. They kind of need to be an attractive marketplace for sellers otherwise they may not be worth the effort of supplying.
But my problem with meta-national tax strategies isn't really with the avoidance of taxes so much, although slightly irksome. Apple does pay a lot of taxes, although perhaps not enough to specific jurisdictions. If anything, I think Apple owe the US government more as it is the civil authority with the most clout to protect its intellectual property and business interests.
My problem is that so many of these big companies are so profitable yet they just hoard the profits without doing much of anything with them but dump them in short term cash-equivalent securities. They invest tiny portions in R&D, pay out tiny portions in dividends, pay large executive bonuses and then sit on the rest, mostly using it to buy up products that challenge their market dominance.
This last bit skews the larger innovation landscape through perverse motivations on innovators who see winning the buyout lottery as the main end-goal in innovation. Instead of focusing on creating new companies with competitive products, they create new companies that look like competitive products but end up just being buyout bait.
IMHO, the main problem with our current iteration of capitalism is that it enables hoarding of capital and hoarded capital doesn't get put efficiently to work in the economy, and only seems to get put to work staving off competition.