If you guys are comfortable letting Apple or anyone else have this, it's just because your brain hasn't digested what it means yet. Don't worry, wait for the first few scandals. It will take a few years - maybe long enough for every asshole company to start doing this. But it will get easier to understand.
I voluntarily send my location to Google every 2 hours (via Latitude). Why should I care if they know where I am? I mean, what exactly are you, some kind of spy?
Nobody cares where *you* are. You're just not that important. Sorry to bust your ego-bubble.
They only really care where people are in aggregate. That information is far more useful.
And if where you are does actually matter, then *turn it off*. Simple enough to do, really.