As if anyone gives a fuck where you drive.
That's far too simplistic a way to look at it.
The problem isn't anyone wanting to know where people drive. The problem is what happens when you combine this database with a few others, and the analytics that are possible.
Do some research. The level of detail of a persons life that they can identify from these "harmlessly tiny bits of information" is staggering.