While i agree with you, looking from another perspective there could be a old man sitting on a chair just in front of his house, he spends his time looking at the people passing by. Noting the time, the recurrence, the way they are dressed, who do they walk by with, etc... He could infer a great lot of information about them. He never asked them for the permission to look at them or to gather information about them, but should he? Is this information theirs or his? Maybe both?
I don't have a solution but I think the problem is that i (like you probably) don't trust what these company are going to do with the information they gather. I have the intuition (i say intuition because i can't pinpoint the reasons) that the amount of information collected and interpreted corresponds to the amount of power that company will have and i'm not happy to give it away that way... But the problem is not only one of personal privacy, it is one of social privacy.