Anytime someone asks questions about my concern for privacy online and why I find data collection so dangerous ("But I am doing nothing wrong and have nothing to hide, so why should I care?"), I point to McCarthyism and the anti-communism mania from the 1940's and 1950's. In the late 1940's, the House Committee on Un-American Activities (Yes, that's really the name of a U.S. House of Representatives investigative committee) began to subpoena Hollywood types (screenwriters, directors, actors, etc.) and ask them to testify about known or suspected membership in the Communist Party, association with its members, or support of its beliefs.
This committee would ask people to name names of colleagues with Communist affiliations. They saw a massive web of dissidents just based on affiliation. No bad deeds required other than showing up at a house party 20 years earlier. A paranoid black spot on our history that had serious repercussions for a lot of professionals when private companies started to blacklist people (See the Waldorf Statement) based on little to no evidence. Just being subpoenaed by the committee was enough cause to lose your job and not be able to find another job. Why would you be subpoenaed? Who know? Did you go to a coffee shop where "Communist" meetings were held? Did you best friend become a Communist? You could be roped in and tarred when you hadn't done anything wrong. It was even worse if you were actually a member of the Communist party (something that was not illegal) ten years before.
Can you imagine how horrible that era would have been if they government had just the Metadata on every single person in the country? Who you messaged, called, etc. tells a lot about your network, and you are going to be 'guilty by association' for a lot of things that you are not necessarily guilty of. It may be wrong of me, but I often draw parallels in modern times with what happened then. Are there any ideas they or their friends hold that may become more unpopular in ten or twenty years? Do they have religios beliefs that will be considered 'bad' in the future? We don't know... but if it does we have a record of every person you talked to. And that's just the Metadata. Browsing habits, actual content of communication: apparently a lot of that was collected (if not retained). Location is just as dangerous. There is a record of nearly everywhere you have been since you they started logging where your phone has been. Every day for a month, were you at Starbucks around the same time as a future terrorist? You wouldn't know, but the government does.
People are damned by what they say, even if what they say wasn't wrong when it was said. It can get taken and twisted by a motivated agent, and we are giving them the ammo future McCarthys need to do horrible damage to society. Sure, no one is actually looking at you when your data sits in a massive data warehouse, but when that data becomes relevant or certain ideas are labeled as 'dangerous', it's there for discovery. What was once 'no evidence of wrongdoing' at one time becomes the noose that is used to hang you in the future.