As for police, the problem is that police investigations reveal irrelevant private information. That's something we've just got to accept if we want the police to do anything at all. However, we don't have to let the police collect irrelevant private information when that isn't part of an investigation of a crime. In other words, the ratio of criminals caught to private information collected is too low.
There's also the general problem of "false positives", which have been notoriously common in previous security-related data collection. This was especially common in the "Red Scare" investigations of the 1950s to 1980s.
Back in the 1970s, there was an example that got a bit of coverage in the scientific press. There was a researcher (in Detroit as I recall) who had applied for lots of federal grants, and had been turned down for all of them with no explanation. Eventually, via the FOIA (Freedom Of Information Act), he eventually found the explanation.
It seems that earlier, his teenage son had been using his car frequently to visit his girlfriend in another part of town. Some agencies looking for "subversives" listed a local group that held meetings occasionally in the same block. When the meetings were scheduled, the investigators visited the block, and wrote down all the license-plate numbers. They compared these with the registry, and the owners of all the cars who didn't live in the area were listed as suspected members of the group. So the father was listed as a suspected subversive, and that information was given to funding agencies.
Presumably the investigators didn't notice that his car was there on lots of other days, because they didn't do their scans on non-meeting evenings. This is one good way to get a false positive.
I never read any followups to this story. It's unlikely he had any legal recourse, since failing to give grant money so someone on the basis of false data about them isn't exactly a crime.
Those who use the "we have nothing to hide" argument should probably consider stories like this. Political investigative agencies have a long, sordid history of such false positives, and they've ruined a lot of lives as a result, while typically catching few "true positivies" in their nets.
I wonder if it would be possible to set up a list of such stories, for the education of the general population. It could be useful to impress on people that, no matter how much of a "good citizen" they consider themselves, they can easily be victimized in this way.