"The problem is that people get irritated when people are casually pointing cameras at them the whole time."
Good theory, but does it match the data? Are the people being assaulted for wearing google glasses being assaulted when they have been pointing the glasses at someone for an extended period of time in an environment when they expect to not be recorded? Or have the attacks occurred in public places which are likely already under video surveillance and full of people snapping photos of friends and bystanders?
Alternative theory: People like hating on overt geekiness. Hypothesis: If hovercars were invented and were sold for ten million dollars each, people would love them, but if only geeky hobbyists had them, people would smash them in the street.