Ah ha... so if I'm out at a bar, and somebody has their smartphone out and is say, checking their email, then I can object and have reasonable expectation that they put the smartphone away in their pocket or leave the bar?
It reminds my of an acquaintance of mine who wrote about his glass experience, saying that he was out at a street festival and was confronted by a street performer who was worried he was recording the show. The response was 'no, I'm not, but there are three or four other people around here with smartphones who are' and he pointed them out.
Obviously, seeing people wearing Google glass the first few times can be off putting (presumably), but I've heard people say more than once that they were pretending to do something on their phone but were actually taking video.
I think it's too late really to do much about this - people already have easy access to video cameras that are commonly carried around in public and give little to no indication that they are recording. The glass is just one more way to do it.