People keep on comparing Glass to bluetooth headsets without actually reflecting on why we hate them. It bears repeating: we hate them because of those several awkward seconds where you try to reply, thinking you're being addressed. The "asshole" part comes when the headset user says something like "hold on, this guy thinks I'm talking to him" or something else that implies you're an idiot for not immediately recognizing the headset. It's embarrassing, and insulting, and dismissive. In short, it takes basic social conventions and protocol and rudely slugs it in the face. Said social conventions, even the customary "good morning" a fuel station clerk greets you with, is lubricant for the social gears of society, and those headset users are sand in the works. It's not the headsets at all - its the people using them that never apologize for the misconceptions they cause, or politely put their conversation on hold when they walk up to a pay window.
Everyone screams and wails about being "recorded in public," which I find hilarious, considering how much we're already recorded, tracked and observed. If you're in public, people can record you freely, and no court of law is going to give a rats ass that somebody was able to SEE you when you went walking around on a public sidewalk. No, the real discomfort comes from having a computer screen between you and the person you're talking to. Google Glass is the first step towards things like augmented reality and other such technologies; but the precedent we've all learned from is the Arrogant Headset Asshole; and so naturally that's the first association we make.