The whole point of getting people into an office together is so that they can communicate and get work done faster than if they were all telecommuting. I think headphones hurt that, it's annoying to have to tap people on the shoulder to ask them something.
Obviously you shouldn't bug people if you don't have to, but this applies with or without headphones.