The same thing applies in an office environment, only instead of one wife you now have 50 colleagues who want help with this and that (often not work related), or just want to chat about the weather...
Many people are single, or have wives/girlfriends who also work and aren't at home during the day.
The other thing is being able to prioritise distractions... If someone sends you an email asking for something, you can wait until you're finished whatever you're currently concentrating on, but if they walk up to your desk or call you then it forces you to immediately stop what your doing to respond to them... This can be very troublesome if you're trying to concentrate.
My job has a mix of home working, office working, and working in client's offices... I find i get a LOT less done if i'm working in our office, partly because of the distractions and partly because it's just a terrible office with bad seating, bad desks, broken climate control etc.
Client offices are a different story, as most of the people there don't know you there are usually much fewer distractions, although the actual conditions can vary... If it's quiet and comfortable then i can get a lot more done there, if it's noisy and/or uncomfortable then a lot less gets done.
That said i still think home working is better overall at least for me, if only because of the time and inconvenience saved on travel... There are more and more businesses being crammed into a small area in most cities and expecting people to all work at the same time, this creates massive congestion on all travel routes at certain hours, and results in inefficient over capacity at other times. I find it utterly ridiculous how they insist on so many people travelling to the same area at the same time, things should be far more spread out.