So much this. It's a great competitive activity for programmers/developers/nerds who don't generally have the most athletic ability. Dare I say it's almost gender neutral as it places a lot more emphasis on hand-eye coordination and quick reflexes rather than any real amount of strength.
I used to play against my mom when I was still in high school and she would regularly slaughter me without even trying. She'd have me chasing her returns from one side of the table to the other and then finish the volley with a nice subtle spin return that would leave me befuddled.
I run and work out regularly and still love to play ping pong. I think the reason it's looked down upon by a lot of people is because it's not so much a macho game. But that's all the better for me.