SOME programmers walk around in the "10 types of people" t-shirt and what-not. They are actually hyper-competitive and offensive people in my experience. They alienate men too. Myself, I wasn't really an outcast as you put it. But I could and do appreciate the astounding edifice we are standing on and I was amazed as a child by the various computers I encountered as they appeared in my lives. I wanted, instinctively, to understand how they worked and how to control them if possible, like any machine or system I come across.
But yet, the computer has never really done what I have told it to do. The normal result, of course, is that the program immediately blows up on the next line of code. Sometimes it works; and that is the reward, the micro-reward who's puirsuit I suspect defines a programmer. Sometimes, so rarely, the project bug list is actually emptied. Troubleshooting and debugging code is what makes it hard, tiring work. But the rewards are intellectually satisfying, like solving cryptic crossword puzzles, even if it's as ephemeral as cocaine.
I am sure it's environmental, the actual work involved can be done by woman as easily and proficiently as men. We just have to kill all the nerds first.