The amount of energy taken out is surely small. It's not like the other side of the windmill has no air movement. The blades only capture a part of the wind's energy; the air has to keep moving beyond the blades, otherwise the wind would hit the blades, stop, and the blades wouldn't move. Gently blow on a pinwheel and you'll feel the air moving on the other side of the pinwheel.
No one asks this question whenever we put up a new 10 story building, but they must absorb more wind energy. A 10 story building is a solid block and stops almost more of the wind that hits it; I assume some air is buffeted out and moves around the building, but not much. In addition, wind mills don't run if the wind speed is above or below a certain speed. I want to say something like 25mph, but I can't be arsed to google right now. Buildings block all the wind, regardless of speed. Well, up until the wind knocks them down.
I'd bet a whole dollar that the buildings in a single large city like New York, London, Chicago, etc. capture and disrupt far more wind energy that every windmill on the planet today and every one planned for the next 20 years.