If we double the population we need to cut the pollution in half to stay constant, it's not higher math. That's a very touch subject of personal freedom, but condoms, birth control and China's one child policy is probably the best long term action for the environment.
It's not higher math, but it's also not correct. ;-)
There is not a fixed amount of CO2 produced per person so doubling the population doesn't necessarily double the pollution. Further there are often serious issues that result from population decline. Just look at Japan. Besides, most of the Western world has near zero population growth and that trend is moving into Asia. My guess Africa won't be THAT far behind. Yes, birth control should be provided and encouraged in developing countries but I don't think we really need to be draconian about it.
Population does matter but it's not everything. Take a look at the Mayans. Many folks feel that the Mayan empire collapsed because they weren't living sustainably. Certainly population growth played a role but so did slashing and burning the rain forest.