I don't use drugs because they are illegal. My employer could terminate my employment, I could be jailed, etc.
If drugs were legalized, then there would be no barrier (however artificial) for me to start using drugs. Maybe I enjoy it and starts affecting my productivity, my way of life, I become addicted, etc.
So while this might be in the minority, there are a number of citizens who are not using drugs for the sole reason that they are NOT legal. Legalizing them would remove this barrier and actually cause more drug use. I am not saying that is horrible, but it is a side-effect that is seldom brought up.