There are several core issues in regards to smoking that make it different in key respects from doing anything harmful to yourself.
1) I am all in favor of allowing things that damage oneself to be legal. That's the essence of the nanny-state discussion. So I'm with people who are against the idea of a nanny-state. BUT smoking is harmful to others. Go ahead and damage yourself, but when you damage others who have not given consent, that is crossing a line.
2) Smoking by its nature releases smoke into the air and is therefore the business of everyone that encounters that smoke. This smoke is a) addictive, b) nasty-smelling, c) a mind-altering substance, and d) causes cancer. To those who smoke, would be fine if a stranger a) put an addictive substance in your food without your consent, b) farted such a nasty fart near you that it stuck in your clothes for days, c) slipped mild mind-altering drugs into your food/drink, or d) released carcinogenic gases into your environment?
3) On mind-altering substances, I'm very much of the philosophy that people should be allowed to use them. But nobody should be allowed to MAKE me use them. Nicotine affects brain chemistry. While it may be a fix for you, it makes me feel ill and causes terrible head pain.
It's a no-brainer. The violent reaction from smokers to the thought of banning smoking from public places and even outside is no doubt related to the thought of being unable to get a fix with one of the most addictive chemicals known to man. Remove that, and you have a case where smoking causes obvious infringements to the people around you, each of which clearly should not be allowed.
If all smokers switched to the patch, the problem would be entirely solved in my mind. Do what you want, but keep it to yourself.