The problem you miss is that of immunocompromised or otherwise medically exempted individuals - namely, people who don't have a choice whether they get vaccinated or not.
Then you just shot down your whole argument that everyone be immunized. Again, if you fear your kid getting a disease then keep your kid away from others or immunize them. You can control that where you can't control the actions (or inaction) of others.
And to answer the AC below about babies, they inherit the mother's antibodies through the placenta and later through the mother's breast milk. Want to protect your baby, then have the mother immunized.