The right of self-determination. The freedom for you to live your life as you wish, making choices for yourself as you see fit, not as someone else thinks you should live.
So it boils down to which of these noble rights you think is greater. The right of self-determination, or the right for society to attempt to rid itself of certain diseases. Unfortunately, when resolving conflicts between two noble rights, most people stack the deck. They pick a scenario which supports their predetermined conclusion. For those supporting vaccination (note: I support vaccination), this usually means proffering the scenario of someone refusing to vaccinate their children, resulting in their children getting sick and the disease spreading to others because of lack of herd immunity.
Just to play devil's advocate, someone arguing the opposite extreme would bring up the case of a corrupt government staying in control by locking up and drugging "troublemakers" (aka dissidents). The reality usually lies somewhere between these two extremes.
As I said, I believe in and support vaccination. However, I cannot in good conscience support forcing people who don't believe in it to be vaccinated. At least not with our current system of government. If you do not grant that right of self-determination to others, on what basis can you argue that others should grant it to you? If there were less corruption in government, if I were more confident about the safeguards build into it, then I would probably go the other way. But based on what I've seen, no. History is replete with those in power doing harmful things to others against their wishes with the best of intentions. Our current government simply should not have the power to require everyone be injected with medicines of its choosing. If that makes me fall within your definition of an anti-vaxxer, then so be it.
The way I see it, the anti-vaxxer problem needs to be solved by educating people so they will make the correct decision on their own. Not by subjugating refusers and forcing them to do something against their will. Yes I know that's the hard way. But the easiest way would be to simply put a bullet in their heads. Denying them the right of self-determination is halfway to denying them their life. (It's interesting to note that people who've grown up in or have experienced repressive governments tend to think the right of individual self-determination is paramount. While those who have lived all their lives under a benign government tend to be the ones who think society's rights should overrule the individual's. I'll leave it to you to figure out which of these groups is deciding based on experience and evidence, and which is deciding based on naive idealism.)