But what of carriers (Typhoid Mary being the obvious example) and conditions where you can be contagious and asymptomatic?
Typhoid Mary is a particularly poor example here because she was well aware that she was contagious, having been informed of that fact on several occasions, and yet repeatedly placed herself in an ideal position to pass the disease on to others. That isn't negligence, it's deliberate harm.
As for the rare cases where one can be asymptomatic and yet contagious, that's a risk you'll just have to take. It's not like vaccination eliminates that risk; even ignoring the fact that it isn't 100% effective, those who are immune can still be carriers. The most effective response in this case is to practice basic sanitation measures and limit direct contact, regardless of vaccination status.
It is reasonable for society to impose certain restrictions upon your freedom in exchange for the privilege of being a participant.
Nonsense. Putting aside the slip into ambiguous collectivist language ("society" does nothing; only individuals are capable of making choices and taking action), it is reasonable for you to exercise your freedom and refrain from contact with the unvaccinated, if that is your choice. Your fears do not justify restricting the freedom of others.
I trust that when I let my child play with your child that you will do a whole list of things, and one of those is that you will do your best to ensure my child is not exposed to life threatening conditions.
Sure, and there's nothing wrong with that. The ability to trust in others on the basis of common experiences and values is a good thing, when it isn't being abused as an excuse for aggression. But don't trust blindly; it's up to you to take steps to ensure that the other parents you associate with are in agreement with you regarding what is reasonable and necessary for the protection of all your children. And if it happens that such agreement is lacking, to find a voluntary response to the situation rather than resorting to violence and threats.
As I said before, I am not opposed to vaccination per se. It's a great invention and most people should choose to be vaccinated and to vaccinate their children unless they have a good medical reason not to. All I'm saying is that people should not be forced to undergo a medical procedure against their will (or against their parents' will, in the case of children), and that the choice to avoid vaccination is not, in and of itself, an act of violence against others—negligent or otherwise.
In the end, you want everyone else to be vaccinated so that you (and your kids) do not run the risk of accidentally contracting a disease against your will, which you consider harmful. To that end, you're willing to deliberately force others to undergo a medical procedure against their will, which they consider harmful. The hypocrisy in this position should be self-evident.