This will make things worse, not better.
One of the reasons we need public schools in the first place is to protect children from their parents, so to speak. Our country has a very real interest in making sure that all children have access to quality education, including children born to parents who don't believe in science, don't believe in government, or don't believe in education in the first place.
The fist thing that will happen if the government gets out of the public schools game will be that a whole chunk of our taxes is going to flow towards supporting backwards "schools" where children will be indoctrinated into any one of the existing religions, and not get to do any actual learning. Over time, these children will lose out to those, who get a real education, but this won't actually be good for anyone. Ignorance begets ignorance, and these people, having lost access to a whole slew of opportunities as adults, will happily send their children to the same backwards "schools", in a never-ending cycle of ignorance and poverty. In the end, our nation is going to be even more polarized and worse off than where we are today.