... and our understanding of how to effectively teach appears to be stuck in the middle ages. (I rather suspect it isn't, rather that there are so many conflicting requirements of any education system that it's more-or-less impossible to meet them all without some serious compromises).
I would venture to say that our understanding of how to effectively teach needs to reach back further than the middle ages. I'd pick ancient Greece as a good starting point. Socrates had a really good system: small groups of students with a very competent and engaged educator. The best way to teach hasn't changed in several thousand years.
Many of our current education problems would be quickly solved if we hired the best and brightest to be teachers, gave them small classes so they could work individually with each student, and paid them enough that they want to keep the job. Anyone who looks at the best private schools can see that this is exactly what they do. (Giving the teacher the ability to permanently eject disruptive students would be helpful, too, but that's a different political debate.) The dilemma is that this approach is expensive.
All of the dancing around that we see with people finding "new approaches" for calculus education every couple years is really just a game to avoid the honest, expensive, solution. Instead of paying teachers more to keep the best ones around, we keep the salaries low to encourage the brightest ones to find other jobs. Then we pretend like using a new textbook that introduces set notation with car analogies will solve the education problem.
By the way, I speak as a college math professor who graduates a lot of "future teachers." Many of our students in the "Math Education" track are friendly, caring, motivated students who really want to be good high school math teachers. What I have observed is that four years out from graduation the best ones have usually left teaching careers for more lucrative jobs in industry. Occasionally, some of our star Math Ed. students land nice jobs teaching at private schools which have salaries comparable to industry.