If Special Education is so expensive, then it needs to be improved. There's simply no way for a school district to be able to afford to pay $250K/year in your extreme example for each kid with special needs. At that rate, you could hire a full-time nurse, a special ed coordinator, a speech pathologist, and a couple of interns from the community college looking for job experience FOR EACH KID. Of course, schools don't operate this stupidly, they spread the cost over numerous schools and SE students. That overhead is going to remain, even if a lot of students use the voucher system. So unless you can point to some concrete figures, your example seems a bit bogus.
Here's the deal that public school fans really have a hard time coping with: the fact that people want to take their kids out of such shining, well-functioning establishments.