Here's the article.

http://www.nytimes.com/2011/03/07/education/07winerip.html
On Education
Evaluating New York Teachers, Perhaps the Numbers Do Lie
By MICHAEL WINERIP
Published: March 6, 2011

I was recalling it from memory. Actually, the Department of Education accountability formula ranked her as 7th percentile, which prevents her from being rehired at that school and probably any New York City school. The confidence interval 0-52nd percentile. Yes, P=95%. She was there 2 1/2 years, and two dozen of her students got into Bronx Science and Stuyvesant.

You can't just rank teachers based on their students' test scores. You have to correct for the students' abilities. (1) Everybody who has studied student achievement (Diane Ravitch for example) agrees that the factor that is most strongly correlated with student test scores is family income. So you have to correct for family income. But how do you do that? Parents aren't required to reveal their income to the school. So they have to use indirect methods of estimating income, which are inaccurate. (2) New York City is trying to correct for the students' past performance, to see how much the teacher improves their scores. But this has a bias against the best students, because if your class has students who already have a 98% average, they don't have any room to go up. So they have to correct for these and many other factors.

They wound up with the complicated formula in the article that tries to incorporate all these factors. The problem is that (1) the formula hasn't been validated and (2) literally nobody understands it.

The fundamental problem is that it's a bad formula that doesn't correlate with teacher ability. In this case, the teacher is obviously qualified, her students do well, her principal loves her, and yet the formula says she ranks at the bottom. How do you know this formula works?

The fact that it ranks this teacher from the zeroth to 52nd percentile demonstrates that the formula doesn't work. Even if you believe the test, the only statistically valid information it gives you is that she's either among the worst teacher or among the top half. The conclusion that she's in the 7th percentile is not statistically valid. You have to understand basic statistics to realize what's going on here. If you have a confidence interval of 0-52, you can't take one point of the distribution.

Get back to me when you've read the article.

If you read TFA http://www.dissentmagazine.org/article/?article=3781 you'll see what the problem is. These billionaires aren't simply giving money to local innovative schools. They're using their money in a heavy-handed way to shape policy for the public schools used by less-affluent, less-powerful taxpayers.

They promote bad policy, and when the evaluations show they don't work, they ignore the evaluations and keep promoting the bad policy -- in public schools as well.

They've affected federal policy, under No Child Left Behind, so that schools can't get this federal money unless they accept the whole package of "reforms," many of which turned out not to work, such as destroying large neighborhood schools and replacing them with small schools, high stakes testing, and promoting charter schools (which according to major nationwide evaluations http://nces.ed.gov/nationsreportcard/studies/charter/ using the tests they love so much, are worse on average than public schools).

I read that article.

I'm not going to respond to a comment that calls me an asshole.

TFA http://www.dissentmagazine.org/article/?article=3781 doesn't have a problem with billionaires helping poor kids get an education.

The problem is that billionaires don't free up resources for public schools. As a result of their lobbying, public schools actually lose money, for example through penalties under No Child Left Behind, if they don't implement these "reforms" many of which have been proven not to work.

As TFA says, the billionaires are trying to change the public schools that only get tax money. They're doing it by using their money not so much for direct teaching but for setting up "think tanks", lobbying, and even paying public school educators in ways that might be considered bribery.

My biggest problem is that they're implementing fad solutions, like charter schools, financial bonuses, etc. that haven't been shown to work -- and have sometimes been shown not to work.

For example, some charter schools have been evaluated by the NAEP http://nces.ed.gov/nationsreportcard/studies/charter/ in rigorous studies, and overall they did worse than public schools.

Tell me -- if the evidence demonstrates that something doesn't work, why would you want to roll it out across the whole school system?

These "neutral" tests are also invalid tests. As TFA http://www.dissentmagazine.org/article/?article=3781 mentioned in passing, the National Academy of Sciences and other organizations reviewed the current tests and found out that they don't do what they're supposed to do: They don't tell you whether a teacher is good or bad.

The New York City department of education was using a test to determine whether new teachers would continue on the job. The test had a complicated formula that (literally) no one could understand, to try to correct for things like the students' family income and previous test scores.

The test told one middle school teacher that she was in the bottom 6%, and had to be fired. Her principal didn't believe it, and didn't want her to be fired, because she was a good teacher, her students got into the competitive high schools, etc. But that 6% had a confidence interval -- from 0% to 51%. So actually, she was either among the worst teachers (0%), or among the best half. If you don't know what a confidence interval is, there's no point in my talking to you, but that means the test results are statistically invalid. You might as well fire teachers by throwing dice.

These tests are made under contract by testing companies, and they haven't been tested for validity. There's a huge amount of research on this. They can't distinguish between the effects of family income and effective teaching. It's not common sense to fire teachers based on these tests.

That's what TFA is all about. http://www.dissentmagazine.org/article/?article=3781

Everybody knows that the one factor that is most strongly associated with grades is family income. If you teach low-income kids, they're going to start with low grades and their grades will rise slowly. Test scores reward teachers for having high-income students.

You make a good point. It's stupid to fire "bad" teachers. If your tests are so good at identifying good teachers, find out why they're good, and teach their techniques to other teachers.

If a teacher is hopelessly incompetent, I don't defend keeping them on, but that's not what Rhee was doing. She was just a right-wing hero who wanted to destroy the union and attack teachers.

Read the USA Today story about Rhee (or the Wikipedia article).

Rhee gave large bonuses to certain principals and teachers. Then it turned out that there was massive cheating on the standardized tests that Rhee used to evaluate them. The teachers erased the incorrect answers and filled in the correct answers. The companies that mark the test sheets can read the test answers, but their machines can also read erased answers, because an unusually high number of erasures indicates cheating. That's what happened in DC. When the company warned Rhee about this, she refused to investigate it and covered it up.

Everybody really ought to read TFA. It explains why all of Rhee's reforms, including the performance bonus, have been tried before and didn't work. http://www.dissentmagazine.org/article/?article=3781 I admit it's not easy reading, because it has lots of facts and data, based on actual research and the experience that people had when they tried these things out and looked at whether they worked.

That's exactly the opposite of what the article said. Did you actually read the article you are linking to? If so, then you're deliberately misrepresenting it.

The actual headline is:

"Yes, 47% of Households Owe No Taxes. Look Closer."

The article says that's true only if you define "taxes" to exclude payroll taxes. It says:

"About three-quarters of households pay more in payroll taxes than in income taxes."

I really get pissed off when people try to have an intelligent, informed conversation and you have to spend 15 minutes checking the conservative sources and have their facts turn out to be wrong. Deliberately distorting facts is the worst thing you can do, IMO. Negligently distorting facts is a pretty close second.

It's a waste of time to try to have an intelligent debate with conservatives. The time is better spent reading Paul Krugman http://www.playboy.com/magazine/playboy-interview-paul-krugman and going to Occupy Wall Street to figure out how to organize politically to stop them from destroying the country.

People tell me that I romanticize the UK health care system too much (probably from reading BMJ and Lancet in my younger days), but my understanding was that the NHS gives a lot more emphasis to systematic evaluations than we do in US.

I thought that when they rolled out their health care software, they did a lot of careful testing and evaluation, compared to what we did in the US. True?

Because IT evangelists think that every problem in the world can be reduced to computer code, and they create medical systems without understanding how medical practice works.

Then the doctors try the system out, it runs into problems ("It takes me longer to enter a prescription into this computer than it does to write it on a prescription blank by hand"), and they correctly go back to the older manual systems that work better.

Doctors aren't stupid. They use lots of new technology every day. When something works, they use it. When it doesn't work, they drop it.

Medical practice is very complicated. The potential cost of error is very high. The cost of developing electronic medical records properly is very high. It's like developing a medical drug or device (which it is). You have to get back to your users (the doctors) continuously, find out what's going wrong, and fix it. Automating medical practice is a massive job, like automating the aerospace industry. At best, it will be slow and expensive. At worst, developers will take shortcuts, waste even more time and money, and have to scrap it all and start over.