olethrosdc's Journal: Terror-journalists wreck havoc at my workplace

Imagine a hypothetical non-english TV network having footage of a certain well-known figure, but it is not certain whether the voice belongs to this figure or not. What better option, then, than to find some research group with experience in speaker verification and get them to have an 'expert opinion' on the matter. Unfortunately, things start to get out of hand when researchers sentences are taken out of quote, numbers are misinterpreted or pulled out of thin air, and when worldwide TV networks get a hold of the story second or third hand and publish completely false information.

Here is the story behind the story: there was some Bin Laden footage. 20 old tapes, verified to be the man himself. A french network also had the latest tape with BL threatening to kill and stuff and since the CIA had declared that the voice in the latter tape belonged to BL, they wanted to verify it for themsleves.

So, they approached a lot of major research institutes in france in order for them to verify the speaker's identity using standard speeker verification techniques. They all declined... but the journos kept on trying until they knocked the door of my own research institute. We agreed to look at their data and set up a standard experiment, similar to that which we use for authentication systems in banks.

When we had the results, some local journalists came and asked questions. The asnwer was this:

1) There was not enough data (only 20 examples)
2) The unknown sample was classified as an impostor, but with a very low confidence.
3) The conclusion is that it is not possible to say whether the sample voice belonged to the man or not.

Those facts were reported accurately in the local press.

Next day, the people that did those experiments were gone because they had other things to do. The french network and other, worldwide networks came to take a piece of the action but failed. Instead, they magnled stories, and the worst bit was when the BBC reported that the data corresponded to an impostor "with 95% confidence". I assume they got this number from a quote that said "in general, these methods have a 5% error", which has nothing at all to do with the confidence when looking at this particular data. They also selectively used quotes and mis-translated the interview (which was in french)...

I guess all those other institutes were wise to decline the french channel's offer of this database for testing. You never know what can happen with journalists..... argh....... death...