Online 'Sand Mouse' Tests Neurobiologists

The Metahacker writes: " A Princeton professor and his former student have created a 'mouse' (really, a neural net) that recognizes the word 'one' as spoken by a variety of speakers. The interesting part? They're challenging the neurobiology community to discover the mechanism it uses, using only the tools available to analyze live patients - observation and experimentation. You can upload your own sound files to test the mouse, and view experiments other scientists have performed. Cash prizes will be awarded to those who explain the mouse's behavior or can train the same number of neurons to perform a new task. You can read the New York Times article about it (free registration), or go directly to the site."

A new Turing test?

The most amusing possibility is that someone outside the research community may come up with the answer. As this doesn't involve building apparatus, getting a grant, publishing a paper or anything other than thinking, it's very possible an undergrad or a total amatuer will come up with the answer.

Dr. Sejnowski sounded like sour grapes when he called this an "advertising gimmick". Yeah, that's what Fermat must have been doing. Too often scientists confuse the stuff associated with the practice of science - grants, publishing, peer review, experimental proof - with science. Science is what happens in your brain when you're not doing all that other stuff...usually while taking a shower.

Re:Seriously...

I think they're saying that the -reason- that it works so much better than expected is a fairly novel reason (meaning not derivative of common neural net principles), and the process of understanding why this novel method works is best understood by treating the whole problem from a biology, rathern than compsci perspective. e.g. as you would go about trying to figure out how some organism that does something in a novel way does what it does.

It looks like there are a couple of things that differentiate Hopfield's approach from the traditional neural net approach. All NNs are biologically inspired to some degree, but so far the really common implementations (like backprop) have been simplified too much to give an accurate reflection of what really goes in a biological network.

The two big differences between this and traditional networks that I can see (based on a quick reading) is that it is using spiking neurons and neurons are given specific computational roles. Spiking neurons add up inputs over time and send out a spike to other neurons after the inputs have reached some threshold value. Inputs also decay over time, so a few inputs occuring within a couple of miliseconds of each other count for a lot more than hundreds of input spikes spread out over a number of seconds. Traditional nets add up all of the inputs at once, decide whether or not to fire, and then reset (sometimes there is a training step in there also). Since time dependence is built into spiking networks as a feature, they are very good at detecting temporal patterns.

The second difference I noticed, computational roles, means that neurons in different parts of the network may be specialized to do certain kinds of computation. One type of neuron could be used to detect patterns in a small frequency range, while other neurons detect patterns relating to which frequency ranges are currently active (I don't know if this is a realistic example, but you get the point). Traditional neural nets treat all neurons the same -- they act more like complex switches than computational units.

This kind of setup is much closer to what goes on in biological networks. Neuroscientists used to believe that neurons are much more simplistic than they have turned out to be. Individual neurons do all sorts of computations that at one time were thought to be fairly complex. Edge detection and motion detection in the visual system are examples of this. It was once thought that these tasks required collections of neurons, but it has been discovered that individual neurons can detect motion in a particular direction and pairs of neurons can detect edges.

I think there is also something interesting going on with the geometry of the network here, but I haven't quite absorbed that yet. Maybe somebody else has noticed this also and can comment (or correct me).

We've had that for ages

A neurologically simple brain for determining whether the number "1" has been achieved? Sounds like a first-poster if I ever heard of one.

My hypothesis: the mouse checks the cid# like the rest of us.

Hopfield's last theorum...

I can see it now...Hopfield and his grad student are going to die in a horrible car accident and scientists are going to spend the next three hundred years trying to figure out what he meant. An obscure professor will finally produce the answer in 2391 in a 1300 page paper that uses quantum theory, the psychology of preadolescent children and a statistical analysis performed by a 300 Exohertz computer, but only five people will actually be able to understand it.

Re:Cool, but..

The mouse was trained on only one vocal pattern and one instance of it saying 'one'. Everything else on the site is just simply ways to test what the trained mouse does when you pass it a sound sample. The fact that the training only had one input set is part of the challenge they are asking others to look at - exactly what is the network topology of the neural net, and what sort of 'objective function' did they use to train it... hopefully, they did not hard code 'one' into the neural net, so that this mouse could have been trained on any single syllable word and still gotten the same results.

But it's not perfect

The peer review process is set up to make sure that the reviewers are anonymous, and un-affiliated with the authors.

Unfortunately, there's a little more to it than that. If you return to the publisher a negative review of a paper written by a respected figure in your scientific community, there is an element of "black mark" against your name in some quarters as a result of the conflict of interest that the publisher has through needing the famous name to appear in his or her journal rather than in a competing one. As a reviewer you're anonymous to the author, but not to the publisher!

And I'm not even going to mention what happens when the journal's editorial board includes researchers interested in the same paradigm or method employed by the famous person, so that publication of that paper validates their own research area ...

Peer review is a fairly good process on the whole, but I doubt that anyone who's been involved in it [I have] would suggest it approaches perfection. :-) A dollup of cynicism is always helpful, here as in so many other areas where humans err. Yes, even in hard science.

Neural Nets and Voice Recognition

We've seen things like this before. One of the major problem with neural nets is their tendancy to specialize.. Building a system to recognize one word doesn't remotely compare to a system that can tell one from fun and done while also having the capacity to tell Bob from Rob from Cobb. The experiments posted on the site only show that the system can differentiate between 1 and the other numbers 2-9 and from various nonverbal tones. A neural net will very likely lock on to the specific differences between the sounds of these numbers. Example: A while back someone was creating a neural net to identify tanks on the ground in satellite photos. Two samples were used and the net learned to successfully differentiate them. When other samples were tried, however, the system was completely wrong. Eventually it was determined that the photo with tanks was brighter than the photo without, and that was what the system used to differentiate the photos.

Philosophically important

I think this experiment could be very important for neurobiological research and maybe other typs of research. There are many fields of science where it is possible to go on forever publishing research without any checks. Obvious areas where this goes on are fields like so-called postmodern literary criticism. But it happens in the sciences too. In behavioural evolutionary biology you can make up just-so stories in paper after paper safe in the knowledge that nobody else can rerun evolution for you and demonstrate that you are wrong. In psychology you can repeatedly perform experiments measuring correlation between this variable and that. By chance one in 20 results are 95% significant and you publish those results as if they are something other than noise. Hopfield's experiment is going to be a sanity check against this kind of work - a kind of experimental control. Here's a situation where somebody does know the answer and work can be checked. A neural net involving only a few hundred neurons. If researchers are unable to reverse engineer this then should they really have jobs supposedly reverse engineering animal or even human brains? We need to see a few more tests like this in academia. Beyond a certain point - after you've taken your last exam - academics are no longer accountable to anyone. Sure - you get peer reviewed. But what happens when you and your peers all belong to a clique that have a vested interest in promulgating a particular scientific dogma? This experiment is a wonderful way to ensure that researchers still are being tested.
--