Teen Plays Videogame With Brain Signals

SkyFire360 writes "A team of ECoG (ElectroCorticography) researchers from Washington University in St. Louis successfully wired a young man's brain up to a computer and began reading the neurological firings in his brain. After analyzing the action potentials created when a neuron fires, they were able to get two-dimensional control of a cursor. Taking the research one step further, they decided to connect an old Atari 2600 to the signal processing computer to see if the young man could control the videogame system."

Action Potentials?

[ThePopeLayton]

After analyzing the action potentials created when a neuron fires, they were able to get two-dimensional control of a cursor.

EEG, does not read action potentials, rather it reads synaptic input into the cortex not output from the cortex. The news article has this backwards.

Re:Did they figure it out, or did he?

[superpenguin]

TFA describes a bit of a learning curve for the kid, so I'd say you're right. But the point is that the interface is feasible. I imagine anyone getting a new robotic limb would have to learn to use it, just like you have to learn to use your own limbs. The neat thing is that he was able to learn to create these signals very quickly with some degree of fine control. THe human brain is actually a very adaptable thing, even for older folks, as evidenced by the psychologist (I think) who had special glasses that flipped everything upside-down and after a few days was able to function quite normally. So as long as the interface works, it should be entirely possible for most people to learn to use it with relative ease.

Re:So what.

[omeomi]

The same sort of thing is already being used for ADHD and depression therapy, as well:

http://technology.timesonline.co.uk/article/0,,204 09-2379616,00.html [timesonline.co.uk]

Re:Uh... isn't that ONE dimensional control?

[MankyD]

One dimensional in a purely mathematical sense, yes. However, to the mind, moving left and moving right are two separate actions. In that sense, you might be able to get away with calling it 2D (not to mention the fact that they also added the ability to fire - a 3rd action.)

Re:Epilepsy?

[Daemonstar]

the boy who had the grids implanted so that neurologists and neurosurgeons can find the area in the brain serving as the focus for an epileptic seizure, with hopes of removing it to avoid future seizures

Because that's the whole reason why he has "grids" inside of his head. No "grids", no "mind control", apparently. :)

Re:So what.

[SkyFire360]

Hi. I'm SkyFire360, I wrote that program. I'm the guy in the blue shirt.

Though we're the first lab to use the ECoG technology, even our resolution is too poor to accurately control things in more than two dimensions. A breakdown of the different resolutions of Brain-Computer-Interfacing is here [imageshack.us]. The problem with EEG is that the skull acts as a signal damper that disperses and blurs the electromagnetic waves created by the neurons. Though we can still detect the waves created, it becomes increasingly more difficult to discern what area of the brain created these waves, much less what neuron(s) did.

A breakdown of the different types of BCI currently being developed and researched:

• EEG - Electro-Encephalograph - Biggest advantage is that anyone can use it, as it can be worn like a helmet or a headband. Though because it is non-invasive, it has extremely poor resolution
• ECoG - Electro-Cortocography - Though it needs to be implanted inside the skull, it produces fairly good resolution. Also, because it only sits on top of the brain as opposed to inside gray matter, it has much less of a chance to form scar-tissue (though still greater than zero). Tough to get more than one dimension of control.
• Single Receptor - A microscopic electrode is placed directly in contact with a specific neuron or group of neurons. This allows researchers to directly measure the potential of one neuron firing. Of course, this requires the electrode to be implanted. This form of BCI is also very prone to scar-tissue buildup, causing the signal to become weaker and possibly lost as the body reacts to a foreign object in the brain.
• Light Reactive Imaging - Still very theoretical - A laser is trained on a single neuron and its reflectance is picked up by a separate sensor. When the neuron fires, the laser light pattern and wavelengths that are reflected change slightly. This allows researchers to monitor a single neuron while leaving the tissue "untouched", negating the issue of scar-tissue buildup. However, this technology is not able to penetrate the skull yet, as would be needed for external/non-invasive BCI

More information about BCI and ECoG can be found in a presentation [wustl.edu] from a WashU professor... actually, he's the guy standing behind the computer. Check pages 9-11 for some good slides

Though keep an eye out for us at BMES... we just found coding for direction and velocity, and it is scalar. :D Oh yeah, anyone have any questions?

Re:Connection?

[SkyFire360]

Anyone who's doing BME research runs into ethical issues, and our lab is no exception. Ethically we cannot implant electrodes into a perfectly healthy human in vitro. The risks of risks far outweigh the benefits. However if we can help epileptic patients while doing research, we can aid in the healing of the patient while getting data at the same time.

Re:Did they figure it out, or did he?

[SkyFire360]

Hi!

We used the program BCI2000 for this task. This program allows us to sample 16 specific electrodes at a rate of 1200 hz, attaining frequencies up to 600hz. By reading data from calibration tests, we can then select the best electrodes that have the highest r^2 value for use with controlling it. I believe we're currently using a form of ICA for the signal analisys and we may move to something mroe complicated in the near future, but I'm the programmer on the team and not the electrical engineer. :)

minor point of accuracy

[weisen]

This is not measuring the action potential caused by the firing of a single neuron, as the writeup seems to indicate. This is measuring the aggregate signal from tens of thousands of neurons. The typical recording grid is a flexible material (silicone?) with a grid of circular electrodes about 5mm in diameter. The surgeon can place the grid on the cortical surface or cut off a strip and push it into a sulcus. Clearly not a single-cell recording.

Not a good solution.

[raehl]

The computing power of a Beowulf Cluster of teens, where T is the power of one teen and n is the number of teens in the network, is T^(1/n).

Even worse, if you connect your cluster to the internet, the effective computing power becomes T^(1/n)/B, where B is the bandwidth of the connection.

There is a special exception to this, however, that takes into account the Mischief Coefficient. For any problem, P, with a fractional mischief component of M, the expected power becomes T^(1/(1-M)n).

As we can see, for any problem with a Mischief Component of 1, the power of the cluster becomes infinite. In fact, using my Beowulf Cluster of Teens, I was able to determine that the more teens you have, the more infinite their power gets. For example, according to my BCoT, if you have 100 teens, your cluster would be 10 times more infinitely powerful than an infinitely powerful cluster of only 10 teens.

Re:So what.

[Anonymous Coward]

I agree, this article is really old news. I worked on an undergraduate project two years ago as a freshman at the Johns Hopkins Biomedical Engineering department that aimed to use EEG signals as a form of non-invasive computer input device (EEG Keyboard). At that time, we had already seen a lot of demonstrations where games were played using implanted electrodes. They were far more impressive than this game.

By the way, using EEG's is definitely possible. We managed to get usable signals from only 8 (full EEG setups usually have 256, 512, etc) EEG channels by using independent component analysis and principal component analysis to filter out the skull's dampening effect and the much stronger electrical signals that come from muscle movement (e.g. blinking). This allowed the user to type characters displayed on screen. Oh yea, I had to wear a tinfoil hat to help eliminate E-M interference and alien brain control waves from contaminating the data stream.

Now, playing a game like this is much easier than typing. We adapted our program to allow users to play pong WITHOUT getting an electrode implanted in the brain by measuring singals from the motor imagery part of the brain, which is much simpler than the ERP's and VEP's we were dealing with for the keyboard. I assume they also used the motor imagery part of the brain. That's why you see the guy moving his fingers (he's not supposed to).

The idea is, your imagine yourself moving a body part. The algorithm picks this up and moves the cursor. Now, your brain notices the correlation between the imagined movement and the movement on screen, and through brain plasticity, learns to associate the movement with some pattern of neurons firing. The signals actually INCREASE in strength as the user becomes more experienced with the device. In the end, you can control the cursor almost like it's a phantom limb.