Comment Re:There are ideas. Here's one. (Score 1) 230
Agreed - the early days of neural nets surprised a lot of people with their ability to "learn" optical character recognition. Then they kind of fizzed out. Numenta is doing some interesting stuff with video processing, but it's a slow grinding progress, not an explosive revolution like Moore's Law. The "nets" can be massive for low cost now, but there's apparently not much (orders of magnitude) more to be gotten from them.
Not that this particular theory is going anywhere, but a fun one I've heard is a sort of "survival of the fittest pattern replication" where an "event" is encoded as a repeating pattern of discharges. Multiple (hundreds even thousands) of groups of neurons take up variations of the theme and the dominant pattern(s) are the ones passed on to the next level of processing, where a similar process occurs - transforming the previous levels' patterns into new patterns on the current level. It's a departure from the resource starved ideas of early digital processing where you have one fragile chance to get the right answer processed through the system. Instead, hordes of parallel processing units reach consensus, with lots of potential outcomes considered and discarded in favor of the eventual result.
Some kind of departure, as different as "fittest patterns" is from AND and OR gates, is likely to be the next big step.