Comment Re:Neither are we (Score 2) 200
I get where you're coming from, but I think we also have to accept that we have only an extremely basic understanding of how the brain works, and still don't really understand what consciousness is at all. Our present situation reminds me of the early experiments with 'evolution' where people thought you could evolve bacteria by putting some chemicals in a glass jar. If they had had access to an electron microscope to see how complex even a single cell is, they would have realised how absurd it was to believe that you could just mix some organic compounds, wait a few days and get mulit-cellular life.
The 'neural nets' we have are a weird proxy for how we understood the human brain to work in the 1980s. We know that the way neurons work is way more complicated (chemical signalling, much more interconnections, each neuron being like a city in complexity etc) than our simplistic models (simple non-linearing gating and single dimensional interconnects etc). It's actually impressive that they are as useful as they have turned out to be.
Of course we may be close to stumbling upon the secrets to AGI, but if we did it would be just blind luck. I think it's much more likely we will continue to develop a suite of algorithms that can do things that humans can do more and more efficiently, though this will be a long slog of incremental improvement.
At some point we might be able to build a robot that can essentially emulate a basic human, and I guess at that point the philosophical question is whether that machine is conscious or not if it pretends to believe it is.
I think this idea of a super intelligence is really dumb though. For all we know (and there is some evidence with autism etc), there is a inherent tradeoff between pure intellectual capability and stability in other areas of the brain. Or there might not be. But we have no idea, so just guessing that once we can create an AI it will be able to improve itself in a pretty naive position to take (though understandable if your goal is to become a mega billionaire).