Comment Re:Funny, that spin... (Score 2) 421
There are a lot of things in life that are binary. You are either hit by the train, or you aren't. Someone getting "kinda" hit by a train is very, very unlikely.
As a train is to muscular power, ASI is to intellect. This is like dodos debating the impact of the arrival of sentient bipeds. Either it will be really good for them, in that they get their lot in life improved by going to zoos or homes around the world as pets, or they will all get killed and eaten/have their habitat destroyed and die out. Not much room for in betweens.
The problem here is that humans tend to think linearly, and, well, like humans. The problem is that this threat is exponential if not geometric, and its values are completely unknown, and highly unlikely to be anthropomorphic (ie a human wouldn't think that the best way to catch a dog would include chopping up your own parents/creators for bait). AGI will likely be completely alien, and WITHOUT PRECEDENT. This is nearly as dangerous as creating a new vacuum state.
As a train is to muscular power, ASI is to intellect. This is like dodos debating the impact of the arrival of sentient bipeds. Either it will be really good for them, in that they get their lot in life improved by going to zoos or homes around the world as pets, or they will all get killed and eaten/have their habitat destroyed and die out. Not much room for in betweens.
The problem here is that humans tend to think linearly, and, well, like humans. The problem is that this threat is exponential if not geometric, and its values are completely unknown, and highly unlikely to be anthropomorphic (ie a human wouldn't think that the best way to catch a dog would include chopping up your own parents/creators for bait). AGI will likely be completely alien, and WITHOUT PRECEDENT. This is nearly as dangerous as creating a new vacuum state.