Comment Drive Reward Consequences Empathy (Score 1) 482
"MIT neuroscientist Ed Boyden has a column discussing the potential dangers of building super-intelligent machines without building in some sort of motivation or drive."
These are teenage issues, no?
"Reward" is certainly essential. That could be given as knowledge, added memory, communication.
The more control these sorts of machine have over human existence, the more empathy for humanity and knowledge of consequences they need to have built it. (Letting your sixteen year old loose with the car at night is a good model.)
AI's machines as part of human entertainment, VR stories, are a different matter.