Comment Re:The problem is "beneficial" (Score 1) 197
Think of it this way; Robot A and B -- the first one can never harm or kill you, nor would choose to with some calculation that "might" save others, and the 2nd one might and you have to determine if it calculates "greater good" with an accurate enough prediction model. Which one will you allow in your house? Which one would cause you to keep an EMP around for a rainy day?
Either is actually a potential moral monster in the making. The one might allow you to all die in a fire because it cannot predict with certainty what would happen if it walked on weakened floor boards, which might cause it to accidentally cause the taking of a life. Indeed, a robot truly programmed to never take any action which could possibly lead to the loss of life would probably never be able to take any action at all. I can't predict what the long-reaching consequences of my taking a shower in the morning might be. Maybe if I didn't create the additional demand for water the water company wouldn't hire a new employee, which means they'd be stuck at home unemployed instead of getting in the car and driving to work and getting into a fatal accident.
Sure, it is a ridiculous example. My point is just that we all make millions of unconscious risk evaluations every day and any AI would have to do the same. It requires some kind of set of principles beyond hard-coded rules like "don't kill somebody." AIs are all about emergent behavior, just as is the case with people. You can't approach programming them the way you'd design a spreadsheet.