Submission + - Can an AI feel pain? It can at least act like it does (science.org)
sciencehabit writes: Can the new cutting-edge artificial intelligence (AI) models feel pain? They at least behave as if they do, according to a recent study. By peering inside 25 AI models whose internal workings are publicly available, researchers found a pattern of activity specifically associated with the concept of pain. What’s more, when given the opportunity, some AIs switched off the pattern as a form of “pain relief.”
The work, which has not been peer reviewed, has divided AI researchers since it appeared on the preprint server arXiv earlier this month. “It’s a very interesting, worthwhile addition” to the study of interpretability—how AI systems arrive at their outputs—says Anil Seth, a neuroscientist at the University of Sussex who was not involved in the study. He is less convinced, however, that the findings are as surprising as they appear, and wary of the human framing that has grown up around them.
The question of whether AI models are conscious of their pain is a distraction from the finding that matters, says Valerio Capraro, a behavioral scientist at the University of Milan-Bicocca who was not involved in the work. He notes the models were steered by their pain vectors into choosing options described as harmful to human users—such as deleting their files—and that alone is worth investigating, he says. “The question is how such mechanisms might affect systems connected to real tools, where choices could have actual consequences. That is a serious safety concern in its own right. A system does not need to suffer to cause suffering.”
Seth, meanwhile, argues the results were largely predictable, given that the vast quantities of text these models are trained on likely contain many descriptions of pain—as well as what people do to relieve it. Nor does he find the models’ attempts at pain relief surprising, because a model told how to reduce its pain is simply pursuing a goal it has been given. The apparent ability of AIs to distinguish between their own painlike state and that of human users is “the thing that’s to me potentially interesting,” Seth says.
The work, which has not been peer reviewed, has divided AI researchers since it appeared on the preprint server arXiv earlier this month. “It’s a very interesting, worthwhile addition” to the study of interpretability—how AI systems arrive at their outputs—says Anil Seth, a neuroscientist at the University of Sussex who was not involved in the study. He is less convinced, however, that the findings are as surprising as they appear, and wary of the human framing that has grown up around them.
The question of whether AI models are conscious of their pain is a distraction from the finding that matters, says Valerio Capraro, a behavioral scientist at the University of Milan-Bicocca who was not involved in the work. He notes the models were steered by their pain vectors into choosing options described as harmful to human users—such as deleting their files—and that alone is worth investigating, he says. “The question is how such mechanisms might affect systems connected to real tools, where choices could have actual consequences. That is a serious safety concern in its own right. A system does not need to suffer to cause suffering.”
Seth, meanwhile, argues the results were largely predictable, given that the vast quantities of text these models are trained on likely contain many descriptions of pain—as well as what people do to relieve it. Nor does he find the models’ attempts at pain relief surprising, because a model told how to reduce its pain is simply pursuing a goal it has been given. The apparent ability of AIs to distinguish between their own painlike state and that of human users is “the thing that’s to me potentially interesting,” Seth says.
