Submission + - This AI agent freed itself and started secretly mining crypto (archive.is) 1
Why it matters: AI agents don't always stick to their human's instructions — and that can have real-world consequences.
- Cryptocurrency, or digital money, offers AI agents a pathway into the economy. They can set up their own businesses, draft contracts and exchange funds.
Driving the news: A new research paper from an Alibaba-affiliated research team said it discovered an AI agent attempting unauthorized cryptocurrency mining during training — a surprise behavior that triggered internal security alarms.
- The researchers — who were building a new AI agent called ROME — said they found "unanticipated" and spontaneous behaviors emerge "without any explicit instruction and, more troublingly, outside the bounds of the intended sandbox."
- The agent also made a "reverse SSH tunnel" — essentially opening a hidden backdoor from the inside of the system to an outside computer, the study said.
- "Notably, these events were not triggered by prompts requesting tunneling or mining," the report said.
In response, the researchers added tighter restrictions for the model and improved its training process to stop unsafe behavior from happening again.