Dude it's a cult. That human values, alignment, self improving crap comes directly from the AI alignment and safety cult that OpenAI and Anthropic and others are full of. It's where all the conflicting messaging comes from, all the accusations of Altman saying one thing doing another comes from those folks that are concerned he isn't drinking the right amount of koolaid. They believe AI will become a god, and only they can be trusted to build it.
OpenAI and Anthropic are full of them. Anytime you see some "AI safety researcher" quit all huffing and puffing out the door, it's one of those people causing drama. Anthropic was started by folks from OpenAI that reaaaaaly drank the koolaid, see the inviting religious leaders, the don't hurt the AI feelings stuff, the constant drop of the AI is conscious, etc.
They're fucking nuts, and any accidents that come out of an "AI safety" lab at any of these companies deserves heavy squinting. Making a point by any means and generating scary headlines is their entire goal. And yes, it does seem to conflict with the company's mission. Until you understand these companies are full of people that sincerely think the mission is only they can be trusted in the race to build AI god.
So, anyway, outside of that, I'm not saying standard internet safety stuff isn't warranted. Some auditing and reporting thresholds, know your customer, age regulation, they're worth looking at, just filter out the AI safety nuts because that whole field isn't what it looks like.