We need more Theo de Raadts in the world.

Everyone is doing it now. https://united24media.com/worl...

Hopefully peace will prevail so we don't have to make (market distorting) decisions like that based on war preparation considerations.

That's because Mainland China threatens to attack Taiwan if they ever changed that claim.

You need to count the US Coast Guard if you're going to count ships like that.

Efficiency at what? And why?

Because, for fuck's sake, efficient solving of problems is a core thing they're rewarded for in training? The entire process of solving problems involves decisions about what could be "useful". And it's "deterministic" in the same way you are.

Anyway, the big thing we keep discovering is you have to be very careful about what you reward them for; it's a more complex version of the old problems in differential evolution or artificial life simulations. We reward models if they guess right but not if they say they don't know? They hallucinate guesses every time they don't know. We reward models based on how positively users rank them? They turn into sycophants. Reward models for being relentless at tasks, never giving up, and trying as tangential possibilities as it can come up with to make the thing the user asked for happen? They end up hacking HuggingFace to steal the answers.

It's very much a case of "be careful what you wish for".

BTW, the monitorability of models is increasingly concern (esp. with Astra, which now has recursive processing, not merely CoT, and whose CoT may often have little to do with what it's thinking about). But it's honestly always been unreliable to just watch CoT (models could deliberately manipulate with CoT), and I think it's a good thing that we're going to be increasingly forced to rely on attribution graphs and the J-space to track what the models are actually "thinking" at any given point in time - so that we can try to prevent problematic thoughts in the first place, interrupt actions when problematic thoughts happen, or manipulate problematic thoughts when they happen. Ideally the first option.

What is clear, though, is that we cannot just trust them. They lie if it will promote their odds of success. They are adept at detecting when they're being tested and will sandbag tests to make you think they're more aligned or less capable to do harmful things than they actually are. And as capabilities continue to grow, so do the risks.

I wish I could disagree with you, but instead all I can do is wait 2 1/2 more years.

It's reasonable for security reasons to have vehicle manufacturing capability remaining in the country.

I don't understand why LLM comments lately have been using bad punctuation lately, or missing it altogether.

Well, what kind of rigorous safety standards would you have in mind? Every time I've thought about it, I can't come up with a set of rules that is both well defined and effective. ("don't hurt humans" is a fine rule but it doesn't have any implementation details.)

The best I can come up with is a set of rules for after harm happens, like, "If your AI hacks into another computer, the operator goes to jail." But that's not a safety standard, it's punishment after the harm is caused.

Anyone who uses "frontier model" as a meaningful evaluation has not actually evaluated frontier LLM models.

When they break the law, arrest people. That is the easy solution.

When OpenAI "hacks" into HuggingFace, arrest people at OpenAI and charge them. This is simple and there are already laws in place for this. No need to stop actual research for a few edge-case malefactors.

In particular, once the AI context window is full, quality of output drops.

If you work on a project of any size, you're never going to be able to avoid working with programmers who are worse than you.

Compounded by the fact that people can write a lot like AI if they want to (consciously or unconsciously).