I don't understand why LLM comments lately have been using bad punctuation lately, or missing it altogether.

Signs that the bubble is about to burst. They are now claiming AGI despite being fundamentally away from it. An LLM can not ever achieve AGI. It may reasonably well PRETEND to be AGI, but at the core it's just a highly successfull letter-prediction machine. Or as a metaphor: It is "intelligence" in the sense that a toddler carefully placing one foot in front of the other while trying hard to not fall over is "running". Nope, isn't and never will be. You need a qualitative change to get there.

If they now CLAIM that they've achieved AGI, it's a sign we're nearing the end of LLM development and they're getting desperate selling the latest incremental updates. Which means the illusion is about to come crashing down.

Which is good news. Fewer data centers and lower memory prices will benefit us all.

Well, what kind of rigorous safety standards would you have in mind? Every time I've thought about it, I can't come up with a set of rules that is both well defined and effective. ("don't hurt humans" is a fine rule but it doesn't have any implementation details.)

The best I can come up with is a set of rules for after harm happens, like, "If your AI hacks into another computer, the operator goes to jail." But that's not a safety standard, it's punishment after the harm is caused.

Anyone who uses "frontier model" as a meaningful evaluation has not actually evaluated frontier LLM models.

When they break the law, arrest people. That is the easy solution.

When OpenAI "hacks" into HuggingFace, arrest people at OpenAI and charge them. This is simple and there are already laws in place for this. No need to stop actual research for a few edge-case malefactors.

In particular, once the AI context window is full, quality of output drops.

Here's the thing: I hope I'm wrong, too. But I'm honest enough with myself to recognize what's going on. No amount of sticking my head in the ground and denying it will slow it.

Surely you recognize the benefits of being the last survivor.

I've been training models for the past 6 months. I'm pretty comfortable doing that at a 200-300 level, thanks.

You're begging the question, but sure, a couple reasons:

1) Output quality is still heavily gated on the person controlling the agents, their knowledge/capability, and the environments they setup, how they're prompting them, and their constraints. Microsoft has not had competent developers for at least a decade. You think they have robust CI/CD, good unit tests for Windows, and so on?
2) Code review is also hard for AI agents. The way they're designed ('next token') does not make them as discerning of common errors as people can be. In aggregate, however, they're still better than we are, and hundreds of times faster. This is particularly true if you tell them what to look for and how to find it. It's how people have been able to find a myriad of (often, very old) security vulnerabilities. You can't have AI both suck AND be a security threat; I don't know how people can make this argument and not see the inherent contradiction.
3) Again, code review is hard for agents, as is discovery. MS code is massive legacy code going back 30-40 years in Windows and Office applications. Fixing their shit is not as straightforward as "just use AI".
4) People ARE using AI to fix Microsoft shortcomings by, largely, avoiding them outright and implementing replacements.
5) What has Microsoft done to demonstrate they're actively improving legacy projects with AI (aside from saying it)? I've not seen any releases suggesting it's true. It's just marketing, likely to get more companies/individuals to start using their products.
6) Any efforts to change Windows will be substantially stymied by legacy application support, which MS has painstakingly managed (to the point of retaining legacy bugs for decades in order to make sure those applications which depend on them continue to run).
6)

If you work on a project of any size, you're never going to be able to avoid working with programmers who are worse than you.

Essentially there appears to be a hole in understanding. Whatever dark matter may be, if it exists, it must fit the hole.

Compounded by the fact that people can write a lot like AI if they want to (consciously or unconsciously).

No, you're wrong. You're also a moron.

Why are you a moron? Because you took your opinion of something and assumed that everyone else has the same opinion. It's also a mistake a four year old might make. By the time you reach age eight, you realize other people have different opinions.

So you're acknowledging that cowboys who slap things into a production system are the problem?

Got it.

I take it you haven't used AI at all recently. It's absolutely amazing how many people haven't even got a clue how AI works and give it a false equivalency of a stackoverflow copypasta bot. Sorry, no. It hasn't been that bad for at least a year.

Nobody considered the metaverse inevitable. That's false equivalency.

The truth is that AI is inseparable from business, at this point. It's just like "cloud": an irritating buzzword with a lot of trumped up buzz, but a meaningful and appreciated advancement which isn't going away.

The neophytes just haven't caught up, yet.