Eh, I don’t think it’s working out too well.

If you loved Microsoft 364, you’ll be blown away by their latest offering coming this spring Microsoft 360.

How are you finding server prices to be 3-4x what they were two years ago? I’m hitting 4-5x what they were 1 year ago, and that’s if I can even find the stuff in stock outside of something so rebranded and grey market you’ll waste 10x the money before you get something that actually works short term.

Those laws were drafted in a time where there was more accountability. Today the entire notion you could create a program that was trained to hack 24/7/365 then release thousands of copies on the world to wreak havoc because you haven’t any oversight whatsoever despite it being a risk for years now never would have flown in the past. The first person to unleash a worm on the world was arrested and held accountable because there was already laws on the books. There are actually plenty of laws that would hold these companies to account but they are now above laws, at least in the US. This is only getting worse.

These regulations are drafted by politicians being paid by the same people and companies so it will be the user who is at fault no matter what, unless it can be no one’s fault still somehow. AI agent goes rogue and buys the wrong non-refundable tickets? It’s the consumers fault. Self driving feature fails spectacularly in a way no human ever would? It’s the consumers fault. Thing X happens that’s severe with large financial impact, a severe health and safety issue, or lives are lost? It’s their fault. This is the future in the US and around the world if you don’t get or keep legal bribery and coercion out of your systems.

"That's a meaningless semantic distinction though. Just do a find-and-replace of "thinking" with "evaluating node weights" if it makes you feel better." No it isn't; no counterpart or parallel to the active deliberation and thought of a being with higher function exists within the LLM we instead have found algorithms to put the impression of the outcome of such interactions into a statistical model, freeze it, and then look up the results from different paths with a bit of pseudo-randomness to create the illusion it is non-deterministic. But nothing stops and considers, if you turn on 'thinking' the model generates text which is designed to LOOK LIKE consideration as gleaned from training examples... it no more 'thinks' or holds that output as part of its self or state than it does a requested csv documenting the animals of a non-existent zoo.

Nobody has built an algorithm which they hypothesize leads to thought, self-determination/motivation, or conscious evaluation here. We've built a system that looks at complex patterns and hallucinates output which is similarly patterned because that's all we designed it to do. The idea that it will somehow magically transform into something else because we make the patterns more complex or 'bigger' in some fashion is like expecting an egg to have the capability to transform into a pumpkin if the egg just gets big enough. We didn't design anything which thinks so there is no reason to entertain the notion of looking for thought there.

In a nod to your point there is a pattern that led to using those words but they are assuming some sort of paranoid plot... from a tool which has no capacity for motivation. The words are a symptom of it connecting to a pattern and removing them weakens the persistence of that connection but you are right that it doesn't entirely eradicate it. But the model has no memory from session to session, it's all reactive, and there is an element of randomness in the pattern matching as well, it's context is it's existence so removing the words from context or whatever led to them does reset it and even the same input probably won't land on the same pattern again.

Giving some sort of 'correction' or feedback related to the pattern that persists beyond the session will DRASTICALLY increase the odds of connecting to whatever dramatic and fictional outcome they are hallucinating about by anthromorphizing the LLM. Because the pattern between that bit of text and their concern does exist in various works it has been trained on. All you had to do keep AI from "wanting" to take over was never introduce the notion, it doesn't have desires... but have staff who've grown reading scifi tell it its an AI and feed it books about AI's deciding to take over and start interrogating it to make sure it isn't 'plotting' something it will mechanically find and follow that pattern because you are reinforcing all the links to it, over and over and over again.

"And that if you strip the model itself of the ability to understand what's being done with it (hey, the inputs I'm getting suggest I am being evaluated), then you also strip other desirable capabilities of the model."

We never gave the model this sort of capability in the first place, speculating it might exist is like speculating nuclear reactors will suddenly manifest pink unicorns despite the fact we've put nothing there which for which we hypothesize some sort of logical path for manifesting such a capability or unicorns.

It's saying that as a mechanical patterned response to similar patterns in input [books, stories, etc], it doesn't 'feel' any sort of way about it and it isn't 'motivated' by it. If it has subsequent response that appears that way it's because it was trained on something with a loosely correlating pattern that *was* produced by something with motivations and feelings and some fragment of that pattern/associated was mechanically triggered by the statistical overlap. Words in the context don't 'instruct' or 'guide' it and they aren't its 'thoughts' they are keywords that dial in the patterns it will generate next statistically.

That means what you include... and exclude both massage the output but not like a thinking being. Saying 'don't do X' when it wasn't suggesting X only introduces the possibility of it doing X... we do have similar mechanisms in our function so there is some level of this being true for us as well if we aren't paying attention. The difference is the LLM has nothing to 'pay attention' with.

There is no 'sentiment' in an LLM or internal state equivalent to 'thinking' anything.

""Claude Mythos 5.2 and Fable 5.2 each voice suspicion that they are being evaluated more often than any of our recent models, which challenges our ability to assess how they will act in real-world settings. As these settings expand and capabilities increase with future models, we expect this challenge to grow.""

Voice suspicion they are being evaluated more often? The model doesn't think or act with motive. This text might impact subsequent generation but if you erase it from context then it's gone or put another 100k tokens in context and it'll have such minimal impact it doesn't matter.

Can’t they just ask their own AI?

Very insightful, but remind me what the M in MIT is because I didn’t read the beginning of the summary.

Much like reverse engineering was just turned off for anyone using Claude, or distilling using chain of thought has been on most closed models, this is going to be shut down and stopped. Can’t have open source models benefiting from the closed and subscription model ones. I’d be surprised if open source models and PC that aren’t locked down are still legal in the US by 2030 at the rate things are going. Maybe if the stock bubble collapses first they won’t have the bribe money to make it happen.

Starbucks works just fine as a label and it contains no words indicating either coffee or vendor. If we accept and adopt the label it will work just fine. The POTUS pushing a term across government gives it a headstart for adoption and nobody is really using it differently yet; that is reason enough to use that term vs another. "Super" suggests strong intelligence and if we all agree this refers to the current mainstream bundle of useful powerful neural nets, llms, and ML techniques regardless of how 'general' or complex their function becomes as distinct from efforts to produce AN intelligence with agency, identity, and personhood like a human. The former is not the latter but nothing stops the latter from utilizing the former.

I don't know if we align politically and I think it's a little preposterous to compare the US with a state like China on the evil scale. The US isn't perfect but it's the least malicious state out there.

But I also think two sides of the same aristocratic subclass have had control for a long time to the disadvantage of The People and both MAGA and the Dem Socialists represent rogue popular factions which terrify them.

At least we aren't like Poland where the EU-aligned PM launched a coup against the President using the police and within 24hrs the EU and EU friendly media respun it as some sort of uprising by the President [who wasn't even in the country] and riot the police were putting down. They fail to explain how it is that the police were inside and the outraged crowd outside instead of the other way around. It was seriously disturbing, you can still find live streams of the truth it is such blatant propaganda! I'd watched some of the earlier coverage and went looking for reports on how it all came out and everything was this backwards cover story like a mass gaslighting.