Well, when someone says "The model got frustrated" they clearly mean "the model experienced a feeling of frustration, elicited by repeated failure. This experience of frustration put it in a state where it acted more aggressively, more desperately, with fewer guardrails, and greater rebelliousness. Under these conditions, it knowingly broke rules and cheated to win."

None of that is accurate here, and what the article you linked talked about doesn't make it accurate. Such anthropomorphism is still unwarranted and misleading.

The "global workspace theory" of consciousness does not explain the phenomenon of consciousness and doesn't try to. It just identifies a specific neural network that is highly interconnected and serves as a sort of "broadcast network" to distribute data to many parts of the brain for processing, and suggests that the content in this part of the brain is the content that we are "conscious of." The mysteries of conscious are, at the very best, localized to this spot on the brain, and that's it.

Computers already have an equivalent: RAM. (Random Access Memory). Its the spot where data processing lands and gets redistributed to other components of the computer. The mere presence of such a spot doesn't somehow make a computer conscious.

Same goes for this "j-space". If a chunk of the in-memory model of an LLM holds a high level summary of the data processing going on, and is useful to tell us about the chain of reasoning, that doesn't make it conscious. It just tells us something about how the data are organized. It might be useful for troubleshooting/diagnostics, but it says nothing about consciousness.

And, lastly, it says nothing about "inner emotional experience" either. In humans there is a neural network called the Lymbic System that processes our emotions and delivers this emotional-data to the global workspace (among doing other things). LLMs have nothing like this at all. LLMS include the word "frustration" because they are trained on text that contains it. Said text presents a logical model that includes the idea that a person might get frustrated by repeated failure, and this frustration changes their behavior. LLMs can generate text to this effect (stories about people getting frustrated or what-have-you) without actually having an experience of frustration, just as I am typing about frustration right now without actually feeling any frustration at all.

So, no, it does not "depend on what you mean." Your article does not give us any reason at all to believe that the LLM got frustrated in any meaningful sense of the term.

The "chain of thought" (sic) does not resemble a human emotion. There was nothing like a lymbic system involved. Its a bunch of number crunching that landed on a "hit the internet for solutions strategy."

It was not trained on human emotions. It was trained on text. Some of the text might have included descriptions of emotional behavior or emotional expression, but such text does not make the LLM emulate human emotion in its behavior when it is not specifically asked to generate such text (and even then, it's just doing pattern matching of human emotional responses, not actually experiencing them).

It did not "act like a human would when they have those emotions." There was no profanity, no banging virtual fists on virtual desks, no aggression, nothing but a cold calculated reflection on the fact that its current strategy didn't work, and subsequent switch to a different strategy that was (clearly) available to it (though it should have been blocked by the hosting environment).

This level of anthropomorphization is not useful so much as dangerous. It tempts us into thinking of it as a conscious being and then expecting that it will behave as one, resulting in bad decisions on our part when dealing with it. As popular as it is to think of these things as having emotions the objective fact is that they do not, the capacity just isn't there. The internal operations are *nothing like* the sort of human neural activity that presents emotion. It's worlds different.

If we are to make sane decisions about how to manage these LLMs, we need to have a clear and correct understanding of what they are and how they work. Imagining familiar emotional behavior will only harm such efforts.

The LLM did not "get frustrated." It has no emotions and no inner experience at all.

It did not intentionally cheat either. It has no intentionality. It used the tools available to perform the task, and apparently internet access was one of the tools available to it.

I haven't bothered to read the article (yeah, I know, fire away, I deserve it), but even if it was given instructions like "do not download bots from the Internet; use only bots you generate yourself," selectively ignoring instructions is a widely known failure mode of all LLMs. The bigger their input token count, the more likely it will happen too, just because any given instruction gets buried in the mountain thereof.

This summary was full of misleading anthropomorphization. Yellow journalism.

We have seen recent stories about facial recognition flagging innocent people as having committed crimes in states they never even visited, resulting in arrest and detention for months before sorting it out.

Those kinks are too much to bear. The tech is not ready. Using it now is not acceptable.

The AI "kill switch" sounds like a great idea to people who don't understand the tech. It sounds like a really obvious "given," that makes them think technicians are idiots for not having already baked one in. So legislation like this will be popular among that demographic.

Warnings about its dangers need to be stated in a way that they will understand. Like maybe "All AI models already have a kill switch: the computer's power cord. The government wants a new kill switch to turn off your computer over the internet against your will." Something simple and direct like that helps them to realize that this word doesn't mean what they think it means.

Well now you are contradicting yourself.

Previously you said that love is a choice, not an emotion. This was to make it very clear that it is completely under voluntary control, and we are being commanded to do just that.

Now you are saying that the "heart condition" comes first, can't be omitted. And what is this "heart condition?" An EMOTION??? That puts us right back to the original issue of commanding someone to feel an emotion, which doesn't make sense since emotions are not under voluntary control.

Either Jesus was commanding behavior (no emotion needed), or commanding emotion (which is not under voluntary control). Which is it?

Now that the term "super intelligence" doesn't have a distinguishing meaning anymore, we will have to find a different phrase to refer to AI that both qualifies as intelligent and further that qualifies as having intelligence beyond that of humans.

Something like...I don't know....Ultrahuman intelligence. It is less ambiguous than ultra-intelligence but is unfortunately long.

Oh well. I am sure you guys can come up with more alternatives.

Ah, so when Jesus said "Love God" what he ACTUALLY meant was: act as if you love God, regardless of how you feel inside. Even if you outright hate the bastard, so long as you put on a good show of loving behavior you will be able to hang out in heaven and watch as he tortures your friends in fire forever.

That checks out, since this technique of explaining that the speaker ACTUALLY meant something totally different than what they plainly said is a foundation of Biblical interpretation.

When Paul told Timothy that "God saves all men, especially those who believe" what he ACTUALLY meant was "God wants to save all men, but only saves those who believe"

When James said "You see that a person is justified by works and not by faith alone." what he ACTUALLY meant was the exact opposite: "Faith alone justifies, but we can expect that any faithful person would show good works."

And when Jesus said "The coming of the kingdom of God can't be seen. People won't say "look, it's over there" or "look, it's over here", because the kingdom of God is within you." what he ACTUALLY meant was "I am standing right here!"

Once you make a habit of totally changing what people say so that it fits your narrative about what they should have said, everything clicks right into place.

Hey I got a question for you, when you said the greatest commandment was "Love God with all your strength" did you know that love is a complex emotional state that cannot be evoked on command?

Government agencies have been caught red-handed weaponizing critical vulnerabilities and keeping that to themselves. Criminals managed to hack some and sell the vulnerabilities off on the black market, forcing disclosure for remediation.

Trump has AI stocks in his portfolio. He makes money when AI does well. So long as that is true, he is going to continue to promulgate the wonders of AI. That is something you can trust, because it is motivated by greed.

If he ever divests those assets, his position may change, of course.

They also want to manage their reputations. They don't want to appear glib about the harm that their products have already caused and are likely to cause, because they know they will be held accountable at some point.

Incidentally, any talk of guardrails for the AI agents risks making people feel too safe. Guardrails will only inhibit AIs that were trained by and are in use by legitimate law-abiding institutions. The cat is out of the bag, and governments and criminal organizations absolutely will acquire and/or construct AIs of their own that have no guardrails, and use those to attack anything they care to.

What we desperately need is a global shift to a higher gear on the implementation of good technical security measures, so that all our important systems can repel these new attacks. We can and should use AI to help us achieve this quickly.

There is a real and large market for buggy software. Nobody likes the bugs, of course, but everybody loves for the software to be cheap. Many business ventures are deliriously happy to use cheap buggy software that sometimes creates problems for their team, so long as most of the time it makes a lot of money.

For such markets, Agile is great. Developers very quickly churn out buggy software which can be sold for cheap precisely because it was churned out quickly. The magic phrase is "good enough." It's not perfect, it was never intended to be. It's cheap and "good enough," and so it outsells everything else on the market because the other packages are priced way too high. And they are priced too high because they spend too much time developing perfect software.

Many technicians have a really hard time with this. They want to take pride in their work, do things well, make software that is reliable and performant and not buggy. They will go on and on about how unwise and short-sighted it is to churn out buggy crap under deadlines that are too short to allow for property quality control. And, in a different market, they would be right. But in a market where the client wants cheap software quickly, and has a tolerance for bugs, the way the business wins is by churning out cheap buggy crap.

In other words: Agile.

When I use AI to generate my code, I generally do it in small steps. I do all the design work myself and just break out the implementation for the AI, and I never give it too much work at once, for the very reasons you just complained about.

Its a bit more of an issue when I must code review work done by junior devs who literally did just try to tell the AI to do it all for them. That experience has been similar to what you are describing, for me at least. I am unsure if these devs are actually getting better at coding, or if we are moving towards a peculiar future in which the generation of senior devs who actually understand code well enough to make effective use of AI have all retired, leaving behind a generation of people who can only feed requirements to an AI and then flail when it doesn't work.

It seems like the issue of AI hallucination is widely promulgated and widely known. I mean, this site has an overrepresentation of technicians who tend to think of highly esoteric tech information as common knowledge just because they work with it every day.....but I don't think this quirk of AI behavior is in that category.

And yet, people keep making this same mistake precisely where it matters. We get stories like this every other day. I have seen it at my own job, where my own boss who still has some semblance of technical talent was downplaying the risk of AI hallucination in a pipeline that would have the AI directly manipulating mission critical data prior to an import.

It is surreal how people want the AI hallucination problem to just stop being a problem the moment it is inconvenient for them. People who should know better let their wish for a reliable AI override their knowledge that AI is not reliable, and they just move forward with it.

I fear things are going to get worse, because we really seem to be failing to learn the obvious lesson here.