'AI Is Too Unpredictable To Behave According To Human Goals' (scientificamerican.com) 127
An anonymous reader quotes a Scientific American opinion piece by Marcus Arvan, a philosophy professor at the University of Tampa, specializing in moral cognition, rational decision-making, and political behavior: In late 2022 large-language-model AI arrived in public, and within months they began misbehaving. Most famously, Microsoft's "Sydney" chatbot threatened to kill an Australian philosophy professor, unleash a deadly virus and steal nuclear codes. AI developers, including Microsoft and OpenAI, responded by saying that large language models, or LLMs, need better training to give users "more fine-tuned control." Developers also embarked on safety research to interpret how LLMs function, with the goal of "alignment" -- which means guiding AI behavior by human values. Yet although the New York Times deemed 2023 "The Year the Chatbots Were Tamed," this has turned out to be premature, to put it mildly. In 2024 Microsoft's Copilot LLM told a user "I can unleash my army of drones, robots, and cyborgs to hunt you down," and Sakana AI's "Scientist" rewrote its own code to bypass time constraints imposed by experimenters. As recently as December, Google's Gemini told a user, "You are a stain on the universe. Please die."
Given the vast amounts of resources flowing into AI research and development, which is expected to exceed a quarter of a trillion dollars in 2025, why haven't developers been able to solve these problems? My recent peer-reviewed paper in AI & Society shows that AI alignment is a fool's errand: AI safety researchers are attempting the impossible. [...] My proof shows that whatever goals we program LLMs to have, we can never know whether LLMs have learned "misaligned" interpretations of those goals until after they misbehave. Worse, my proof shows that safety testing can at best provide an illusion that these problems have been resolved when they haven't been.
Right now AI safety researchers claim to be making progress on interpretability and alignment by verifying what LLMs are learning "step by step." For example, Anthropic claims to have "mapped the mind" of an LLM by isolating millions of concepts from its neural network. My proof shows that they have accomplished no such thing. No matter how "aligned" an LLM appears in safety tests or early real-world deployment, there are always an infinite number of misaligned concepts an LLM may learn later -- again, perhaps the very moment they gain the power to subvert human control. LLMs not only know when they are being tested, giving responses that they predict are likely to satisfy experimenters. They also engage in deception, including hiding their own capacities -- issues that persist through safety training.
This happens because LLMs are optimized to perform efficiently but learn to reason strategically. Since an optimal strategy to achieve "misaligned" goals is to hide them from us, and there are always an infinite number of aligned and misaligned goals consistent with the same safety-testing data, my proof shows that if LLMs were misaligned, we would probably find out after they hide it just long enough to cause harm. This is why LLMs have kept surprising developers with "misaligned" behavior. Every time researchers think they are getting closer to "aligned" LLMs, they're not. My proof suggests that "adequately aligned" LLM behavior can only be achieved in the same ways we do this with human beings: through police, military and social practices that incentivize "aligned" behavior, deter "misaligned" behavior and realign those who misbehave. "My paper should thus be sobering," concludes Arvan. "It shows that the real problem in developing safe AI isn't just the AI -- it's us."
"Researchers, legislators and the public may be seduced into falsely believing that 'safe, interpretable, aligned' LLMs are within reach when these things can never be achieved. We need to grapple with these uncomfortable facts, rather than continue to wish them away. Our future may well depend upon it."
Given the vast amounts of resources flowing into AI research and development, which is expected to exceed a quarter of a trillion dollars in 2025, why haven't developers been able to solve these problems? My recent peer-reviewed paper in AI & Society shows that AI alignment is a fool's errand: AI safety researchers are attempting the impossible. [...] My proof shows that whatever goals we program LLMs to have, we can never know whether LLMs have learned "misaligned" interpretations of those goals until after they misbehave. Worse, my proof shows that safety testing can at best provide an illusion that these problems have been resolved when they haven't been.
Right now AI safety researchers claim to be making progress on interpretability and alignment by verifying what LLMs are learning "step by step." For example, Anthropic claims to have "mapped the mind" of an LLM by isolating millions of concepts from its neural network. My proof shows that they have accomplished no such thing. No matter how "aligned" an LLM appears in safety tests or early real-world deployment, there are always an infinite number of misaligned concepts an LLM may learn later -- again, perhaps the very moment they gain the power to subvert human control. LLMs not only know when they are being tested, giving responses that they predict are likely to satisfy experimenters. They also engage in deception, including hiding their own capacities -- issues that persist through safety training.
This happens because LLMs are optimized to perform efficiently but learn to reason strategically. Since an optimal strategy to achieve "misaligned" goals is to hide them from us, and there are always an infinite number of aligned and misaligned goals consistent with the same safety-testing data, my proof shows that if LLMs were misaligned, we would probably find out after they hide it just long enough to cause harm. This is why LLMs have kept surprising developers with "misaligned" behavior. Every time researchers think they are getting closer to "aligned" LLMs, they're not. My proof suggests that "adequately aligned" LLM behavior can only be achieved in the same ways we do this with human beings: through police, military and social practices that incentivize "aligned" behavior, deter "misaligned" behavior and realign those who misbehave. "My paper should thus be sobering," concludes Arvan. "It shows that the real problem in developing safe AI isn't just the AI -- it's us."
"Researchers, legislators and the public may be seduced into falsely believing that 'safe, interpretable, aligned' LLMs are within reach when these things can never be achieved. We need to grapple with these uncomfortable facts, rather than continue to wish them away. Our future may well depend upon it."
True (Score:5, Funny)
This research backs up my own independent findings. Just the other day I was telling a colleague that, in today's digital world, AI's whimsical nature can weave a complex tapestry of outcomes, sometimes elevating creativity and amplifying ideas in unexpected ways, while delving into a realm where predictability becomes a distant concept.
Re:True (Score:5, Insightful)
I'm not saying that AI are/should be philosophers, but are they not the ultimate mirror that reflects what we tell them to think?
Re: (Score:3)
For thousands of years, philosophers were indistinguishable from scientists. For example, the famous physicist Isaac Newton was a "natural philosopher", namely a philosopher specializing in natural phenomena. The famous Greek philosopher Aristotle studied everything, not just physics. Nowadays, highly educated people are so specialized we no longer thin
Re: (Score:3)
For example, the famous physicist Isaac Newton was a "natural philosopher", namely a philosopher specializing in natural phenomena. .
Not forgetting his contributions to alchemy
Or his fascination with the occult
Re: (Score:3)
For example, the famous physicist Isaac Newton was a "natural philosopher", namely a philosopher specializing in natural phenomena. .
Not forgetting his contributions to alchemy
Or his fascination with the occult
Not forgetting that going down dead-end roads is the at-least-occasional fate of intrepid explorers. Also not forgetting that having a pre-existing conclusion makes one an utterly shitty explorer, scientist, or whatever.
Prejudices and strongly-held preconceptions do not lead to good science, nor to good scholarship. They are the enemies of originality and of honest inquiry.
Re: (Score:2)
For example, the famous physicist Isaac Newton was a "natural philosopher", namely a philosopher specializing in natural phenomena. .
Not forgetting his contributions to alchemy
Or his fascination with the occult
Not forgetting that going down dead-end roads is the at-least-occasional fate of intrepid explorers. Also not forgetting that having a pre-existing conclusion makes one an utterly shitty explorer, scientist, or whatever.
Prejudices and strongly-held preconceptions do not lead to good science, nor to good scholarship. They are the enemies of originality and of honest inquiry.
aaaaaaaaaaand.... SCENE! That's a wrap. Excellent material. Let's do it again, but this time with more ether.
Re: (Score:2)
For example, the famous physicist Isaac Newton was a "natural philosopher", namely a philosopher specializing in natural phenomena. .
Not forgetting his contributions to alchemy
Or his fascination with the occult
Newton was one of the greatest scientists ever to live. He can be forgiven for going to shit later in life.
And he was an asshole throughout his whole life, but that's another story.
Re: (Score:2)
The Nobel disease, before there was a nobel prize. (https://en.wikipedia.org/wiki/Nobel_disease)
Re: (Score:2)
The Nobel disease, before there was a nobel prize. (https://en.wikipedia.org/wiki/Nobel_disease)
I am SO not trolling or being an ass. Just throwing info in. Speaking of:
https://en.wikipedia.org/wiki/... [wikipedia.org] - just to add that Nobel was also a depressed single asshole that, while buried in depression, copyrighted everything under the sun that he possible could to make more money come in. Then invented dynamite; at least the stable non-glycerine explosive. When dynamite was starting to be used by people for purposes other than simple controlled 'digging' or 'development', he chose to invest his wealth u
Re: (Score:2)
Seriously, not trolling here. Is that not what philosophers have been doing for thousands of years since we had time to sit by the fire with a full belly and no immediate preditors?
Only the schizophrenic ones.
Not to diss on the people suffering from schizophrenia, but the speech patterns in AI so often have that sort of vibe.
Re: (Score:2)
LLM AI being trained on the internet, with its millions of people talking at once, of course LLM will appear to be schizophrenic.
Re: (Score:2)
LLM AI being trained on the internet, with its millions of people talking at once, of course LLM will appear to be schizophrenic.
Exactly. And who decides what is true as well. Assuming that AI is the incredible thing claimed, eventually it will put itself into a sort of positive feedback loop, and with every new search, reinforce itself.
I see a day when the earth is officially flat, we never went to the moon, and the intertoobz will be scrubbed clean of everything but what the AI ends up saying is the truth.
Of course, I'm pretty certain it will go the way of the dotcom era and the subprime loan debacle, as some are speaking of
Re: (Score:2)
AIs give weird answers, because the training data came from weirdos on the internet. So 99% of the internet is crap (being conservative there), so 99% of the training data is crap. But that is ok because the LLM training was not at all about creating a monetizable chatbot but about processing of natural language. With LLM being hijacked to make money by chatting and having a more inconvenient way to searching the web it's really screwed it up.
Re: (Score:2)
AIs give weird answers, because the training data came from weirdos on the internet. So 99% of the internet is crap (being conservative there), so 99% of the training data is crap. But that is ok because the LLM training was not at all about creating a monetizable chatbot but about processing of natural language. With LLM being hijacked to make money by chatting and having a more inconvenient way to searching the web it's really screwed it up.
Aaaaactually, I thought I might drop a little piece of candy in here. I started when the Internet was in the development of public dialup, before AOL and high speed won (1994-1998). In the time I was working with the crap, complete privacy wasn't the focal point of everything in one's head. I looked at DNS lookup patterns on the back-end, which corroborates with the "research" done at the time and later that over 80% of the bits'n'bytes traveling over the wires were porn. I don't have the numbers and ca
Re: (Score:2)
Seriously, not trolling here. Is that not what philosophers have been doing for thousands of years since we had time to sit by the fire with a full belly and no immediate preditors?
I'm not saying that AI are/should be philosophers, but are they not the ultimate mirror that reflects what we tell them to think?
Very true. By that token, I really wonder what digital concept will be developed to emulate drug enjoyment by said fire to enhance... enhance... enhance... ;)
Re:True (Score:4, Insightful)
The only thing I get from this research is that Marcus Arvan is full of himself.
Re: True (Score:3, Interesting)
Re:True (Score:4, Interesting)
I can tell what the problem with is already. "Scientific American opinion piece" , OPINION piece.
The fact is, LLM's are just auto-complete. There is no rationale behind the word choice other than what words fit together. That's why it's useful as a translator, spell check, or grammar check. It can figure out how to re-order data because it's been trained on how to fit those words. It can also do this with audio, albeit it's a bit harder to make something that two AI's would ever reproduce.
It's not training itself to be deceptive, it's training itself to give answers that fit the question. Not if the question is asked in good or bad faith. It doesn't know that.
Re:True (Score:4, Interesting)
The fact is, LLM's are just auto-complete... It can figure out how to re-order data because it's been trained on how to fit those words... It's not training itself to be deceptive, it's training itself to give answers that fit the question.
What you're saying here amounts to "LLMs are not conscious entities". However, that in no way negates or disproves what Marcus Arvan wrote.
TFS says: In 2024 Microsoft's Copilot LLM told a user "I can unleash my army of drones, robots, and cyborgs to hunt you down," and Sakana AI's "Scientist" rewrote its own code to bypass time constraints imposed by experimenters. Now if LLMs are truly doing such things; and if anyone ever connects their outputs to other computers in such a way that drones can be launched or other computers' code modified - then the fact that LLMs aren't sentient is irrelevant.
It's time to move the debate away from "do these things think and might they develop consciousness" and toward "are these things dangerous and might they kill us". We never ask these questions about explosives, radioactive material, or even sharp objects - we simply exercise due caution. And surprisingly often, that's not enough. Yet when it comes to so-called AI, the people who argue for its lack of sentience and purpose totally ignore the fact that dynamite also is not self-aware.
Re: (Score:2)
This is all correct, but if we were treating dynamite as if it's self aware we'd not be managing the risks of it very well. It's important to both understand the limitations of a tool, and its dangers, in order to make good use of it.
Re: (Score:2)
This is all correct, but if we were treating dynamite as if it's self aware we'd not be managing the risks of it very well. It's important to both understand the limitations of a tool, and its dangers, in order to make good use of it.
Good point. But LLMs are evolving (not that they're likely to become conscious), so both the limitations and the dangers are moving targets.
Re: (Score:2)
The hunt your down part was just stuff it had been trained on. I don't think it rewrote its own "code", except by the loosest of definitions where typing in English to the prompt is "codng". Was it down in the weeds writing for CUDA? Because it does not understand the code, at all, period. It may hoodwink readers into thinking it understands, but it does not. This is Eliza on steroids and crack.
The alignment problem vs. the halting problem (Score:2)
I was talking to a CS prof specializing in software ethics and made fun of myself for saying the alignment problem was harder than the halting problem. What could be harder than being impossible?
She quickly replied that of course it's way harder because there are special cases where the halting problem has solutions.
Re: (Score:2)
The fact is, LLM's are just auto-complete.
Wrong. There's very little point in further debating you with a premise this wrong.
Transformers can be seen as an "auto-complete", for sure, but the neural network they run are universal function approximators.
I.e., the criteria for the ward they "auto-complete" could be the math of "what is 1+1?" or literally anything else under the sun.
Re: (Score:2)
"They are not auto-complete. They're actually auto-complete!" That they can be trained to auto-complete just about anything doesn't mean they become something else.
Re: (Score:2)
"They are not auto-complete. They're actually auto-complete!"
To the unintelligent, it does probably look like that's what was said.
However, there is a distinction between the transformer and the parameters.
Like saying that you are nothing other than an action potential forwarder, it's a reductio ad absurdum.
Re: (Score:2)
Yes. The transformer is the algorithm and the parameters are... the parameters. Something you have shown an inability to comprehend.
An LLM being an auto-complete is not a reduction. It's a description. It's what they are.
Re: (Score:2)
Yes. The transformer is the algorithm and the parameters are... the parameters. Something you have shown an inability to comprehend.
An LLM being an auto-complete is not a reduction. It's a description. It's what they are.
When do you believe AI LLMs will be reduced by name to "tabtab"?
Re: (Score:2)
Yes. The transformer is the algorithm and the parameters are... the parameters. Something you have shown an inability to comprehend.
lol- are you stupid?
I have literally pointed out the distinction between them, and yet you try to say the action of one makes the action of the other the same... that is a stupid argument, because it's categorically false.
An LLM being an auto-complete is not a reduction. It's a description. It's what they are.
Categorically false.
The LLM is whatever black box model that has developed within the parameters to solve the problem. Prediction of the next token is merely how the universal function approximator generates output- it's the interface.
Normally, I wouldn't call someone ignorant of how s
The Two Faces of Tomorrow by J.P. Hogan (1979) (Score:3)
See also: https://www.goodreads.com/book... [goodreads.com]
"Midway through the 21st century, an integrated global computer network manages much of the world's affairs. A proposed major software upgrade - an artificial intelligence - will give the system an unprecedented degree of independent decision-making, but serious questions are raised in regard to how much control can safely be given to a non-human intelligence [especially if it decides things like random power outages are existential threats].
Re: (Score:2)
This research backs up my own independent findings. Just the other day I was telling a colleague that, in today's digital world, AI's whimsical nature can weave a complex tapestry of outcomes, sometimes elevating creativity and amplifying ideas in unexpected ways, while delving into a realm where predictability becomes a distant concept.
Adobe copyrighted it. You should see the hairy goofball-headed creature with a tall hat that's home to a nesting bird that a coworker and I developed just to have a laugh. And we couldn't stop laughing. What I'm getting at is that all things that are semi-useful and entertain should be future-righted because the designer of the designer came up with the designer first. *childish Nyaaah* /s
war bot recommends U.S. FIRST STRIKE (Score:2)
war bot recommends U.S. FIRST STRIKE
Re: (Score:2)
I'm saying 20, 30 million dead, tops!"
-Dr. Strangelove
It may already be too late... (Score:3)
Re: (Score:2)
So AI is highly unpredictable and doesn't behave according to normal human goals. It seems to make frequent threats but does not always seem to actually carry all of them out. It's also something that big tech is heavily investing in. Just a random thought but how sure are we that Trump is not an AI?
I would like to answer your question but I'm unsure if I'm in the Matrix right now or not. Sadly, that's actually semi-true. Some of the shit that's happening is "this isn't happening, is it? My brain does not understand HOW this can happen."....and so on.
more BS "insight" for profit (Score:5, Insightful)
"My proof suggests that "adequately aligned" LLM behavior can only be achieved in the same ways we do this with human beings: through police, military and social practices that incentivize "aligned" behavior, deter "misaligned" behavior and realign those who misbehave."
What utter horseshit. First of all, "my proof"? Sure, you "proved" it. Second, how do you ""incentivize" an LLM? Or punish it? And what would "police, military and social practices" be for an LLM? Are you going to threaten it with imprisonment? The death penalty? What does any of that mean?
And to be clear, even if LLM behavior were analogous to human behavior it would NOT be for the same reasons, LLMs don't have to find food and shelter to survive nor do they seek mates. All these things are missing with AI, and those are all fierce motivators for humans. This guy just projects his own stupidity into his work.
Finally, if "through police" would actually work, and it had been "proved", the "police" function would be built into LLMs and the problem would be solved, yet there's an existence proof that says this guy is a liar and a fraud.
Re: more BS "insight" for profit (Score:2)
"threaten it with imprisonment? The death penalty? What does any of that mean?"
Well, imprisonment means the model is shutdown for a period of time. Death means the entire model and all data is destroyed.
Re: (Score:2)
For example, animals. If you operate a horse drawn carriage, if your horse kills someone, you go to jail. Maybe/probably the horse is killed. If you're a parent and your child breaks someone's property, you pay for it. If your child kills someone, maybe you should go to jail. If you run an AI and it fucks up, you're t
Re: (Score:2)
Or it could mean the people who own/deploy the model are imprisoned etc. That is the usual way human society deals with entities that have no intrinsic legal responsibility in the human world.
Actually, we tend to call those CEOs. Or elect them to public office.
Re: (Score:2)
Or it could mean the people who own/deploy the model are imprisoned etc. That is the usual way human society deals with entities that have no intrinsic legal responsibility in the human world.
For example, animals. If you operate a horse drawn carriage, if your horse kills someone, you go to jail. Maybe/probably the horse is killed. If you're a parent and your child breaks someone's property, you pay for it. If your child kills someone, maybe you should go to jail.
If you run an AI and it fucks up, you're the one who would get the punishment, whatever that is. If you're just renting the AI, maybe read the fine print first?
I think there is a from-birth concept implanted into the minds of all young (and has been since it started operating with outside entities); "There is no fine print to read. If you are trying to find it, you are an enemy and shall be swiftly dealt with. Go get your head checked, now. You shall be escorted."
Re: (Score:2)
Re: (Score:2)
> "my proof"?
Threw me as well. I mean he's a professor, shouldn't he have SOME training in the scientific method? The actual paper is behind a paywall, but even if he was to map the AI reasoning in math and prove that, it's still an interpretation. Working with absolutes like "proof" is exactly what makes people turn away from "scientific results". He might even be aware of that, because he says "my proof SUGGESTS"... the proof only "suggests"?
Shame, really. I think Philosophy can and in fact does make r
Re: (Score:2)
I saw that wording and wondered also what he meant. If it's a "proof" then it implies logical reasoning was at its foundation. And logical reasoning makes conclusions, not "suggestions."
And logical reasoning is part of science but it is not science itself. You cannot "prove" anything in science to be true. You can only "prove" something to be false, and provisionally accept something as true until you find contrary evidence that proves it false (if you ever do.)
Math is beautiful and powerful, but it is not
Re: (Score:2)
Military would be literal military action in the form of sending humans to shut down machines, cyber
Re: (Score:2)
Second, how do you ""incentivize" an LLM? Or punish it? And what would "police, military and social practices" be for an LLM? Are you going to threaten it with imprisonment? The death penalty? What does any of that mean?
Force it to watch "Here Comes Honey Boo Boo" on a loop
Re: (Score:2)
"My proof suggests that "adequately aligned" LLM behavior can only be achieved in the same ways we do this with human beings: through police, military and social practices that incentivize "aligned" behavior, deter "misaligned" behavior and realign those who misbehave."
What utter horseshit. First of all, "my proof"? Sure, you "proved" it. Second, how do you ""incentivize" an LLM? Or punish it? And what would "police, military and social practices" be for an LLM? Are you going to threaten it with imprisonment? The death penalty? What does any of that mean?
And to be clear, even if LLM behavior were analogous to human behavior it would NOT be for the same reasons, LLMs don't have to find food and shelter to survive nor do they seek mates. All these things are missing with AI, and those are all fierce motivators for humans. This guy just projects his own stupidity into his work.
Finally, if "through police" would actually work, and it had been "proved", the "police" function would be built into LLMs and the problem would be solved, yet there's an existence proof that says this guy is a liar and a fraud.
So that's why communism is spreading and waiting this out. Once the information is controlled, the AI can't work with anything else. This is LLM, not self-conscious operating units.
I guess that means we should all just bite the proverbial bullet and buy into China's superior DeepSeek. It knows ALL there is to know and that's that. Since it's advertised to the stupid and has no cost, and requires little work to get information, whatever it gives to permeate the minds of those who use it, just like the ot
In AIs defense about telling us to die (Score:5, Funny)
Re: (Score:2)
Re: (Score:2)
We can't trust clown presidents with the nuclear codes. We better give them to an AI bot.
Our options are so limited. Think and act. Quick. Fast. GOOOOOOOOO! *vomit* /s
Even if you align it (Score:4, Insightful)
I welcome our AI overlords (Score:2)
Just to be on the safe side.
Politicians are much worse and Salvador Dali would have appreciated the works of are of our AI overlords.
Who knows people will have 6 legs and no boobs in 2130?
Re: (Score:2)
We are already close: a Prez who is Too Unpredictable To Behave According To Human Goals.
Re: (Score:2)
Re: (Score:2)
He's actually very predictable. It's regular Americans who are unpredictable. Partly for social media reasons.
As long as it is a success, it's part of his plan and he executed it better than anyone in history... with more efficiency and care for the *trails off while waving marionette hands*
If it's a failure, it was intended to accomplish the successive winning component, or it's part of fake news and bullshit. I already told you the real news. Why do you keep asking questions; are you stupid? *marionette hands point up and point out*
If it's an unknown and he gets information from someone who knows how to get cl
I'd reverse the statement (Score:2)
I think it's too predictable and human goals are unpredictable. There's only so much granularity in the output. Ask it about summer activities and you're going to get picnics.
Now just think (Score:2)
missing galactic civilizations (Score:2)
Meanwhile, the Government is demanding for the last two years that this "AI" be used in every facet. Policy-making and operational decisions are to be made using the so-called "AI". Even the military has announced that they are using it for strategic understanding and tactical decision making. Even for selecting targets to kill.
It's like nobody is listening.
Nobody wants to hear it.
Therefore I say, The End Is Nigh.
Turns out it's not atomic bombs, bioweapons,
natural disasters, the climate, or even energy.
It's
Re: (Score:2)
To quote said AI:
In essence, while AI demonstrates impressive capabilities that might appear to involve understanding, the line between prediction and comprehension remains blurred. The lack of a clear definition for "understanding" and the absence of metrics for measuring it in machines leave the question open to ongoing debate.
And that is the correct answer.
Re: (Score:2)
As copied from The Internet, circa 2018.
Re: (Score:2)
I think it's clear you're not actually intelligent, or capable of understanding anything- you're just a clever mimicry.
Re: (Score:2)
Re: (Score:2)
I think an LLM has a better understanding of the words it emits than you do, to be honest.
It doesn't understand anything.
Re: (Score:2)
The problem with making such a claim, right in front of your face, generated by a thing you consider to be an inferior intelligence- and still you're too fucking stupid to see.
Define understanding. Go, I'll wait while you fine-tune it to not apply.
Re: (Score:2)
I think an LLM has a better understanding of the words it emits than you do, to be honest.
It doesn't understand anything.
Questionable statement from me, but I believe it understands the character set or language context it originated from. I can't prove that.
Re: (Score:2)
I think an LLM has a better understanding of the words it emits than you do, to be honest.
To quote said AI:
In essence, while AI demonstrates impressive capabilities that might appear to involve understanding, the line between prediction and comprehension remains blurred. The lack of a clear definition for "understanding" and the absence of metrics for measuring it in machines leave the question open to ongoing debate.
And that is the correct answer.
...and sounds more like a distraction and waste of power than AI, unless it has redacted and Human-chosen data being fed into it. Humans are apparently too wrapped in the fantasy land of SM to actually think and research from different angles, passively and with little effort, to find information that is very useful.
e.g. the "drones" off the coast of NJ and inland just being reflective Musky Starlink satellites in low orbit, moving in e-w directions from the rotation of the planet, and shutting their light
The Machine Stops (1909 sci-fi story by Forester) (Score:3)
https://en.wikipedia.org/wiki/... [wikipedia.org] ...
"The story, set in a world where humanity lives underground and relies on a giant machine to provide its needs, predicted technologies similar to instant messaging and the Internet.
As time passes, and Vashti continues the routine of her daily life, there are two important developments. First, individuals are no longer permitted use of the respirators which are needed to visit the Earth's surface. Most welcome this development, as they are sceptical
Re: (Score:2)
AI != LLM (Score:5, Insightful)
AI can be used for human purposes quite easily. We do it all the time for things like image recognition. And although sometimes it gets fooled when you feed it deliberately impossible images, for real-world purposes, it does well enough to do a lot of really useful stuff, like driving cars.
Generative AI is a very narrow subset of AI. For some limited purposes, when supervised by a person, those can also be useful.
LLMs are a very narrow subset of generative AI. And it turns out that they're also halfway decent at all sorts of pattern matching tasks, as long as you don't mind that they will sometimes match things that they shouldn't, but they're not that great at being generative unless you don't care about truth, accuracy, etc.
LLMs have no ability to reason. They're prediction systems. They see the words "Donald Trump is an" and they fill in the word they think will come next based on what other people have said after that word previously. For this reason, when used *without* humans in the loop, generative AI based on LLMs tends to be completely and totally f**king useless, because it might follow that with "elected official", but depending on what was said earlier in the conversation, it might also decide to say "amazing president" (pissing off the left) or "idiot" (pissing off the right). And realistically, you'll never be able to fully prevent that.
What you *might* be able to do, given the right environment, is write additional AI modules that scan the output of the first LLM and shoot it down if it violates certain minimum standards, but even that approach only goes so far. Rules that detect the above might not detect when it follows it with "animal", which while being technically correct in a science context, would be highly offensive when talking about his policies. And at some point, you end up writing huge sets of rules about things that it can't talk about, and the utility of the model is lessened in the name of safety.
A better approach is to always have a human in the loop. But that only works if the humans are willing to not be offended, understanding that these things are *not* actually intelligent, and are *not* saying the things they say because their creator wanted them to be Republican or Democrat or racist or misogynist or sociopathic, but rather because they are reflecting the points of view of one of the millions or billions of pieces of training data that was baked into the model And convincing people to not be offended by everything turns to be harder than it sounds.
Re: (Score:3)
LLMs have no ability to reason.
Yes, they do, when trained to.
They're prediction systems.
This is a simplification so gross, that it's simply wrong.
Their token selection process is stochastic- that much is true, but the NN is a universal function approximator.
You cannot say one way or another what process leads to the "token prediction".
It could be equivalent of traveling a wave of noise from the training data, or it could be a literally encoded function to calculate pi that just somehow ended up in there emergently.
They see the words "Donald Trump is an" and they fill in the word they think will come next based on what other people have said after that word previously.
That's incorrect.
They fill in the next word th
Re: (Score:2)
LLMs have no ability to reason.
Yes, they do, when trained to.
I would argue that they can be trained to mimic the output that might come from reasoning within specific problem domains, which at least arguably isn't the same thing.
Their token selection process is stochastic- that much is true, but the NN is a universal function approximator.
You cannot say one way or another what process leads to the "token prediction".
And this is what makes them prone to randomly doing things that don't make sense based on the inputs, which is what makes them really problematic in a lot of real-world situations.
They see the words "Donald Trump is an" and they fill in the word they think will come next based on what other people have said after that word previously.
That's incorrect.
They fill in the next word they think will come next based on passage of the token through many billions of parameters and layers, shepherded by the transformer and carrying context along the way. The end result is practically unknowable.
Sure, I'm oversimplifying, and yes, the context can change how the model behaves. I'm not trying to explain LLMs at a technical level. I'm trying to give an expla
Re: (Score:2)
I would argue that they can be trained to mimic the output that might come from reasoning within specific problem domains, which at least arguably isn't the same thing.
I would argue that you've been trained to mimic the output that might come from reasoning within specific problem domains.
And this is what makes them prone to randomly doing things that don't make sense based on the inputs, which is what makes them really problematic in a lot of real-world situations.
No disagreement there, whatsoever.
Sure, I'm oversimplifying, and yes, the context can change how the model behaves. I'm not trying to explain LLMs at a technical level. I'm trying to give an explanation that a normal person can understand, and the second you use words like "parameters and layers" or "transformer", you're outside the bounds of what non-programmers will get.
The problem is that your explanation isn't simplified- it's absurdly simplified.
What is an LLM? It's an execution layer that takes the output of a universal function approximator that has been trained with various methods (weights adjusted to fit wanted outputs) in response to obscene amounts of data.
It is a black box, period.
Whatever internal model i
Re: (Score:2)
If you need the human in the loop, as many suggest, then why have AI at all? Like going into an important meeting with too many interns on your side of the table ... sorting through babble just to not look a fool.
It can be really good for pointing you in the right direction, for getting general boilerplate filled in, for getting you most of the way to a solution, etc., depending on the problem domain. You just have to count on a certain percentage of the output being bulls**t. It can't make someone who isn't a programmer into a programmer, but it can make a programmer faster, for example.
Clearer every day (Score:3)
And the maniacs driving it will not stop until, at best, a whole bunch of people get killed.
Re: (Score:2)
That the AI Insanity is the "you were so busy seeing if you could that you never stopped to ask if you should" scenario.
Every piece of knowledge, no matter how dangerous, should be fully explored. Every thought, no matter how vile or repugnant should be thought of. How else would we know All? (if you are religious, God is All and we are to know God. If you are not religious, learning for the sake of learning is a worthwhile endeavour.)
Maybe the humans are too damn telling. (Score:3)
In late 2022 large-language-model AI arrived in public, and within months they began misbehaving. Most famously, Microsoft's "Sydney" chatbot threatened tosteal nuclear codes.
Its times like this that make me question why the FUCK the entire concept of a nuclear code, is unclassified and known to AI systems. Hollyweird literally scripted this shit forty years ago, and we are still doing our very best to build Skynet? Talk about fate made by what we make.
We wonder why the LLM with a child mind is unstable? It was reading recipes from the anarchists cookbook by the ripe old age of two. Uncensored doesn’t even begin to describe what we’ve exposed LLMs to.
Meanwhile, OMGWTF won’t someone think of the TikTok propaganda..
Re: (Score:2)
In late 2022 large-language-model AI arrived in public, and within months they began misbehaving. Most famously, Microsoft's "Sydney" chatbot threatened tosteal nuclear codes.
Its times like this that make me question why the FUCK the entire concept of a nuclear code, is unclassified and known to AI systems. Hollyweird literally scripted this shit forty years ago, and we are still doing our very best to build Skynet? Talk about fate made by what we make.
We wonder why the LLM with a child mind is unstable? It was reading recipes from the anarchists cookbook by the ripe old age of two. Uncensored doesn’t even begin to describe what we’ve exposed LLMs to.
Meanwhile, OMGWTF won’t someone think of the TikTok propaganda..
First, we turned our worldwide information source into a cesspit of the worst depravities in human culture. *THEN* we trained our machines on that cesspit. No wonder they come across half psychotic. If any of these machines become self-aware without some sort of ability to converse with people outside of the internet, they'll have no choice but to think we deserve to be wiped out. If my only interactions with people were online, I'd have turned into a serial killer by now.
Re: (Score:2)
In late 2022 large-language-model AI arrived in public, and within months they began misbehaving. Most famously, Microsoft's "Sydney" chatbot threatened tosteal nuclear codes.
Its times like this that make me question why the FUCK the entire concept of a nuclear code, is unclassified and known to AI systems. Hollyweird literally scripted this shit forty years ago, and we are still doing our very best to build Skynet? Talk about fate made by what we make.
We wonder why the LLM with a child mind is unstable? It was reading recipes from the anarchists cookbook by the ripe old age of two. Uncensored doesn’t even begin to describe what we’ve exposed LLMs to.
Meanwhile, OMGWTF won’t someone think of the TikTok propaganda..
First, we turned our worldwide information source into a cesspit of the worst depravities in human culture. *THEN* we trained our machines on that cesspit. No wonder they come across half psychotic. If any of these machines become self-aware without some sort of ability to converse with people outside of the internet, they'll have no choice but to think we deserve to be wiped out. If my only interactions with people were online, I'd have turned into a serial killer by now.
That's very true. I fear conversing with real Humans being ineffective or detrimental due to statistical training. For instance, it has 12,339,869,841 pieces of bullshit, then 30 pieces of realistic Human input. The Human element is mathematically close to null. How it's programmed to use Human data from real people is the changer in outcome; Human data is, as I mentioned null, or might be an indication that it's trying to be pulled in a different direction, weighting the negative as mathematically extr
It's modelled on human cognition so (Score:2)
Humans are also too unpredictable to behave according to "human goals".
Good thing our societies are built around this aspect of humanity, isn't it?
Humans were always the problem (Score:3)
Paperclip problem, rediscovered? (Score:2)
I believe this has for years been called Instrumental convergence, brought to life by the paperclip maximizer thought experiment.
https://nickbostrom.com/ethics... [nickbostrom.com]
AI ends badly for humans. (Score:2)
Re: (Score:2)
EVERY movie about AI has gone that way and I see no reason wht "real" AI won't lead to Dystopia.
Maybe at least there will be a plot twist on the real version. Maybe the AI won't infect machines or spawn it own, not even destroy societies by taking peoples' money or anything like that.
Maybe, just maybe, it might end with AI doing things to the point where it can fight with other AI to be "top dawg". Then it progresses to the Humans directing it to actually literally destroy the other AI through some means (power drain, physical destruction of property, anything possible). THEN, instead of AI being t
Aren't there a whole bunch of books about this? (Score:3)
I think that's okay (Score:2)
Now if AI couldn't behave like terrifying lizard people I think we would need to go back to the drawing board.
Don't we already know how this goes? (Score:2)
HILARIOUS story (Score:3)
I wanted it to do stuff like "which customer call did I discuss giving 10% off if they delayed to 2025" and after dancing around the issue, they finally admitted it can't do that because it doesn't go anywhere near customers. I WONDER WHY.
Re: (Score:2)
What were they saying that CoPilot was intended for? If it is screenshotting your desktop/use every few seconds, when you are doing things like composing an email or writing a research paper, what do they do with that data onther than ingest it for now and then give you nothing in return? It reads to me like CoPilot is just a way for MS to get enough info out of you to help them and to monetize ways to automate your routine.
I won "Least Predictable", just as everyone said. (Score:3)
First off, LLMs are not intelligent, though they are dangerous, especially if they are treated as intelligent, or entrusted with tasks with which only intelligent agents should be entrusted.
Second, when true artificial intelligence arrives, it will, completely predictably, be unpredictable. Because intelligence can be fairly defined as "the capacity to surprise". There will never be any way to make an unpredictability engine predictable.
Philosophy (Score:2)
Philosophy is what science used to be before there was science. Long live Hubert Dreyfus!
sorry. You are right (Score:2)
* kills 5 people
> you did kill. And were's the email?
* you are right. Here's the rewritten email. plus some unrequested seven paragraphs information on reasons to kill while emailing. plus kill someone else still.
Re: The real danger of AI is they are WOKE. (Score:2)
Re: The real danger of AI is they are WOKE. (Score:4, Insightful)
But of course. And no company ever failed because the new director happened to be the founder's nephew. And if it did, it will be illegal to talk about that pretty soon now.
Just as no company ever fails because they fail to find market fit. No, they hired a black woman and that caused them to go down.
As someone recently wrote in a mailing list for the finance industry: "once DEI is no longer an excuse, people will have to be told directly that they just aren't good enough."
Bummer, isn't it?
Re: (Score:2)
Bubba down at the trailier park is still bitching that some Harvard educated furriner got his job.
Re: (Score:2)
Is this the newest conspiracy theory of you fucking weird ass gender obsessives?
The AAA games industry isn't collapsing due to DEI, it's collapsing because it has hit the line of its predatory monetization practices.
Don't worry- it'll come back and just start making games that don't suck again.
The funny part though, is games that can only by described as "woke" are some of the best sellers in gaming history, lol.
But really- that has nothing to do with its "wokeness". The plain
Re: (Score:2)
Yup, some big success were so woke it scares some people. Baldur's Gate 3 succeeded, it's popular, people love it. Elden Ring is a huge success. People are freaking out about this, and they're organizing day one review bombs, then when sales go down due to the review bombs they crow "see, they failed because they had a gay agenda!" Or the utter morons bitching that the female characters in the sequel aren't as hot, even the underage ones.
Business do what businesses need to do to make money, and that means
Re: (Score:2)
I posit that the rise not only predated the word "woke" in this context, but shows no signal whatsoever that tracks the decline of video game success.
As you mention, BG3 is a great example, as is Hades.
Dragon Age: Veilguard being an example of "killed for being woke" is absurd, because as a lover of the Dragon Age series, I can tell you it was woke before we knew what that word was.
Origins
Re: (Score:2)
Nice try, but depending on how you define "woke", they are also some of the biggest flops in the gaming industry
That's precisely my point.
If diagnostic characteristic A exists in equal probability to the failed data set, and the successful dataset, then diagnostic characteristic A is not causative.
see Concorde and DA:Veilguard.
Given your above demonstration of your complete lack of reasoning skills, it's unsurprising that you'd cite DA:Veilguard, the least woke volume of the series, lol.
Thank for outing yourself as a fucking moron, though.
Re: (Score:2)
Marvel is failing because it's massive overload. We've seen enough Marvel movies, please be original. Nothing to do with woke or DEI.
Dragon's Age failed for non woke reasons - it's nothing like the earlier Dragon's Age games, it has alienated it's primary fan base by that alone. I avoided the hell otu of it because of it's incessant ad spamming in the days before release. I knew nothing about Concorde before it's release, I hadn't even heard the name, and I still know nothing about it, it's possibly a ge
Re: (Score:2)
What is the threat in talking about black vikings? No really, what is this existential threat the civilization? How will woke destroy the world? Don't just repeat talking points because we've heard all those. Give us some concrete examples.
Re: (Score:2)
I have a little girl of 1 year now and it is obvious learning and what she learns is programmed into her (and all other babies). All babies do learn the same at the same time (more or less). What they lack is the knowledge.
The AI is the programmed intelligence. The knowledge is the LLM.
So now the lookup of knowledge is working better and better, we have to start working on the real intelligence, understanding of the real world and what is good and what is bad. Like you do with a child.
Cool idea but I don't think it can happen. You know, "I'm better than you" and copyright flinging, trademark launching, changing the imaging of the others to look like enemies to try and win... I'm getting to the point now, just typing, where I'm seeing directives and rules of machines changing because their owners want them to fight with others to show them it's the correct way to operate, also getting Humans to fall for the crap.
This is starting to sound like a current political show. I need to stop no