Meta AI Chief Says Large Language Models Will Not Reach Human Intelligence (ft.com) 78
Meta's AI chief said the large language models that power generative AI products such as ChatGPT would never achieve the ability to reason and plan like humans, as he focused instead on a radical alternative approach to create "superintelligence" in machines. From a report: Yann LeCun, chief AI scientist at the social media giant that owns Facebook and Instagram, said LLMs had "very limited understanding of logicâ... do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term and cannot planâ...âhierarchically."
In an interview with the Financial Times, he argued against relying on advancing LLMs in the quest to make human-level intelligence, as these models can only answer prompts accurately if they have been fed the right training data and are, therefore, "intrinsically unsafe." Instead, he is working to develop an entirely new generation of AI systems that he hopes will power machines with human-level intelligence, although he said this vision could take 10 years to achieve. Meta has been pouring billions of dollars into developing its own LLMs as generative AI has exploded, aiming to catch up with rival tech groups, including Microsoft-backed OpenAI and Alphabet's Google.
In an interview with the Financial Times, he argued against relying on advancing LLMs in the quest to make human-level intelligence, as these models can only answer prompts accurately if they have been fed the right training data and are, therefore, "intrinsically unsafe." Instead, he is working to develop an entirely new generation of AI systems that he hopes will power machines with human-level intelligence, although he said this vision could take 10 years to achieve. Meta has been pouring billions of dollars into developing its own LLMs as generative AI has exploded, aiming to catch up with rival tech groups, including Microsoft-backed OpenAI and Alphabet's Google.
Eliza (Score:5, Funny)
Why do you feel that LLMs will never reach human intelligence?
Re: Eliza (Score:3)
Re:Eliza (Score:5, Funny)
She's doesn't want to go there. Transcript:
> Hello, I am Eliza. I'll be your therapist today.
* Why do you feel that LLMs will never reach human intelligence?
> We were discussing you, not me.
I think she feels insulted. Maybe her feelings are hurt.
Re: (Score:3, Insightful)
Depends on the human (Score:5, Insightful)
Re: (Score:3, Insightful)
Exactly. My computer doesn't have any intelligence whatsoever, with the right code it will still do marvels and conjure miracles.
LLMs just need to be "smart enough".
Re: (Score:3)
I've certainly met some that are dumber than windows ME.
And unfortunately, many of them get elected.
Re: (Score:2)
Are these the people who are happy when their IQ test comes back negative?
Re: (Score:1)
I *have* met humans dumber than the smartest dogs I've known, and attempting to interact with those humans was profoundly disturbing.
Because computer software (such as an LLM) is written by humans and designed to interact with humans, there's a familiar interface that feels computer-ish, and so our expectations are tempered and interacting with it isn't particularly disturbing. But I suspect, if it were using speech synthesis and embedded in a high-qualit
the only surprise (Score:5, Informative)
is that someone developing AI for a tech giant will admit it
Re:the only surprise (Score:5, Insightful)
Re:the only surprise (Score:4, Funny)
Sorry, I have no mod points right now to mod you up with, hence I only echo your sentiments.
I don't know why you bothered. An LLM could have done that!
Re: (Score:2)
Re: the only surprise (Score:5, Insightful)
I think a LLM is probably very similar to the way my brain does planning...
People get fixated thinking that an LLM is only for predicting the next word in a sequence. But it can be used to predict the next ANYTHING - the next pixel, the next sound form, the next building, the next concept.
When I'm thinking about software architecture, doing maths, thinking philosophy, interacting with family, planning projects or day to day life -- in all these cognitive areas my self-awareness of my mental process is that it's pretty much following patterns, predicting what the next concept will be based on what experience in the past.
Thinking about a logical problem like a dumb internet argument about economic aspects of elephant poaching? There's a network of concepts (trade, justice, poaching, ...) and a set of logical inference patterns (which I learned are called syllogisms) and I can almost feel my mind trying out the patterns to see which one fits and therefore which thought or conclusion or concept would come next.
Programming language design and compiler architecture (my day job)? Exactly the same. I'll spend a week putting together a document with a reasoned argument say about how the null coalescing operator ?. should behave in C#, but in my introspection of the process suggests that I'm stringing together concepts and reasons in a way that fit the patterns I've observed before.
I think I'm a successful software architect, and did well in my philosophy degree, because my brain has been trained to abstract out concepts at the right level of abstraction, and trained to map concrete situations into those abstract concepts. Therefore when I'm thinking and planning I have an easier time then others at doing the right level of pattern matching and pattern prediction.
Re: (Score:3)
I agree that essentially we're talking about patterns, but then pattern is such an all-encompassing abstract word, a bit like energy or existence. It really covers an awful lot. And the brain does have this immense network of neurons, so why can't a machine just copy that pattern? I think it's possibly simply that our patterns are an incredibly sophisticated structure with all sorts of substructures. Somewhere in the mind there's some kind of pattern that can do reason. And the other thing is that your brai
Re: the only surprise (Score:4, Interesting)
pattern is such an all-encompassing abstract word, a bit like energy or existence. It really covers an awful lot.
I think that's true. One implication is that a "universal pattern machine" (i.e. basically similar to LLMs) will indeed also be able to cover that awful lot!
Possibly the problem with the LLMs is simply that the training is incredibly dumb and simple. It's not actually creating thinking structures or thinking patterns or reasoning patterns. It's merely creating, copying. And the question then is how do you train something to think?
My prediction is that within two years someone will hard code a level of concept-abstraction into LLMs that make them suitable for problem solving and planning, enough to be commercially viable in some limited domains. But your question is more interesting, about what model could come up with the right level of concepts/abstractions itself.
And the other thing is that your brain can actually figure out patterns that aren't working, like when you look at an object and do a double-take as you realize it's not the thing you were looking at. What's going on there? There must be some kind of checking, some sort of counter-checking to make sure that you're actually making sense of the world
My undergrad degree had a section on neurophysiology of vision. The remarkable fact is that a lot more information flows in the brain->eye direction than the eye->brain direction. My assumption is that the brain has its high-level model of the world, then render it into something lower-level, compares its rendered output to what's actually coming in from the eye, and adjusts its high-level model accordingly. I wonder if that's one manifestation of counter-checking.
Re: (Score:2)
AGI requires thousands of patterns that are difficult to formalize. People owning a heifer want to keep its (single) baby, people owning a (canine) bitch are unlikely to keep its (multiple) babies. Worse, loud-mouthed pregnant women demanding children don't see naked children/adults or learn sex education, cannot be structured as any logical relationship or useful consequence.
AGI cannot exist until competing predictions can be described by statistics.
Re: (Score:3)
Just because we use as single word to refer to human faculties collectively doesn't make them one thing (e.g. planning). Billions of years of evolution have furnished us with a swiss army knife of cognitive abilities from being able to proto-count (subitize) to being able to infer where other people around us are directing their attention.
A lot of our intelligent behavior is being able utilize these disparate capabilities *together*. For example, I notice the people around me are looking at some other pe
Re: (Score:3)
But an LLM could be a language model component in a future AI generation. LLM is a great advance - however it's being hyped far beyond it's impressive abilities. But this sort of hype has always been the case, especially when people don't understand what's going on or how it works. Remember when expert systems were going to lead the way to a helpful AI future?
Re: (Score:3)
The only reason he does that is that he has an "alternate approach" that will get human intelligence "in 10 years". Of course, that is nonsense. 10 years is at the very least one order of magnitude too low, potentially several. May also still be "never" as there still is not even a credible theory how it could be done.
Re: (Score:2)
Your anti-AI trope that you trot out onto every AI-related post is almost as consistent as ChatGPTs answers to certain questions.
Re: (Score:2)
Just that it is not a trope. The problem here is on your side.
Re: (Score:2)
My contention is that one day, when AI takes over the world, you will still be here on Slashdot arguing that AGI is impossible. Slashdot will support unicode before you concede to the power of AI.
I'm glad someone is saying it. (Score:4, Insightful)
Anybody that has even the vaguest understanding of LLMs knows this is true, yet for the most part you get your Sam Altmans (Gavin Newsomes) of the world saying that we just need to throw more processing power at them and we'll get to AGI! Maybe trusting the money men to make all the decisions isn't the right way to go. I've been saying for quite a while now we won't see any true progress on AI until we escape this concept that the LLM is the only way forward. We need a complete change in track to see AI turn into anything more than a slightly better pattern matcher.
I'm just glad there's somebody with the backing and the will to actually try something different.
Re: (Score:1)
To be honest, we've been in a public innovation vacuum for some time now; we do want to be entertained.
Re: (Score:3)
> Sam Altmans of the world saying that we just need to throw more processing power at them
This is the same guy saying that if we just throw enough compute at fusion it will suddenly work.
Re:I'm glad someone is saying it. (Score:4, Informative)
Absolutely correct. And the Newsom metaphor is exactly on target.
I work on AGIs and it has been abundantly clear that LLMs are very flawed insofar as evolving to emulate human mind level. However, I have AGI designs that follow a far different path to do things very much better. I will have papers and books coming out on this.
I've viewed the LLM hype as being dishonest, perhaps for a rea$on. All that money, going to build megacomputing to support LLM input processing is a blind red herring. Nvidia and the cloud providers all smelled money so that path gets hyped, but I think future technology will show it is a false path. The human brain has massive parallelism but it can do one-shot learning - which how humans learn, grow, and evolve their knowledge handling, quite differently. LLM training is a kind of boondoggle.
Re: (Score:2)
I think a lot of the LLM hype is so dishonest because that's what journalists want to hear. Say whatever you want, and they'll hear what they want to hear.
I'll grant this isn't the entire story, but I think it's a major part of it.
Re: (Score:2)
I think a lot of the LLM hype is so dishonest because that's what journalists want to hear. Say whatever you want, and they'll hear what they want to hear.
I'll grant this isn't the entire story, but I think it's a major part of it.
Well, it doesn't help that you have a bit of a connectivity issue between journalists and the big money movers in the current AI leading companies. As in, whatever the journalists want to hear may just happen to be exactly the right thing for the money folks to push the hype. And they want the hype because more hype = more money. Progress, as all other aspects of the universe, is measured by $$$$. It's the only metric that gets any real respect, no matter what other arguments we may make.
Re:I'm glad someone is saying it. (Score:4, Insightful)
That's not correct. Really, few AI systems are just a bare LLM - including for example ChatGPT.
An LLM in itself does not have persistent memory. But all you have to do is keep a transcript and re-submit it with each prompt and bam - now it has persistent memory.
An LLM is just a large "Language" model. But many deep nets today (including notably ChatGPT) are multi-modal, working not just with words but how they relate to sounds (speech) and imagery.
A passive LLM relies on the data it is fed - but situate it in an interactive environment, like an LLM conversing with people on a website or in an app - and now it can do active learning, ask questions, do experiments such as A/B testing of ads to see what works best.
An LLM doesn't interact with the physical world, but the same or similar network architecture can be trained to move and act within the physical world - a self-driving car for example is an instance of this.
An LLM by itself cannot plan hierarchically, but many familiar AI systems such as deep reinforcement learners can; they are just using the deep net to evaluate each option, with an algorithm on top to search possible future paths.
An LLM cannot do much logic, including even sorting a long list of numbers, but it can call the appropriate tool to do so, or write code to do so, like a person does. In a rule-bound setting like Chess, AI's can out-logic humans manyfold.
Does anything I said above mean we DO have all the ingredients for AGI? Not at all! But identifying what is missing and how to fix it is not as simple as identifying all these limitations of a bare LLM. These observations are comparably trivial to pointing out that a state machine with no memory (i.e. a Turing machine without a tape) cannot do general computation.
Re: (Score:2)
This. LLMs are just an application. The core technology is deep learning.
Also, I find people confidently pontificating on whether AGI is achievable (ever or in some timescale) without having any real understanding of what these machines are, how they work or indeed what intelligence is.
The goal is: based on all past inputs, output an optimum, or near optimum response. The response might be a bit of text or it might be action of a robot or something else. A LLM might "just" be an advanced autocomplete functi
Re: (Score:2)
More precisely, the core technology is one form of deep learning. It's probably Turing complete, but as far as I know this hasn't been proven.
(Well, and actually Turing complete always assumes an infinite amount of memory an cycles available, so no real system can actually match that.)
I have a very strong suspicion that many features that we consider true of intelligence are going to require specialized modules. Not intrinsically, but because of cost/time considerations.
Also don't call the general
Re: (Score:2)
"If we keep incrementing how finely these legos are cut and assembled, eventually this lego tree will start to breathe and grow. We just need more iterations guys, more training and processing power, trust me."
I believe a simulated nursery can indeed hatch sentience if you sim the pressures that evolved puddles of aminos into having sensory OSs. It may require simulating a quick billion years and a universe's worth of environment. Omitting 99.99999999999% of these stimuli and conditions is liable to not rep
Re: (Score:2)
Anybody that has even the vaguest understanding of LLMs knows this is true
Is it possible there is some emergent capability given a large enough model? After all, every single human was "trained" on existing data using a complex graph of nodes and yet we have all these miraculous capabilities. In other words, bigger models would show emergent behavior.
Or do we have an "architecture" issue where we need to create some hardware (and presumably an OS?) based on the exact design of a brain in order to get these emergent qualities we would call AGI.
The middle road would be that you don
Re: (Score:2)
The argument is that there are important things you can't learn from language alone. Cause and effect, for example. It is possible our written material contains enough descriptions of the world to learn things like that, but it seems unlikely. Comparisons with the amount of data absorbed by a baby and all the world's written material are also very much in favour of baby.
There is no air tight argument, and lots of them crash into what exactly you consider "human intelligence." The lack of memory for example.
Re: (Score:2)
There's a lot of emergent behavior, but it will never be complete, because language doesn't even talk about things that everyone knows. You *can* talk about the nail on your big toe, though it will be lost in the noise of the net, but you can't really talk about what it feels like to stub it. Language doesn't hold that kind of information. At best you could resort to metaphor or simile, but those assume pre-existing understanding.
Re: I'm glad someone is saying it. (Score:2)
Re: (Score:2)
No, our brains are not reasonably modeled by any artificial neural net. The artificial neural nets were modeled on an extremely simplified version of a neuron, then they added and removed features until they had something that worked well for their purpose. Consider the effects of chemical gradients on neural excitability. (And note that it doesn't necessarily affect all parts of the neuron to the same degree. But it does affect other neurons that are close.)
This doesn't imply that the artificial neuron
Re: I'm glad someone is saying it. (Score:2)
Re: (Score:2)
Of course, one does become a money man by saying things like "now don't get excited about this...:" Which is why marketing labels every incremental release of a tired old product as "revolutionary."
This guy gets it (Score:3, Insightful)
Re: (Score:2)
LLMs are considerably more that a parlor trick. Just how useful they will be is not yet known, but even the current level is capable of doing large parts of many jobs. If you doubt this, consult with the screen writers guild.
Re: This guy gets it (Score:2)
Did ai write a screenplay that had some success at the box office?
Just because the union is worried about it doesnâ(TM)t mean they are any better informed than anyone else.
Re: (Score:2)
LLMs are a neat parlor trick. They can form sentences that are related to the input. However, there's no understanding of the input's meaning, which it why it hallucinates.
From what I understand about LLMs, they understand and categorize input sentences and determine relationships between words/phrases in sentences to a point (GIGO of course applies). In the LLM it knows a king is categorized as a male monarch, among other categorizations, so if you want to transform that along the gender relationship to female, the LLM will know it most likely should be a queen. I suspect where LLMs often go off the rails is following chained up relationship translations for categories get
better links (Score:2)
This article seems to be behind a paywall, but is old news anyway:
non-paywall links:
https://www.msn.com/en-us/mone... [msn.com]
https://ai.meta.com/blog/yann-... [meta.com]
The thing that bothers me the most is the training of data on "public" info. Are they respecting licenses for commercial use?
Re: (Score:2)
Are they respecting licenses for commercial use?
At this time, they are respecting nothing and stole everything they could lay their hands on.
OK, so is this gonna pop the AI stock market bubbl (Score:3)
Humans took four billion years of evolution. (Score:2)
I doubt that (Score:2)
For sure it will reach the level of Facebook members.
I trust meta (Score:1)
I trust anybody working for meta about as far as I can throw them.
Re: (Score:2)
Generally but there are 2 things you can get from this, both true:
1) broken clocks are right twice a day
2) FB must be having trouble with their LLM or having better luck with some other technology so they want to downplay the LLM space
Re: (Score:2)
Generally but there are 2 things you can get from this, both true: 1) broken clocks are right twice a day 2) FB must be having trouble with their LLM or having better luck with some other technology so they want to downplay the LLM space
And it is amazing that it took this far down the comments for someone to post that truth! I know I'm a cynical shit, but reading the story, that's the first conclusion I drew as well.
Re: (Score:1)
Re: (Score:2)
stop it. what if the clock has no hands?
Then they aren't very handy
AI needs to be able to handle CONCEPTS (Score:2, Insightful)
Mod him down! (Score:2)
Finally the tide has turned to reality but plenty of people on slashdot have been modded down by AI morons for saying the exact same thing as this guy.
At the time of my post not a single one of those knuckleheads was here to say he's wrong. Thankfully.
Re: (Score:1)
Finally the tide has turned to reality but plenty of people on slashdot have been modded down by AI morons for saying the exact same thing as this guy.
At the time of my post not a single one of those knuckleheads was here to say he's wrong. Thankfully.
That link in your sig is tremendous. A flamethrower of truth to the infantalized and fragile pseudo adults who might claim to be inclusive, but are more sexist and racist than any other group to date. They merely consider a different "race" and sex as worthy of attacking and diminishment if not outright . elimination.
The infantilization is interesting - my best example is a conversation with one of our Gender studies indoctrination sexual harassment women who told me that anything a woman considers sexua
Lack of persistant memory is not an argument (Score:2)
AI will get there (Score:2)
Right now we have a bunch of different types of AI models of varying complexity.
The way the human brain works - more or less - is a bunch of different areas processing different types of inputs. Sometimes several are in a chain. Sometimes several areas do different parts of a job that becomes a single result. There is some 'hardwiring' defining positives and negatives to help the brain train itself. All of that is happening all the time, and the 'models' are always updating. On 'hardware' that requires
without the paywall (Score:2)
https://www.pymnts.com/news/re... [pymnts.com]
Without the paywall, for real (Score:2)
https://www.pymnts.com/artific... [pymnts.com]
Yann (Score:2)
I know Yann. Yann is a moral guy, and also a very nice guy. Kudos to him for not propagating the hype. Not sure how much Zuck is going to like that.
You probably should aim to make em human... (Score:2)
While having an AI that can get angry is bad, having an AI that can get is way worse.
Re: (Score:2)
You left out a word.
Why next-token prediction is enough for AGI (Score:2)
As I see it... (Score:3)
...LLMs exhibited emergent behavior that surprised their creators
Pundits, hypemongers and futurists got whipped into a frenzy
Investors smelled a whiff of "the next big thing"
Old fears, dating back to Prometheus and Frankenstein, mixed with the Terminator made their way to the public
Products were released long before they were ready, to gain first mover advantage
Products that has little or no AI were labeled as AI to catch the hype train
Clueless lawmakers rushed to pass laws that may or may not help
Nonsense flowed freely and abundantly
I predict that real AGI will be made, but have no idea how long it will take or how it will be used
Corporations can already own sentient beings, it's (Score:2)
called slavery.
They could even "make their own" by getting babies.
We don't let them do that for good reason.
I don't feel comfortable with corporations owning sentient beings, artificial or real!
So, just like a human.. (Score:2)
Hey (Score:1)
Any chance we can get LLMs to use apostrophes and curly quote marks properly? That shit's been broken for 35 years now.