AI Isn't Creating New Knowledge, Hugging Face Co-Founder Says (x.com) 75
An anonymous reader shares a report: AI excels at following instructions -- but it's not pushing the boundaries of knowledge, says Thomas Wolf. The chief science officer and cofounder of Hugging Face, an open-source AI company backed by Amazon and Nvidia, analyzed the limits of large language models. He wrote that the field produces "overly compliant helpers" rather than revolutionaries. Right now, AI isn't creating new knowledge, Wolf wrote. Instead, it's just filling in the blanks between existing facts -- what he called "manifold filling."
Wolf argues that for AI to drive real scientific breakthroughs, it needs to do more than retrieve and synthesize information. AI should question its own training data, take counterintuitive approaches, generate new ideas from minimal input, and ask unexpected questions that open new research paths. Wolf also weighed in on the idea of a "compressed 21st century" -- a concept from an October essay by Anthropic's CEO Dario Amodei, "Machine of Loving Grace." Amodei wrote that AI could accelerate scientific progress so much that discoveries expected over the next 100 years could happen in just five to 10.
"I read this essay twice. The first time I was totally amazed: AI will change everything in science in five years, I thought!" Wolf wrote on X. "Re-reading it, I realized that much of it seemed like wishful thinking at best." Unless AI research shifts gears, Wolf warned, we won't get a new Albert Einstein in a data center -- just a future filled with "yes-men on servers."
Wolf argues that for AI to drive real scientific breakthroughs, it needs to do more than retrieve and synthesize information. AI should question its own training data, take counterintuitive approaches, generate new ideas from minimal input, and ask unexpected questions that open new research paths. Wolf also weighed in on the idea of a "compressed 21st century" -- a concept from an October essay by Anthropic's CEO Dario Amodei, "Machine of Loving Grace." Amodei wrote that AI could accelerate scientific progress so much that discoveries expected over the next 100 years could happen in just five to 10.
"I read this essay twice. The first time I was totally amazed: AI will change everything in science in five years, I thought!" Wolf wrote on X. "Re-reading it, I realized that much of it seemed like wishful thinking at best." Unless AI research shifts gears, Wolf warned, we won't get a new Albert Einstein in a data center -- just a future filled with "yes-men on servers."
Surprise! (Score:5, Insightful)
Re:Surprise! (Score:4, Interesting)
Most of the human inventions are also combining existing techniques in a clever way to solve existing or emerging problems.
This is very true. I was gobsmacked long ago when I referenced what I believed was cutting edge technology that I was working on, and found it was originally proposed 50 some years ago. It just had to wait until materials and working those materials caught up and enabled me to continue with a final technological implementation. Happens all the time.
Now all that said, I do have concerns AI will end up self referencing to the exclusion of human research? Governments or agenda driven groups can manipulate it, and we could end up with "proof" of things that don't exist, changing history, or many other things to suit various objectives.
Connecting disconnected ideas and concepts (Score:1)
Connecting disparate unconnected ideas and concepts is what AI does.
Re: (Score:3)
Connecting disparate unconnected ideas and concepts is what AI does.
People can do that as well. I do that all the time. Sometimes people tell me I'm hallucinating! Maybe I'm an AI artifact? 8^)
Re: (Score:2, Informative)
Also? AI isn't doing anything with regards to finding clever, i.e. novel or unexpected combinations.
Re:Surprise! (Score:4, Insightful)
Re: (Score:2)
That I don't know. A hallucinating LLM outcome could easily be interpreted as clever. For the program, it is just an outcome. It is humans that attach special meanings to those outcomes.
Mmm.. no, but in the same way you can interpret a five-year-old's drawing of a treehouse attached to your attic as clever. Making something that LOOKS like a solution to a problem is not the same thing as having a working mental model of a problem space and probing its limits.
An architect can start from an imagination of a five year old that sounds possible, and expand on a theory of what is possible. Give the kid credit for being clever, but they didn't have a working model of reality, it was more of a dre
Re: Surprise! (Score:1)
Ya, who would've thunk it?
Re: (Score:2)
Most of the human inventions are also combining existing techniques in a clever way to solve existing or emerging problems.
It doesn't matter if that most is 60, 90 or 99%, the other part is a whopper of complexity and AI that we have right now cannot do it. Many people apparently can't. This says more about us than our tools.
LLM!=AI (Score:4, Insightful)
Re:LLM!=AI (Score:5, Informative)
No. In the "protein folding thing", AI is just used as a filter, it creates nothing.
Re: (Score:3)
I am not entirely sure that is the case. Researchers have been using AI's tendency to hallucinate to their advantage. It can produce novel protein folds. The kicker is that some are impossible, some are useless, some are marginal, and some precious few might be usable....you just have to sort through the crap to get the golden droppings. My guess is that there are too many possibilities to simply enumerate them except in theory and even if you did, it is not clear which are useless, which are marginal, and
Re: (Score:2)
Well, randomization can occasionally (very rarely) create new things, but only in low-complexity scenarios. You could also just have done that randomization on the input.
Re: (Score:3)
Well, randomization can occasionally (very rarely) create new things, but only in low-complexity scenarios.
Ah, yes. Life is well known for its low-complexity.
You are, as usual on this topic, completely fucking wrong. [nih.gov]
Re: (Score:2)
> You are, as usual on this topic, completely fucking wrong. [nih.gov]
Please explain more or could you include a quote from the article.
Re: (Score:2)
Life is nothing but a series of random mutations to DNA, guided by the universe's most complicated fitness function that we call natural selection.
The idea that "randomness can occasionally (very rarely) create new things" sounds like some creationist bullshit who will argue with you about whether "macro evolution" is possible.
Randomness created all the astronomically diverse and complicated lifeforms you see around you today.
Now, FTA:
However, since AF2 is deterministic in inference mode, in AF2design the same designed sequence will be obtained for each run with the same input. To address this issue, we randomly mutate 10% of the amino acids in the starting sequence for each design trajectory, resulting in the generation of diverse sequences.
Through error gradient backpropagation combined with MCMC optimization, we explored several protocols to generate sequences given a target structure using AF2based pipelines we call AF2design. We analyzed the generated sequences both in silico and in vitro, showing that our AF2based protocol can be leveraged for de novo protein design.
Re: (Score:2)
So, before AI we did not know how hundreds of millions proteins are folded.
After AI, AI filtered out incorrect ways to fold proteins. And now we know how hundreds of millions of proteins are folded.
But even so, AI created new knowledge, because we didn't know as much about those proteins as we do now. And we would probably never know without AI.
AI is Creating Knowledge (Score:2)
In my own research group my former PhD student used Graph Neural Networks to find a subset of rare events and the paper from that is currently being written. Go look in any current particle physics journal and most of the experimental p
Re: (Score:1, Redundant)
Not really. Filtering stuff out is not "creating new knowledge". It is merely "better search" and that is basically (besides "better crap") the only actual LLM application at this time.
Filters Revealed the Higgs Boson (Score:5, Informative)
Not really. Filtering stuff out is not "creating new knowledge".
It is when the filter reveals new types of things that have never been seen before. The way filtering can create knowledge is by removing backgrounds that obscure a new signal revealing something that is unknown to science and thus creating new knowledge. The higgs boson was found this way, using filters and reconstructions that in some places used machine learning. So unless you would like to argue that finding the higgs boson was not new knowledge, machine learning can and has found new knowledge.
Indeed, if you are going to argue that filtering cannot create new knowledge then you are arguing that the Standard Model, Cosmic Microwave Background (that was compelling evidence for the Big Bang) and a whole load of other physics that underpins our modern understanding of much of science is not new knowledge!.
Re: (Score:2)
This has been argued ad fucking nauseum, but it's well established that NNs with hidden layers can encode all kinds of novel functionality in them, to the point where we can reduce any kind of formal logic to merely "a filter".
Re: (Score:2)
That is just bullshit magical thinking. The kind that happens when you want tomething so much that you stop being rational.
Yes, it is just a filter. The results were there before. It was just a bit harder to filter them out and hence not econimic.
Re: (Score:2)
Your brain is just a filter.
That inner monologue you think you have? Just filtered noise from a sea of stimulus.
Re: (Score:2)
It does not. Seriously. Stop hallucinating.
Re: (Score:2)
Re: (Score:2)
That is an entirely different claim from the one that is being made.
Re: (Score:2)
GP's claim seems... strange to be honest.
We're talking about the "creation of new knowledge", which is pretty fucking abstract.
I mean, humans didn't invent x-rays, but the experimental filter used to find them certainly produced new knowledge, no?
Seems to me that Parent's assertion that the filtration of something is indeed how new knowledge is gleaned is quite cogent.
Unless you believe in divine inspiration, no human invention didn't involve the filtration of what was apparently noise from the
Re: (Score:2)
Is it?
Obviously. An LLM producing "new knowledge" is a very different thing from, as stated above, using AI to do image classification on a massive scale.
One claim is laughable nonsense, the other is just a day at the office.
Seems to me that Parent's assertion that the filtration of something
The only thing you should be posting in a discussion about AI are questions asked in good faith. As for the "filtration" comment, while that is something we use AI to do, it is far from the only way AI is used for proteins. This is very much not my area, but from what I understand, pLMs
Re: (Score:2)
Obviously. An LLM producing "new knowledge" is a very different thing from, as stated above, using AI to do image classification on a massive scale.
There's no LLM involved anywhere in here.
AF2 is not an image classifier.
You never fail to disappoint.
Re: (Score:2)
Learn to read, troll. That was the example the parent used, which I very explicitly said was an entirely different thing.
What a waste of time you are.
Re: (Score:2)
Learn to read, troll. That was the example the parent used, which I very explicitly said was an entirely different thing.
Wrong.
You seem to be getting caught up on the clause, "While LLMs are generally useless at creating new knowledge"
And ignoring that the second clause perfectly agreed with, and refuted the GP.
I never referred to the obviously irrelevant, "While blah blah- [actual assertion we care about comes here]".
As you mentioned, the latter clause is just another day at the office, yet it is precisely the latter clause that GP is talking about, ignoring the fact that he's flat out wrong, and as I said, AF2 is not a
Re: (Score:2)
Gaslighting? Get a clue. Anyone can scroll up and see that you're completely full of shit.
Also, anyone can read your post history and see that you're not only completely incompetent, you're a liar as well. Fuck off, troll.
Re: (Score:2)
The list of shit you have asserted with absolute confidence that don't survive 12 seconds of of research is longer than this thread. You're basically a fucking meme.
You're so confidently fucking wrong that I've wondered if you aren't actually someone playing with ChatGPT.
Also, anyone can read your post history and see that you're not only completely incompetent, you're a liar as well. Fuck off, troll.
hahah- projecting much?
Re: (Score:2)
The list of shit you have asserted with absolute confidence that don't survive 12 seconds of of research is longer than this thread. You're basically a fucking meme.
I think he has dysexecutive syndrome. A while back he told me that he's tired of getting kicked in the face (his actual words) by his employers. Probably sulcal cavitation in his frontal lobe.
Re: (Score:2)
Re: (Score:2)
Further, its ability to find novel folds is [nature.com] well [nature.com] documented. [nature.com]
Re: (Score:2)
That Nobel prize winning paper does not really agree with "ai is creating new knowledge". The method described in their paper uses a neural net to perform "unconstrained protein hallucination" to create lots of data to test against a rule. That's way oversimplifying it. It's not "infinite monkey theorem" territory. I'm not a computational biologist, but just scraping the surface of that paper tells me it's not "AI creating new knowledge".
Re: LLM!=AI (Score:2)
It is literally doing tests we have not done before leading to knowledge we did not have before, there is absolutely zero logic to claiming that it is not creating new knowledge.
It's knowledge we knew how to develop before AI but didn't have the time, how is that not transformative?
Re: (Score:2)
Biochemists test millions of compounds to see if they bind to biochemicals.
When they find ones that do, we call that new knowledge.
The "unconstrained hallucination" doesn't mean "RNG".
It means the network is purposefully designed to loosen up its restrictions on probabilistic matching. I.e., it's fuzzy.
Ultimately, it means it's testing random permutations of foldings that seem plausible to its training.
Re: (Score:2)
AI absolutely is creating new knowledge.
That is not how they work. They're not little electronic brains that think and learn and grow like biological brains, they're just 'simple' functions. LLMs are trained to make predictions about the next token in a sequence. The goal of training them this way is to capture enough information from training data that we can use them to produce text that looks similar. The chat interface is just there to get you to inappropriately attribute human qualities to them. It's the same trick Weizenbaum used to make
well, duh... (Score:3)
Did anyone thing otherwise? The thing that current AIs do well, is bring together knowledge from disparate sources. Where you might enter some search terms, and look at 20+ sites to find information on some obscure topic, the AI has already consolidated that information.
AIs that can extend and extrapolate, or even come up with completely new concepts? That's going to take some serious breakthroughs.
Re: (Score:3)
Re: (Score:2)
Indeed. Somewhat better search and aggregation of existing knowledge if you are lucky. "Better crap" if not.
Fixing that would require AGI. We are not getting AGI anytime soon, despite what some liars like to claim.
Re: (Score:2)
Did anyone thing otherwise? The thing that current AIs do well, is bring together knowledge from disparate sources.
I've used it to aggregate knowledge, but the results can sometimes be better for a laugh at times.
Re: (Score:2)
Well we have AlphaFold, which alone is enough to prove that knowledge question can only be limited to chatbots, not AI in general. But that is obviously not the only AI: https://deepmind.google/discov... [deepmind.google]
We have also interesting cases where chatbot is used to create functions that can call other functions generated by AI and construct larger functions from smaller functions. The end result of these functions is an AI that can solve Minecraft faster than old systems. The AI itself didn't exactly create anythi
Re: (Score:2)
The vast majority of the laymen, and plenty of folks here on Slashdot too, are expecting AI to solve everything in the click of a finger. That's what we're being sold after all.
Duh (Score:1)
At least for LLMs. Next blatantly obvious statement?
Depends on how you use it (Score:3)
For example, you can come up with a novel gameplay concept never done before, and ask grok or chatGPT etc to describe parts of levels that use the concept, and they will "use" your concept and create pretty good ideas based on it.
That's arguably "new knowledge".
It's not because most use cases for it don't create new knowlege that you absolutely can't create new knowledge with em.
Autocomplete can't write a novel (Score:1, Flamebait)
Re: (Score:3, Insightful)
However, they're lying to investors and saying they're laying off these people due to AI productivity gains, not due to market saturation and reduced enthusiasm for their new offerings. Thus for those who don't work with LLMs, you've spent your lifetime thinking AI is what you saw in the movies, the biggest tech companies say they have AIs and thus you think they've invented Jarvis Junior...because they overtly lied and said they did.
AI is not much more than a bubble, like other bubbles, will burst at some point, and billions of dollars will evaporate overnight. I wonder how far along the renovation to restart the shuttered Three Mile Island reactors will be when that happens. If we need our very own nuclear power station to generate High school level term papers, it might be a time for introspection, not driving over the cliff to be the first to do so
Splitting the difference. (Score:2)
Splitting the difference usually doesn't work very well, but for this question I think it gives us a pretty plausible picture of what's going to happen -- indeed what *is* happening.
AI isn't going to user in a Utopia in which machines provide a superior answer to human beings can for every problem we apply our brains to. Not yet, anyway, and probably not ever. But it's not going to accomplish *nothing*. It is certainly going to change things, although not consistently for the better.
Both humans and AIs
Re: (Score:2)
AI will be used *by a small number of the most capable humans* to create new knowledge, but those accomplishments will be against a backdrop of a rising tide of computationally supercharged mediocrity.
Probably. And whether the first part of that will happen remains to be seen. The second part is a pretty solid prediction though.
Human intelligence (Score:2)
Re: (Score:3)
Lets see, Deepmind's revanue is about 1.5 billion and it was founded 14 years ago. This is way off, but upper limit is just multiplication so 21 billion.
Deepmind created many things, but one of those is AlphaFold. AlphaFold created protein folding for 200 million proteins.
Cost to fold single protein is about 120 000 dollars. So in total the work is worth about 24000 billion.
So you want to compare the 21 billion of work put into AI with the 24000 billion it would have cost to do without the AI. Humans can no
Re: (Score:3)
I'd have thought that was self evident? Guess not. (Score:2)
I see AI as a supremely useful tool (eventually) for managing the information glut we live in. I look forward to that. Ie a backward-looking tool that can (for example) pore through molecular models faster than any human to propose new credibly-possible synthetic materials.
However, right now it seems more like people are trying to crowbar it into the 'solves all problems we can't figure out' category, where it's prone to (in my experience) creating misleading (but credible-appearing) cul de sacs of inform
Dilemna (Score:2)
I've had an ongoing discussion with friends about what is a more significant waste to humanity. It's two parts.
1) Crypto Mining or AI LLMs
2) Electrical Usage by each
Both require Terawatts of power annually.
If LLMs are simply repacking old information, are they more or less beneficial than current search engines? Or
Crypto Mining, with all the energy that it consumes, just paying enough to cover the costs of generating the currency?
That's not what it's for (Score:2)
My understanding is that LLM's as currently being designed and built are for quickly gathering and distilling current knowledge.
Most of the use cases I've seen are around augmenting human effort, not necessarily replacing it.
I'm sure others may see it differently. I also anticipate that as the technology grows and matures the use cases will likewise change.
In Other Words, AI Needs To Dream (Score:3)
"Yes men on servers" (Score:3)
Perfect employees! No wonder executives are rushing to replace all of their employees with AIs. This is what they've always wanted.
Does anybody have a copy of the Twitter posting? (Score:2)
Because I thought it was very insightful when I read it on the train two hours ago. It seems Melonia has removed it now or something...
Re: (Score:2)
Sorry, meant "Elonia". It is hard to keep track of how is in bed with whom.
Re: (Score:2)
Hmm. Indirect access via https://x.com/Thom_Wolf [x.com] works. The direct link fails for me on Linux, windows and with two different browsers each.
only works on instructions? (Score:2)
Ask the wrong question, get an answer that will lead you down the garden path... to billing by the minute.
You can learn, you can "smarten up"... but many people never do.
This suggests an interesting new use for AI (Score:2)
Re: (Score:2)
They cannot experiment (Score:2)
Re: (Score:2)
Current AI cannot interact with the real world.
AI attached to the real world has been around for a long time.
If you're referring to AI as in LLMs, then that's more recent, but still exists- by varying definitions of "real world".
They're a bit on the resource intensive side to slap onto a robot by any means, but they are commonly attached to linux shells and taught how to request information from the internet.
I've played with one that builds and executes python scripts on your local machine to fulfill your natural-language requests.
As for the AI pro
New (Score:2)
How much originality does a created piece of knowledge require to be labelled "new"? For an academic thesis to be accepted, it needs some originality, but the examiners don't mind if it builds on the work of others. It would be interesting to try to define "original" in a manner that could be used to measure the originality of a new piece of work. Consider a "new" blues song - the chord structure, basic melody, lyrical subject matter could be borrowed, but nevertheless it contains something new. Johnny Cas
Alpha0 and Leela (Score:2)
Also, computer AI's are proving theorems. They are not good theorems, but they are theorems t
Well... it's complicated (Score:2)
My first thought when I read the article is that Thomas hasn't met any of my agents.
But, I mean, if we're talking the happy path of the standard use case? I have to agree with him. Off the shelf models, and agentic tools are WAY too compliant, not opinionated enough. And they behave like abuse victims. Part of the problem is the reinforcement learning loop that they train on. Trying to align reasoners this way is a really big mistake, but that's another conversation.
It doesn't have to be that way though.
Ali