OpenAI Says It Has Begun Training a New Flagship AI Model (nytimes.com) 40
OpenAI said on Tuesday that it has begun training a new flagship AI model that would succeed the GPT-4 technology that drives its popular online chatbot, ChatGPT. From a report: The San Francisco start-up, which is one of the world's leading A.I. companies, said in a blog post that it expects the new model to bring "the next level of capabilities" as it strives to build "artificial general intelligence," or A.G.I., a machine that can do anything the human brain can do. The new model would be an engine for A.I. products including chatbots, digital assistants akin to Apple's Siri, search engines and image generators.
OpenAI also said it was creating a new Safety and Security Committee to explore how it should handle the risks posed by the new model and future technologies. "While we are proud to build and release models that are industry-leading on both capabilities and safety, we welcome a robust debate at this important moment," the company said. OpenAI is aiming to move A.I. technology forward faster than its rivals, while also appeasing critics who say the technology is becoming increasingly dangerous, helping to spread disinformation, replace jobs and even threaten humanity. Experts disagree on when tech companies will reach artificial general intelligence, but companies including OpenAI, Google, Meta and Microsoft have steadily increased the power of A.I. technologies for more than a decade, demonstrating a noticeable leap roughly every two to three years.
OpenAI also said it was creating a new Safety and Security Committee to explore how it should handle the risks posed by the new model and future technologies. "While we are proud to build and release models that are industry-leading on both capabilities and safety, we welcome a robust debate at this important moment," the company said. OpenAI is aiming to move A.I. technology forward faster than its rivals, while also appeasing critics who say the technology is becoming increasingly dangerous, helping to spread disinformation, replace jobs and even threaten humanity. Experts disagree on when tech companies will reach artificial general intelligence, but companies including OpenAI, Google, Meta and Microsoft have steadily increased the power of A.I. technologies for more than a decade, demonstrating a noticeable leap roughly every two to three years.
Training it on what ? (Score:1, Funny)
Training it on what ? last i heard tech bros don't create content
Re: (Score:2)
Re:Training it on what ? (Score:5, Interesting)
From what I hear, they seem to be finetuning it with a steady flow of sweatshop labor creating answers to trivia and programming problems. I imagine that companies will realize it's still pretty stupid for their difficult problems.
Re: (Score:2)
Probably the Reddit data that was provided to them. Get ready for /r/samepicofsteveharvey to answer all your questions.
By the time itâ(TM)s ready⦠(Score:1)
Question is (Score:2)
Re: (Score:2)
LLMs won't lead to AGI (Score:5, Insightful)
Re:LLMs won't lead to AGI (Score:5, Interesting)
All predicated on the hypothesis that there is a point of escape velocity (the singularity) for AI. Just have to nudge the ball over the edge of the cliff and gravity will do the rest.
I don't happen to believe this will be the case, but we do have examples in nature (like the formation of the eye). Intuitively you would think that even if one of these LLMs get's "good enough" to be able to produce an incrementally better LLM on it's own, that wouldn't be the fundamental change necessary to unlock new modalities.
But as we have seen, sometimes these models hallucinate answers that are logically consistent but based on imaginary/unknown (and most often incorrect) fundamental elements. Introduce enough of these and it seems technically possible to produce something new; in the same way it's possible for a money to reproduce Shakespeare by randomly banging on a keyboard.
Re: (Score:3)
it seems technically possible to produce something new; in the same way it's possible for a money to reproduce Shakespeare by randomly banging on a keyboard.
Freudian typo, or self-referential monkeying?
Norman, Coordinate (Score:3)
Intuitively you would think that even if one of these LLMs get's "good enough" to be able to produce an incrementally better LLM on it's own.[...] Reproduce Shakespeare by randomly banging on a keyboard
Right now, it cannot synthesize a working pizza recipe or design for a mechanical part. Or anything else. The LLM is not a million monkeys banging, but even if it were, how would you test that the output (daughter LLM) is what you wanted?
For one thing, there are only so many actual monkeys with which to review the results and train the model. And if you could automate that, wouldn't that imply you had solved the original problem already? (This is similar to the problems of detecting AI-authored material, an
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
So much of the current AGI talk feels like a balloon operator saying: "Last year, my balloon reached one mile above sea level. I'm aiming for ten times that now. Next year, I will get to space, and then I'll visit the moon!". Good luck.
Interesting analogy. Of course, the real question is a matter of scale. The AGI-optimists are thinking that the balloons currently reach 10 feet, but we can reach 1000 feet with more equipment, then eventually we'll reach a mile. The two really big questions are (1) whether the point of AGI occurs within the stratosphere or beyond and (2) whether a balloon at the top of the stratosphere could be modified and augmented to reach the moon. After all, today's AI models are not simply the same models from de
appeasing (Score:2)
'robust debate', like how to avoid terminator scenarios.
If there is anything "dangerous" about this tech, it's the users dumping shit on the internet like it's their job (it is), which is going to poison future data.
"OpenAI also said it (Score:3)
Re: (Score:2)
was creating a new Safety and Security Committee to explore how it should handle the risks posed by the new mode"
I could have sworn they recently disbanded a safety committee..
https://www.axios.com/2024/05/... [axios.com]
General Intelligence will be ARTIFICIAL all right (Score:5, Interesting)
The usual trivia, revenge posts, put-downs, etc from social media.
Influencer B.S. & media.
Books of a bygone era.
Electioneering verbiage from the dozens of places having elections this year.
Propaganda from the various sides in the many wars going on, etc.
. Surely the Internet is the LAST PLACE one should look trying to make sense of the world. Yet that willbe what their model will be trained on. Will they isolate qualities like: Politeness; discretion; consideration; unselfishness? I don't think so.
Re: (Score:1)
Word don't exist in a vacuum. Words are a product of the universe that leads to their creation. To get good at predicting words, you have to get good at modeling said underlying universe.
Re: (Score:2)
Start-up? (Score:4, Insightful)
An 8-year old company valued at $100 billion is still a start-up??
Things I'm looking out for: (Score:5, Interesting)
Q* (A* Q-learning, aka iterative tackling of possible routes to solve a problem via perceived distance from the solution)?
Massively MoE, with cos distance between hidden states feeding back to provide the model with an assessment of its own confidence?
More "distributed" models where portions are more decoupled from others and don't need as much bandwidth, to make it easier to train and run on cheaper hardware?
Will we finally see a lower O() context memory model? Could we at maybe close in on O(logN)?
Non-Transformers based (Mamba, etc)? Doubt it, but possible.
PCN-based with neuromorphic hardware? I *really* doubt that, but it would be an utter game changer.
Will surely be highly multimodal from the start. I wonder what the full modality list will be, though...
Not everything is news worthy (Score:1)
Of course the new model will be an improvement (we hope). :)
They must know the right buzz words to have AI bots publish stories about them
Finally Caught (Score:1)
Re: (Score:2)
I'd like my brain to do what the human brain can do.
stop the presses..sam scratched his ass... (Score:4, Insightful)
How else are we going to make an announcement?!
We're the nukular fusion of AI. 10 years away from
SNORE.... zzz...
I don't feel that well... I think I ate glue on my pizza....
Close the door behind you (Score:2)
Wouldn't it be funny if all the BS and scaremongering backfired? The feds just waltz in, shut down Open AI and take all their shit for playing with "nuclear weapons".
AGI (Score:2)
An AGI that can do everything a human can.
Except THINK, of course.
These models have no reasoning (or facts with which to reason).
Only statistics about word sequences.
Given how much actual humans do all the time without any "thinking", I am sure this will be useful. Given how much info-pollution it will cause, and all the fantastically inappropriate contexts in which it will be deployed, I am sure it will be an existential disaster.
Re: (Score:2)
These models have no reasoning (or facts with which to reason). Only statistics about word sequences.
That is one way of saying that you don't know anything about the topic at hand.
With those "statistics about word sequences" I can take a photo of a Finnish language high school matriculation exam physics problem (with long verbal description on what needs to be solved) - and those statistics will a) do image recognition to parse all text and mathematical symbols b) parse finnish language to sufficiently understand the problem to be solved c) do the actual problem solving, giving all calculation and reasonin
Re: (Score:1)
Or more typically, it will respond a seemingly correct analysis and answer for a basic math problem while being completely incorrect. Saw this today: https://twitter.com/cleaner_ed... [twitter.com]
Re: (Score:2)
These models have no reasoning (or facts with which to reason).
Only statistics about word sequences.
That is one way of saying that you don't know anything about the topic at hand.
With those "statistics about word sequences" I can take a photo of a Finnish language high school matriculation exam physics problem (with long verbal description on what needs to be solved) - and those statistics will a) do image recognition to parse all text and mathematical symbols b) parse finnish language to sufficiently understand the problem to be solved c) do the actual problem solving, giving all calculation and reasoning steps and reaching final, correct, result for that assignment.
That may be partly true, except mostly it doesn't get the correct answers. (Moreover, it most certainly has been trained on those exact high school math and physics problems. Or ones that sound just like them. What gave you the idea that it had not been trained on high school textbooks?)
Re: (Score:2)
With those "statistics about word sequences" I can take a photo of a Finnish language high school matriculation exam physics problem (with long verbal description on what needs to be solved) - and those statistics will a) do image recognition to parse all text and mathematical symbols b) parse finnish language to sufficiently understand the problem to be solved c) do the actual problem solving, giving all calculation and reasoning steps and reaching final, correct, result for that assignment.
That may be partly true, except mostly it doesn't get the correct answers. (Moreover, it most certainly has been trained on those exact high school math and physics problems. Or ones that sound just like them. What gave you the idea that it had not been trained on high school textbooks?)
Have you actually experimented yourself, with e.g. ChatGPT 4o model? You are saying that an experiment I did and described is "partly true" - which part would you claim to be false? And, based on (again, my own) experiments, the answer it gets are most often correct (not always).
Of course it has been trained on textbooks - what I wrote is that it has not seen those particular assignments I experimented with (they need to be new and unique each year, as it is a country-wide exam for graduation). I.e. the mod
Re: (Score:2)
Re: (Score:2)
That is one way of saying that you don't know anything about the topic at hand.
[...]With such things already now possible with "statistics about word sequences", I don't think we will need AGI at all, as sufficiently advanced statistics will probably suffice..
Every day since they were brought to the public, all of these systems fail spectacularly at all kinds of problems, including the ones you are claiming. However, what the models *can* do is impressive.
Your last comment above raises the question of whether we will "need AGI" because the LLMs will "suffice". Suffice for what "needs"?
I'd say for things that don't really matter, like getting your travel plans totally fucked up, the LLMs will "suffice". It will certainly eliminate many humans who have that job to
Controlling Information, maybe how you think (Score:2)
These “AI safety” committees smack of censorship. Since when in American history have we argued that adults not be allowed information - and I’m not talking classified government secrets-type information. I’m talking anything you can find in a library. But right here we have a model that can consume all of humanities publicly published information and not give you answer because it’s “not safe.” Or not analyze published statistics because the answer “impact
Where's my jetpack ? (Score:1)
Hinton Prompt that made me think AI understands (Score:2)
Watching an interview with Geoffrey Hinton ("The Godfather of AI" Google DeepMind etc) He says some prompts just don't seem like some kind of super advanced autocomplete (stochastic parrot).
He asks GPT-4 : 'I want all the rooms in my house to be white, at present there are some white rooms, some blue rooms and some yellow rooms and yellow paint faints to white within a year. So what should I do if I want all the rooms to be white within 2 years?' The answer GPT-4 gave was: 'You should paint the blue rooms y