OpenAI's Text-Generating System GPT-3 is Now Spewing Out 4.5 Billion Words a Day (theverge.com) 43
One of the biggest trends in machine learning right now is text generation. AI systems learn by absorbing billions of words scraped from the internet and generate text in response to a variety of prompts. It sounds simple, but these machines can be put to a wide array of tasks -- from creating fiction, to writing bad code, to letting you chat with historical figures. From a report: The best-known AI text-generator is OpenAI's GPT-3, which the company recently announced is now being used in more than 300 different apps, by "tens of thousands" of developers, and producing 4.5 billion words per day. That's a lot of robot verbiage. This may be an arbitrary milestone for OpenAI to celebrate, but it's also a useful indicator of the growing scale, impact, and commercial potential of AI text generation. OpenAI started life as a nonprofit, but for the last few years, it has been trying to make money with GPT-3 as its first salable product. The company has an exclusivity deal with Microsoft which gives the tech giant unique access to the program's underlying code, but any firm can apply for access to GPT-3's general API and build services on top of it./i
"""""Open""""" AI (Score:3, Interesting)
So Open that, unlike GPT-2, they locked it behind a paywalled API and put up politically correct terms of service that say they'll cancel your subscription if they find you doing anything too offensive. You can no longer even run a scaled-down version on your own GPUs/Titan/etc.
Re: (Score:2)
Actually, you can run a scaled down version - the model is there. The thing is, it's trained with a limited corpus. That's it - you can get the limited model. it will lack the ability to train the model with new corpus, so you're stu
Re: (Score:2)
I suppose it's coincidental that the most ethical path is also the one that monetizes their product and ensures perpetual exclusivity of their "open" research.
Re: (Score:2)
Re: (Score:1)
Gosh, I guess OpenAI would have to do whatever they were doing for the previous five successful years of their organization. Heavens me, it's a miracle Gym, RoboSumo, Debate Game, Dactyl, MuseNet, CLIP, Microscope, GPT, and GPT-2 got created at all!
Re: (Score:2)
Access to the full model is being restricted - they don't want deepfakes to be a thing until there are better ways to detect and control it.
You would think people who work with these things would know it's impossible.
Anything that can be used a a detector can be used as part of a GAN to train the model better.
Re: (Score:2)
Yeah, it's a good excuse. Deep Mind does the same kind of thing. So someone duplicated Alpha Go Zero. Someone will do the same thing to GPT3.
There's no real magic in any of this stuff.
Plagiarism? (Score:2)
Re: (Score:3)
Instead they'll know the pain of being barely literate and unable to express and communicate ideas effectively. Assuming people of the future will even have their own ideas.
Re: (Score:2)
AI psychology has already been handled by the emacs doctor for decades.
Re:If it is so powerful as they say (Score:4, Insightful)
It's not as powerful as they say. It's an Elizabot with a big database.
GIGO (Score:4, Insightful)
billions of words scraped from the internet
Without some system to test the output and feedback how can this possible be anything more the GIGO?
Re:GIGO-economy. (Score:2)
Human beings still have a future then. Being paid to correct the errors of machine learning.
Re: (Score:1)
Wasn't it Jaron Lanier who referenced a future "planet full of help desks"?
Truth and fact are already obscured... and now? (Score:3)
We all know we are living in a world where trying to get to a source of truth is almost impossible. ... lies.
The rise of AI - even relatively simplistic AI - is just going to make it even more complicated to actually know where truth
We have deep fakes that whilst are still currently reasonably easy to distinguish from reality, won't be for long.
We have AI generated content that is getting more sophisticated.
Where does this lead?
Well, given that we can't really trust news "facts" generated by humans, just further into the mire of confusion.
May as well just give up consuming all news and ... heck, tune out.
"commercial potential of AI text generation" - really, for what?
Sure, I can see some logic in training the AI text generation to hunt down "facts" for all manner of subjects and present them in AI generated white-papers, but what if they just ended up replicating other AI generated content? - some kind of vicious circle jerk of content?
AI generating deep fake videos where the fake "actors" are spouting fake content generated by AI ... mind blown.
Re: (Score:2)
May as well just give up consuming all news and ... heck, tune out.
I did that for one out of the last five years, and it made things more pleasant.
Re: (Score:2)
>"commercial potential of AI text generation" - really, for what?
Like everything else ever invented, it will be used to sell more stuff. I suspected early on, and my suspicions were confirmed, that the whole GME thing was "boosted" by bots posting hype and fud alike. That was a mere documented example, but how many more events got hyped up by "nothing" and are now considered text-book examples of How To Sell With AI?
One thing is certain, Neo-ludditism is on the rise. And as less and less trust will be pu
Re: (Score:2)
Just search for "best widgets" or "widget reviews" for any common consumer item. It's nothing but fake review sites full of amazon affiliate links. The reviews, if anyone beyond the google bot they are SEOed for were to read them, are just repetitive generic blathering about widgets interspersed with paraphrased bits of the production description from amazon. GPT-3 in action.
There some bot that takes a search term and then makes a fake review site from the top 10 hits of that term on amazon.
Re: (Score:1)
We all know we are living in a world where trying to get to a source of truth is almost impossible. The rise of AI - even relatively simplistic AI - is just going to make it even more complicated to actually know where truth ... lies.
We have deep fakes that whilst are still currently reasonably easy to distinguish from reality, won't be for long. We have AI generated content that is getting more sophisticated.
Where does this lead?
Well, given that we can't really trust news "facts" generated by humans, just further into the mire of confusion.
May as well just give up consuming all news and ... heck, tune out.
I've been there for years. It's beyond my ability to sift through the mountains of biased and just plain false information that we are constantly bombarded with in the hope of correctly identifying a few shreds of truth that are barely detectable through the noise. The Timothy Leary strategy is working well for me.
Not really (Score:5, Informative)
"any firm can apply for access to GPT-3's general API and build services on top of it./i"
I applied for access, so yes anyone can do that, but I never got it. I don't know what magic formula I need to actually gain access to it.
Re: Not really (Score:2)
Same here.
Re: (Score:3)
Why would you want access to something you call merely "an elizabot with a big database"?
Re: (Score:2)
I thought the normal Elizabot was really cool the first time I saw it. I still think it's cool.
Re: (Score:2)
The magic formula involves more dollar signs.
Magic Formula (Score:2)
Then again, my engineering background is something of a magic formula: https://en.wikipedia.org/wiki/Apprentice_(software) [wikipedia.org]
Re: (Score:2)
Nice
GPT-3-omancy (Score:1)
AI text generation is fun if you use it like a divination/fortune cookie generator or verbal surrealism game. Think of the revenue that could be generated by employing it as "psychic advice services" - it would put the "real psychics" out of a job and the customers would be none the wiser!
Re: (Score:2)
AI text generation is fun if you use it like a divination/fortune cookie generator or verbal surrealism game.
It's bloody annoying if one of your friends starts using it to make posts on, say, Mastodon, and you have to read a couple of sentences before you realise it's content-free gibberish.
Re: (Score:2)
it's content-free gibberish.
Wow, AI has come a long way, it can now accurately emulate social media posts.
Re: (Score:1)
Fair. But even still, if the AI-bot gets you to turn on Mastodon, all the better. You could be a bot and I'd never know, but right now Crack The Skye, which I started playing at your mention, has objectively improved my mood :)
Spewing Out 4.5 Billion Words a Day ... (Score:2)
Re: (Score:2)
Could be worse. One could be at a BBQ listening to a group of dads discuss the best brand of waterproofing product to use for your deck and fence. (I'm partial to Thompson's WaterSeal, it does need to be reapplied every two years but it does keep the wood from aging and going grey, especially for the softwoods used for fencing)
Feedback loops? (Score:3)
No doubt a lot of the data generated is being thrown away, but at that scale, more and more of it is going to make its way online, where it could be swept up as part of the training set for future machine learning projects. I wonder how/if that will affect things.
Straight from 1984 (Score:2)
From George Orwell's 1984:
"Presumably--since he had sometimes seen her with oily hands and carrying a spanner--she had some mechanical job on one of the novel-writing machines."
Why are we using AI to implement 1984? Don't these folks understand it's a dystopian novel, not a blueprint?
The prank I'd like to see... (Score:3)
Use this Ai to run a Twitter account for Thomas Jefferson, and then see how long it takes Twitter's own AI to cancel him for wrongthink.
Why? (Score:1)
Is anyone going to ever read them?
Re: (Score:2)
You could try. But a random word generator is faster, cheaper, and provides equally intelligible output after the first sentence.