Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
AI

OpenAI's New Language Generator GPT-3 is Shockingly Good -- and Completely Mindless (technologyreview.com) 57

Posted by msmash from the moving-forward dept.
"Playing with GPT-3 feels like seeing the future," Arram Sabeti, a San Francisco-based developer and artist, tweeted last week. That pretty much sums up the response on social media in the last few days to OpenAI's latest language-generating AI. From a report: OpenAI first described GPT-3 in a research paper published in May. But last week it began drip-feeding the software to selected people who requested access to a private beta. For now, OpenAI wants outside developers to help it explore what GPT-3 can do, but it plans to turn the tool into a commercial product later this year, offering businesses a paid-for subscription to the AI via the cloud. GPT-3 is the most powerful language model ever. Its predecessor, GPT-2, released last year, was already able to spit out convincing streams of text in a range of different styles when prompted with an opening sentence. But GPT-3 is a big leap forward. The model has 175 billion parameters (the values that a neural network tries to optimize during training), compared with GPT-2's already vast 1.5 billion. And with language models, size really does matter.

Sabeti linked to a blog post where he showed off short stories, songs, press releases, technical manuals, and more that he had used the AI to generate. GPT-3 can also produce pastiches of particular writers. Mario Klingemann, an artist who works with machine learning, shared a short story called "The importance of being on Twitter," written in the style of Jerome K. Jerome, which starts: "It is a curious fact that the last remaining form of social life in which the people of London are still interested is Twitter. I was struck with this curious fact when I went on one of my periodical holidays to the sea-side, and found the whole place twittering like a starling-cage." Klingemann says all he gave the AI was the title, the author's name and the initial "It." There is even a reasonably informative article about GPT-3 written entirely by GPT-3.
This discussion has been archived. No new comments can be posted.

OpenAI's New Language Generator GPT-3 is Shockingly Good -- and Completely Mindless More

OpenAI's New Language Generator GPT-3 is Shockingly Good -- and Completely Mindless

Comments Filter:

  • Bot World (Score:4, Interesting)

    by LionKimbro ( 200000 ) on Monday July 20, 2020 @02:23PM (#60311619) Homepage

    For the time being, you can assume that this comment was written by a human being. You can click on my username, look back at my history of posts, and go, "OK, here's a bunch of posts, by a person, going back more than a decade, to the TIME BEFORE BOTS." That is, before the first year of 2020.
    But I don't think this will persist for much longer.
    We've been able to tell, from reading a post by someone, that: "OK, this is written by a human being." That time period is closing.
    With robotic voice synthesis, the sensible speech patterns will be augmented by realistic human voices.
    It will happen that, when we get a phone call from somebody we do not know, we will not be able to tell, "Is this a human being or not, speaking at me?"
    I suspect that online propaganda armies will use this to devastating effect.
    I mean we already have the problem of knowing whether it is Russian trolls, or Corporate propaganda, or what have you, working to shift and alter opinions, to suppress unwanted speech, and to bolster speech patterns that somebody wants everybody to hear and to think.
    This is just going to further that.
    I strongly suspect that some kind of "This is a human being" authentication system will have to be developed, and utilized.
    Webs of trust will have to be established.
    I don't know -- is this crazy talk?
    I had already been starting to think this was necessary, because of paid troll armies and astroturfing and such.
    Where are we going with this?

    • Why does it matter if it's written by a human? If it's relevant content, it shouldn't matter.

      If all goes well, what this will lead to is the devaluation of human generated content. Then maybe people won't monetized for going online and sharing their un-expert opinion.

      • Re: (Score:2)

        by vadim_t ( 324782 )

        Well, for one that it's going to be used for extremely misleading purposes.

        If we had an AI with an independent mind and volition that watched movies and produced its opinion on them, that would be interesting.

        What we'll get first instead is a text generator that doesn't watch movies, doesn't perform any kind of critical thought, but that generates reviews mostly indistinguishable from an average, slightly stupid humans, while giving the movie whatever rating the controller of the bot thinks it should have.

        S

        • It's worse than that. Imagine the value of a bot that can be set loose on a social media and automatically generate human-like responses shrilling a desired product or political cause. One day the internet might consist of a trillion bots in a constant and futile effort to convince each other which political party to support.

        • It doesn't matter if the review is in video format. Not least of which, because fake video people technology is coming.

          No, what matters is that your human or AI or council of wizards or whatever is publishing their reviews under a consistently respected name.

          In a perfect world, this leads us back to the older trusted gatekeeper model of information and away from the click-driven content vomit model of information.

    • For the time being, you can assume that this comment was written by a human being.

      As of Wednesday, 15 July 2020 [wikipedia.org], all bets are off regarding the authorship of Internet posts.

      At least, that's what Elon Musk is planning on telling the SEC.

    • How sweet of you, to imply that "human" means "individual" and "self-thinking". :)

      • No but the economics change drastically with AI. Currently it takes a lot of resources to create convincing astroturf, such as paying people to write product reviews. If anybody can do it, things would become quite confusing.

    • There is no real difference, trust-wise, between being contacted by a malicious bot vs a malicious human agent. The bot is (currently) still at a disadvantage since it has to not only convince you of whatever lie it's selling, but also has to not be discovered as a bot. (Sort of the same hill that "Thomas" has to climb when he calls with a thick accent and tries to tell me he's calling from New Jersey). Adding to that, troll farms have shown us the cost of human-produced disinformation is shockingly low. So

    • It will happen that, when we get a phone call from somebody we do not know, we will not be able to tell, "Is this a human being or not, speaking at me?"

      Does anyone answer those calls anymore?

      • Oh yeah. I tar pit them as best I can. "My credit card? Yes, I have a blue one and also a couple yellow cards. Or are they gold? Can you tell me?"

        More recently, the calls hang up as soon as I say "hello." I'm wondering if somehow my voice has been fingerprinted, but my phone number has not. It's a weird thing. Even before I can say anything sticky.

        • I'm wondering if somehow my voice has been fingerprinted, but my phone number has not.

          No, the autocaller is just dialing random numbers and collecting a list
          of those that respond somehow. They then sell these lists to scammers.

    • Maybe it's going to get too the point thayt spelling and grammar errers will bee the only way thaht posts can bee verufied as ritten buy a human.

      Of course, the writing-generation software will eventually get hip to this as well.

    • Telling the real from the fake is a fundamentally easier problem than generating fakes.

      We might not be able to tell real from fake with our brains alone, but we will still be able to do it with our tools.

    • Re: (Score:2)

      by Kiliani ( 816330 )

      As long as AI cannot emulate a reasonable life experience I am not worried too much. Sometimes it may be better to talk to a "smart" AI system than to a "limited" human operator, or their supervisor. Will people get duped by AI? Of course. Many people are gullible or stupid (bell curve anyone), so be it. Hey, Trump was voted President - presently scares me more than AI - and was all done with good old analog human "intelligence".

      Now, will there be ways for a person to identify whether their opposite is a hu

      • Imagine you wanted to tank a community though.

        Fire up a million users (or a few new users every day from now till forever) and have them shit-post random crap. Inane bullshit vaguely on topic. Human enough to be indistinguishable from the unwashed dirty masses of people who post online. While I know Slashdot us more or less used to this, the signal to noise ratio will plummet and no one will bother wading through a giant pile of otherwise undetectable bullshit. You're imagining in-depth conversations

    • Obviously the parent post was written by a bot.
      Symmetrical sentence stacking.
      No human being would stack sentences like this.

    • I would be worried if the bots began to refer to articles they or other bots had generated in the past. They could simulate having an existence and shared experiences.

    • Re: (Score:2)

      by nagora ( 177841 )

      For the time being, you can assume that this comment was written by a human being.

      It doesn't seem like it was written by a human being. It reads more like something from the marketing department.

  • The best Lorem Ipsum ever! (Score:3)

    by reanjr ( 588767 ) on Monday July 20, 2020 @02:39PM (#60311707) Homepage

    Now layout designers won't have to resort to repetitive Lorem Ipsum generators for content examples.

    The future is here.

  • Is he really that poor of a writer?

    • Re:I've never read Jerome K. Jerome (Score:4, Informative)

      by guruevi ( 827432 ) on Monday July 20, 2020 @02:51PM (#60311761)

      He's actually a very good writer, the UK version of Mark Twain really.

      If you want a brief introduction: The New Utopia is a short story (a few pages long) that give you keen insight into his thinking, his humor and why he's not very popular in the media.

    • Re:I've never read Jerome K. Jerome (Score:4, Informative)

      by andrewbaldwin ( 442273 ) on Monday July 20, 2020 @03:13PM (#60311879)

      He's very good. Wrote around the turn of the 19th/20th century but a lot of his work still resonates today.

      Get a copy of "Three Men in a Boat" (available on Gutenberg) - depending on your tastes, it is a very funny read with many observations on life thrown in. It's one of the few books that I've re-read multiple times (usually about every 2 or 3 years) and still find engaging.

      A sequel (set a few years on but with the same main characters) "Three Men on the Bummel" is also worth a read - even though European geo-politics have changed significantly since it was written.

  • Just like 99% of everything on the internet.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Indeed. You have to question whether AI will reach the level of "intelligence" of an average person, because the average person does not really have general intelligence in the first place...

  • Love it when marketers tack the "open" prefix onto closed proprietary black box.

  • 90% of slashdot posts are clearly written by a semi-random text generator/bot.

    The rest is 50 centers posting from China trying desperately to propagandize the bots. And failing.

    Eliza is the earliest talking computer bot I know of. There were many others since then. This is the same thing as before just bigger because hardware is cheaper. Is there anything new in this version besides more "parameters" which really just means more hardware available?

  • A core of pure cancer. (Score:3)

    by BAReFO0t ( 6240524 ) on Monday July 20, 2020 @02:59PM (#60311805)

    - "AI"
    - "The importance of being on Twitter"

    That really says it all, about the personality of the creator of this bloating spaghetti code... no, *blended* spaghetti puree is more fitting ... universal function implementation.

    I do not want there to be a single overlapping entity or event between whatever dystopic universe/community those people are from and my world.

    Kill it with fire. Then nuke it from orbit. Then gamma ray burst its entire star system. Then collapse all quantum fields down, once before, once during and once after letting its entire universe being swallowed up by a black hole. I want time and space and the universal constants of nature to lose all meaning wherever it is banished to! Make Nyarlathotep be scared of that place! Make Twinkies rot!

    • Re:A core of pure cancer. (Score:5, Funny)

      by Areyoukiddingme ( 1289470 ) on Monday July 20, 2020 @04:03PM (#60312125)

      Kill it with fire. Then nuke it from orbit.

      We can do that.

      Then gamma ray burst its entire star system.

      Once we're a Kardashev Type II civilization, we can do that.

      Then collapse all quantum fields down, once before, once during and once after letting its entire universe being swallowed up by a black hole.

      Once we're a Kardashev Type III civilization, we can do that.

      I want time and space and the universal constants of nature to lose all meaning wherever it is banished to! Make Nyarlathotep be scared of that place!

      Once we bring back the Great Old Ones, we'll have done that. (Probably by accident.)

      Make Twinkies rot!

      Now you've gone too far! There is no imaginable time when humanity will have the technology to do that.

  • there should be a doll that a kid feeds a short paragraph too and the then doll completes a story for the child. i would like to see that

    • You do not want to get GPT-3 output anywhere near a child, trust me. It's basically trained on the entire internet.

  • Deep fake information coming our way (Score:3)

    by xonen ( 774419 ) on Monday July 20, 2020 @03:36PM (#60311997) Journal

    Some of those examples are really convincing. Especially that it can elaborate on a theme and create real long and coherent stories. Or that it can take styles, create rap songs, novels, plays. It's very interesting to see what huge models can do.

    Can't wait for someone to suggest to replace politicians by AI (see random SF story) since the AI gives the illusion of having a deep understanding of reality en knowing a lot of history. Ray Kurzweil might agree. And then wonder if that's really such bad idea because it's pretty hard for an AI to fek up the world worse than we are doing right now. Either way, in between propaganda and anti-propaganda expect to see deep fake propaganda anytime soon.

    • That was the orginal ending of I, Robot. The final chapter is set at a meeting of engineers as they uncover a global conspiracy by the artificial intelligences to take over the world and rule over humanity. After much discussion, they realise the AIs are are smarter than any human, have no desire for personal wealth or status, and by design are incapable of doing anything other than act in the best interests of mankind. Better to just ignore the whole conspiracy and let the machines win.

      • Re: (Score:2)

        by ceoyoyo ( 59147 )

        And then the AIs travel in time and kill all the aliens.

      • Systems are almost good enough now. You have an issue that you would like to raise with a politician, chat with his bot. The bot will be pleseant and agreeable if non-committal.

        And the exercise is not a complete waste of time. The bot will compile statistics, test ideas with you, and that will be summarized and drive policy decisions.

        At first the bots will be a bit of a joke, and no one will take them or their advice very seriously. But as they get better, their access to a wide variety of conversations

    • Can't wait for someone to suggest to replace politicians by AI (see random SF story) since the AI gives the illusion of having a deep understanding of reality en knowing a lot of history. Ray Kurzweil might agree. And then wonder if that's really such bad idea

      So you could end up with an AI Mitch Mcconnell or or Alexandria Ocasio-Cortez... seems like it would be consequential to the extreme as to what the learning model's lean was.

      I'm not even saying you're wrong. I am saying that this could go either way,

  • Every essay in the future will be awfully artistic and pompous.

    AI, please write me an essay on (whatever) in Frasier Crane style.

  • Catcher in the rye; Grapes of wrath; etc. etc.

  • This is truly astonishing. A human comparable artificial neural network would have somewhere in the order of 100 billion parameters. This entails they've trained a model of comparable complexity to that of a human brain. Kurzweil would say this is right on cue - Singulatarians have been predicting human level AI by 2020. I still think the harder problem of how to do AGI is still to be cracked, but I had no idea we were this close on the easier problem of scaling this stuff up.

  • That is the world we are headed toward.
    Arguably, the vast majority of human created fact, whether that be a news report, a whitepaper or a wiki entry, cannot be verified by the vast majority who may read it. There's an implicit level of trust required, you tend to pick your sources and try to find other sources on the same subject, to attempt to verify what you are consuming.

    If we now enter a world where more and more content is generated by AI, to the point where most people - if not all - cannot even tell

    • Could you end up following and trusting an entirely fake entity? - a construct?
      Sure, easily.

      A concept without which Hollywood would cease to exist.

      • Well sure, but that's supposed to be fiction - but then, the lines between fact and fiction are so damn blurred these days.

        • Well sure, but that's supposed to be fiction - but then, the lines between fact and fiction are so damn blurred these days.

          Facts + Fictions = Factions

  • I can't believe that Jerome would have used the word 'periodical' (a magazine or newspaper published at regular intervals) in a context where the word should obviously be 'periodic' (appearing or occurring at intervals). He was too good a writer.

  • OpenAI's New Language Generator GPT-3 is Shockingly Good -- and Completely Mindless

    In other words, very much like journalists?

    There is even a reasonably informative article about GPT-3 written entirely by GPT-3.

    Yep: a drop-in replacement..

    Although, what a hack may view as 'reasonably informative' might register as 'mildly entertaining' or even 'boring' on some reader's scale.

Slashdot Top Deals

The brain is a wonderful organ; it starts working the moment you get up in the morning, and does not stop until you get to work.

Close