Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
AI

Generative AI's Next Frontier Is Video (bloomberg.com) 41

Posted by msmash from the pushing-the-limits dept.
AI has made remarkable progress with still images. For months, services like Dall-E and Stable Diffusion have been creating beautiful, arresting and sometimes unsettling pictures. Now, a startup called Runway AI is taking the next step: AI-generated video. From a report: On Monday, New York-based Runway announced the availability of its Gen 2 system, which generates short snippets of video from a few words of user prompts. Users can type in a description of what they want to see, for example: "a cat walking in the rain," and it will generate a roughly 3-second video clip showing just that, or something close. Alternately, users can upload an image as a reference point for the system as well as a prompt. The product isn't available to everyone. Runway, which makes AI-based film and editing tools, announced the availability of its Gen 2 AI system via a waitlist; people can sign up for access to it on a private Discord channel that the company plans to add more users to each week.

The limited launch represents the most high-profile instance of such text-to-video generation outside of a lab. Both Alphabet's Google and Meta Platforms showed off their own text-to-video efforts last year -- with short video clips featuring subjects like a teddy bear washing dishes and a sailboat on a lake -- but neither has announced plans to move the work beyond the research stage. Runway has been working on AI tools since 2018, and raised $50 million late last year. The startup helped create the original version of Stable Diffusion, a text-to-image AI model that has since been popularized and further developed by the company Stability AI. In an exclusive live demo last week with Runway co-founder and Chief Executive Officer Cris Valenzuela, this reporter put Gen 2 to the test, suggesting the prompt "drone footage of a desert landscape." Within minutes, Gen 2 generated a video just a few seconds long and a little distorted, but it undeniably appeared to be drone footage shot over a desert landscape.
This discussion has been archived. No new comments can be posted.

Generative AI's Next Frontier Is Video More

Generative AI's Next Frontier Is Video

Comments Filter:

  • We all know what this will be used for (Score:1)

    by Anonymous Coward

    Generating cute cat videos :-)

  • Thing I would pay AI to do: Translate books to English understanding that it will be better than fanTLs but worse than the pro stuff.
    Has anyone started doing this and I am not aware. Or is this something ChatGPT won't let you do because of copyright and stuff like that.


    I couldn't think of anything at first, but while type that above, I thought of something. Something that would prohibitively expensive and prolly run into oh so many legal issues, but could work for this.
    But porn. Custom porn. Anything a

  • This might just be the bane of Hollywood, because it could produce videos with better acting, better plotlines, and less BS than the big studios could do. Even if one generates 10-20 videos of the same "/imagine" text, it would be more unique and original than the same storylines coming out of US box offices in the past two decades.

    • Someone is def gonna make a fortune taking filmmaking to new lows.

      Asylum makes somewhat better stuff now but they launched a shitty movie empire right as video rental was on its’ deathbed.

      Their early films were so awful I remember spending 15 or 20 minutes watching it wondering about the motives and feelings of the actors and producers instead of following the nonsensical plot.

      “Do any of them believe in what they’re doing right now or do they all know nobody will ever watch this from begin

    • more unique and original than the same storylines coming out of US box offices in the past two decades.

      Really? More unique and original than these: Judas and the Black Messiah, Ford V Ferrari, The Irishman, Jojo Rabbit, Once Upon a Time in Hollywood, Roma, Three Billboards Outside Ebbing, Missouri, The Big Short, Whiplash, American Hustle, Dallas Byers Club, Her, The Wolf of Wall Street, Life of Pi, Moneyball, The Kings Speech, The Kids Are All Right, Inglorious Bastards, Slumdog Millionaire, Frost/Nixon, No Country for Old Men, The Departed, Letters from Iwo Jima?

      Sorry, I don't see generative AI coming up

      • Re: (Score:2)

        by vux984 ( 928602 )

        Yeah, but it could probably manage something as good as the Marvel cinematic universe. Just copy paste stuff from the last few, update costumes, and rearrange a few things. Don't worry about it making any sense.

  • Oh yeah! (Score:4, Interesting)

    by LondoMollari ( 172563 ) on Monday March 20, 2023 @02:46PM (#63385769) Homepage

    Season 4 of Star Trek: TOS coming right up! With the original actors, too.

    • The good news is they are done! The bad news is they had to morph it with Gilligan's Island, Season 4 to shave costs.

      • The good news is they are done! The bad news is they had to morph it with Gilligan's Island, Season 4 to shave costs.

        We've already got that [imdb.com].

        • Re: (Score:1)

          by Tablizer ( 95088 )

          Oh yes, I forgot about that. One of the best theme songs in my opinion. It's already ear-worming me.

          Cheap-ass production, though. It's as if they hired a bunch of 6th graders to put aluminum foil and cooking utensils on everything to make it look spacy.

        • Re: (Score:2)

          by dgatwood ( 11270 )

          The good news is they are done! The bad news is they had to morph it with Gilligan's Island, Season 4 to shave costs.

          We've already got that [imdb.com].

          I'm so disappointed. I felt sure you were going to link to this [imdb.com].

          • The good news is they are done! The bad news is they had to morph it with Gilligan's Island, Season 4 to shave costs.

            We've already got that [imdb.com].

            I'm so disappointed. I felt sure you were going to link to this [imdb.com].

            Clever take! But no Bob Denver...

    • And all creepy A.F., if it's anything like the other generated content I've seen.

    • Re: (Score:2)

      by AmiMoJo ( 196126 )

      I've been dreaming about that for decades. Another season of Firefly, a few more of Stargate Universe, a Stargate Atlantis movie...

      Even if it needs some actors to provide voices, and some manual clean-up of the output, we could finally get satisfaction on some many things.

      My only worry would be that if you remove the actors from the process, it will lose something. Actors bring a lot to a show. Maybe they could mo-cap some of it, use AI to fill in, change voices etc.

  • Do an FFT on each of them. Keep the primary points and zero out the rest. Do the same for the color pallet. Average the results. Add in a measured amount of randomness to make it unique. Map it onto a generic 3D model of "flight over landscape".

    Boom "AI-generated video omg everyone shovel $$$$ at me".

    Yes, I'm jaded and cynical. Get off my lawn. I'll act all surprised then chatGPTv26 comes and takes my job.

  • May make that "genre" even more generic though.

  • I would think that they would use AI to "up-sample" video, first.

    You can currently take a crude stick-drawn picture of a guy standing by a hillside and have the AI generate photo-realistic scenes based on the drawn picture.

    I don't see why you wouldn't start in this territory with AI video generation, first - it largely satisfies the difficult task of figuring out frame composition and such, and allows for the AI to just stitch together incremental changes based on a preconceived image.

    It seems like it'd be

  • I can finally make Jar Jar Strikes Back without selling my house. (Had a hard time getting investors, for some reason.)

    • Re: (Score:2)

      by dgatwood ( 11270 )

      I can finally make Jar Jar Strikes Back without selling my house. (Had a hard time getting investors, for some reason.)

      I definitely do NOT want to see the Rule 34 version of this....

  • The next frontier for AI (Score:3)

    by Jeremi ( 14640 ) on Monday March 20, 2023 @04:13PM (#63386055) Homepage

    ... is a service that will take as input the first paragraph of a news article and generate for you the rest of the article that lies hidden behind the paywall.

  • ... that I can give an AI the ebook of, say, Pandora's Star by Peter F. Hamilton (first half of an excellent, mind-blowing SF duology) and ask it to adapt it as a 25 hour movie including all dialogue, and the specific request that main characters are played by young Michelle Yeo as Paula Myo, etc. Wouldn't that be fabulous? And the end of Hollywood of course.

Slashdot Top Deals

GREAT MOMENTS IN HISTORY (#7): April 2, 1751 Issac Newton becomes discouraged when he falls up a flight of stairs.

Close