Nvidia DGX Cloud: Train Your Own ChatGPT in a Web Browser For $37K a Month 22
An anonymous reader writes: Last week, we learned that Microsoft spent hundreds of millions of dollars to buy tens of thousands of Nvidia A100 graphics chips so that partner OpenAI could train the large language models (LLMs) behind Bing's AI chatbot and ChatGPT.
Don't have access to all that capital or space for all that hardware for your own LLM project? Nvidia's DGX Cloud is an attempt to sell remote web access to the very same thing. Announced today at the company's 2023 GPU Technology Conference, the service rents virtual versions of its DGX Server boxes, each containing eight Nvidia H100 or A100 GPUs and 640GB of memory. The service includes interconnects that scale up to the neighborhood of 32,000 GPUs, storage, software, and "direct access to Nvidia AI experts who optimize your code," starting at $36,999 a month for the A100 tier.
Meanwhile, a physical DGX Server box can cost upwards of $200,000 for the same hardware if you're buying it outright, and that doesn't count the efforts companies like Microsoft say they made to build working data centers around the technology.
Don't have access to all that capital or space for all that hardware for your own LLM project? Nvidia's DGX Cloud is an attempt to sell remote web access to the very same thing. Announced today at the company's 2023 GPU Technology Conference, the service rents virtual versions of its DGX Server boxes, each containing eight Nvidia H100 or A100 GPUs and 640GB of memory. The service includes interconnects that scale up to the neighborhood of 32,000 GPUs, storage, software, and "direct access to Nvidia AI experts who optimize your code," starting at $36,999 a month for the A100 tier.
Meanwhile, a physical DGX Server box can cost upwards of $200,000 for the same hardware if you're buying it outright, and that doesn't count the efforts companies like Microsoft say they made to build working data centers around the technology.
Oh look, why AI is mostly worthless (Score:1, Troll)
Re: (Score:2)
It only costs like one bitcoin, come on!
Re:Oh look, why AI is mostly worthless (Score:5, Insightful)
Re:Oh look, why AI is mostly worthless (Score:4, Interesting)
Yep, and it took 50 years, standards, and luck to get to this point.
It took 50 years to go from the Model T to a man on the moon. It took 20 years to go from dial-up internet to high-speed HD streaming in your home. Seems we'll never learn when it comes to underestimating the actual speed of progress.
Maybe we should learn from the past and maybe not repeat the same mistakes.
What, learn to keep corrupt levels of greed in check? Kinda doubt we humans are going to "learn" from that anytime soon. The only thing different from today and the fall of Rome, is the date on the calendar. Same humans. Same greed. Same predictable results.
Re: (Score:2)
Yep, and it took 50 years, standards, and luck to get to this point.
It took 50 years to go from the Model T to a man on the moon. It took 20 years to go from dial-up internet to high-speed HD streaming in your home. Seems we'll never learn when it comes to underestimating the actual speed of progress.
Maybe we should learn from the past and maybe not repeat the same mistakes.
What, learn to keep corrupt levels of greed in check? Kinda doubt we humans are going to "learn" from that anytime soon. The only thing different from today and the fall of Rome, is the date on the calendar. Same humans. Same greed. Same predictable results.
Same humans, but not the same tech.
Re: (Score:2)
Re: (Score:2, Insightful)
The man that broke multiple laws, gave big tax breaks to the rich, was generally self-serving, and then tried to ignore a valid election? That guy you love so much is not very upstanding, in fact he is an asshole on all fronts.
Re: (Score:2)
Its $37,000 a month to train it. That is before even using it. This is nothing but a toy for the rich.
When Chatxxx starts taking jobs by the thousands from whiny meatsacks who are always bitching about taking time off every day to sleep and needing health insurance, we might not find it so "toy" like.
Re: (Score:2)
It's funny how NVIDIA became popular by bringing 3D to the masses 25 years ago, by allowing the average joe to run cool stuff at home. How the times have changed, and the business models evolved
Re: (Score:2)
Its $37,000 a month to train it. That is before even using it. This is nothing but a toy for the rich.
$37k for even a year for a guy at home is crazy. Only rich guys can think about that. However, even for a small company, $37k/month is nothing. And for large companies, it's less than nothing.
OK, there are other significant costs besides the $37k/month, but even considering the true cost, the issue is not the money. It's whether the trained model can do something useful and moreover can do something that can be monetized. That's the real question. If the answer is that it's useless or non-revenue gene
Re: (Score:2)
Either way it is a *lot* for something that just costs $200k up front. Ignoring residual capital value, it's *still* cheaper to buy than rent in less than 6 months.
Can I use my Beowulf cluster to train ChatGPT? (Score:3)
I've got eight Pentium 90's PC's, 64MB RAM, linked with 10mbit ethernet cables. That should do the trick
Train AI to mine bitcoins? (Score:2)
Sounds like a win win situation....
"Starting at" (Score:1)