Forgot your password?
typodupeerror

Submission + - Why AI token prices are about to plummet (businessinsider.com)

ZipNada writes: The main force driving token prices lower is a new wave of technology that's sweeping through AI data centers.

Nvidia's Blackwell GPUs are being installed in huge volumes right now. By the second half of this year, these systems, which are really supercomputers rather than chips, will be operating at scale, helping AI labs train new models and run them more efficiently.

These systems took a while to install properly, partly because they needed to be water-cooled and required other gnarly new data center setups. But the payoff could be huge.

50 x more, 35 x cheaper
SemiAnalysis, a respected AI research firm, compared Nvidia's top Blackwell system, the GB 300 NVL72, to Nvidia's previous system, called the Hopper HGX 200.

With the older system, each GPU generated 90 tokens per second, while the new Blackwell system generated 6,000. That's 65 times more.

Comment Re:No, they didn’t (Score 1) 96

>> "Build it and they will come" seems to have been the entire planning of how to power and cool these centers.

Bullshit. The local data centers here in Texas underwent an extensive review process. They are sited near high voltage lines and get power directly from there through power purchase agreements. They use closed-loop cooling.

There's a lot of local opposition to them but it has been pumped up by misinformation similar to what you are repeating. One data center in Truckee California may have bought all the local electricity but that doesn't apply to the entire industry.

Comment Re:Blocks ..... (Score 1) 96

>> take up local water at below cost

I did some checking on the local data centers here in Texas. The use closed-loop cooling, and therefore use very little water. There's no sign that they have caused electric bills to rise. They are sited in designated industrial zones or on the outskirts of town where noise will not be a problem.

>> What benefit does it provide them???

They pay taxes. One of them I looked at will be paying about $7 million over the next 10 years.

Comment good competition (Score 1) 36

The AI companies are competing for market share at present, but it isn't very surprising to see prices for the top models ramp up as they become more complex and use more compute resources. But there's a long trail of older, lesser models still available and some of them are pretty good. My suspicion is that casual users can do just fine with relatively inexpensive mid-or-low-tier models.

I use AI most every day for technical work (to great benefit, I'm often amazed). A couple of months ago the vendors started rationing the token consumption as they reach the limits of their existing compute infrastructure. Now I have to adjust the models I use based on what I'm trying to accomplish at the moment in order to economize. The hard stuff goes to models that are very good but at least a generation behind the very expensive cutting edge. Things that are more mundane can be allocated to older and cheaper models. Sometimes they struggle with a task and I have to bump up to something better. Yesterday I used up my daily token quota after a few hours of very productive work, a bummer but its understandable.

Comment Re:NIMBY CEO says what? (Score 1) 87

>> I can't imagine why.

It's because there's a vast amount of cheap, available land there. Data centers are generally built out in the boonies, but near enough to a town that the employees can have a decent place to live and shop. Adjacent to a hefty power line is also a requirement.

Comment what we call 'race to the top' (Score 2) 83

We're seeing rapid new generations of these AI systems now. New versions with even more impressive capabilities are coming out every 2-3 months and sometimes they are a significant step change.

The 'frontier' models we are seeing now will be nothing special in 6 months. There's a trail of somewhat lesser products racing to catch up, and at the current velocity they will reach this scary level of capability within a few months. It's hard to see how there can be any sufficient guardrails. I hope we can adapt.

Comment Re:the problem is (Score 1) 27

I'm not 'proposing' a distributed and crowdsourced faceprint database. I'm merely pointing out that something like that could be done by a reasonably knowledgeable person using widely available tech. With AI assistance I could probably come up with a cellphone app coupled to a cloud server that does the basics. It's simple enough that it is not preventable.

You can argue that accurately assigning face prints to human identities could be spoofed without appropriate safeguards, but that's a different issue.

Comment the problem is (Score 3, Interesting) 27

The problem is that this isn't very hard to do these days. It's pretty near impossible to prevent things that are easy to do.

The Meta device is constantly getting a stream of image frames from the camera in the glasses. Probably their device has enough compute horsepower to detect human faces, smartphones sure do. The faces can easily be cropped out of the images and passed along to whatever recognition system you happen to have on hand to develop a faceprint. It all goes into a database, local or remote, and then its a SMOP (simple matter of programming) to correlate a faceprint to a human identity. Gather all of that into a central database and presto.

You could just wander around with your cellphone in your shirt pocket recording everything and an app there could do much of this. Meta is getting some pushback because they are so visible and pervasive, but smaller players could definitely implement a mobile facial recognition system under the radar and probably have.

Comment The era of effortless documentation (Score 1) 86

I get the impression that many of the commenters here are still writing their own code, which they may or may not document. And if they do the documentation quickly becomes obsolete. Like it or not, that mode of working is now obsolete.

These days the savvy programmer starts out by describing what he wants and tells the AI (LLM) to generate a detailed proposal. When you're satisfied with that you tell it to create a phased implementation plan with milestones. The AI works through the plan step by step. After each milestone is reached you tell it to update a readme that describes what was accomplished and how, any API's that were created, and how to run the unit tests. You can tell it to detail the interactions and dependencies between the various components. You can even tell it to record the significant prompts that were used to drive the development. The documents will reside in a project subdirectory and can easily be updated by the AI on an ongoing basis.

The next person who comes along and wants to make a change or a fix will point an LLM to the documents directory. It will ingest the entire project context and organization in a few seconds, be ready to offer well-informed suggestions, and update the docs when it's finished with the modifications.

Slashdot Top Deals

You are in a maze of little twisting passages, all alike.

Working...