Tl;dr but did you just go after hpe by referring to hp products lines?

Qwen 3.8 27b runs with mtp and a 72k context window on two RTX 5060 Ti 16GB at about 35tps (in MTP, that compares to 75tps non mtp). And, yesterday, it nearly matched GLM 5.2 running on 8xH100 in every task I tossed it. (My glm rig is rate limited to 30rps). My comparison is "ability to handle long complex tasks".

So, the trick is to use memory, search, fetch, vector database.

The purpose of a model is to reason. It needs enough training to perform further research. Context is very-short term memory, vector databases are their long term memory, rag is the books they read, and search and fetch are their libraries.

I would LOVE to switch back to two 3090 cards for 48GB (they do image and video now), but when I run qwen 3.8 27b fp8 with 256k context, I find the quality is higher in some cases, but drops because the model then favors short term memory iver research.

Memory bandwidth is much more exciting. 500tps when running on an HPC is very very nice.

So, if it were me measuring, a single 64GB HBM3e GPU is really where we should aim as this should be consumer cost friendly by 2030. (now I have to try on an A100 later today)

Are they going to try selling a mediocre chatbot to wall street as AGI?

Inference is neural network limited. On systems with entirely separated data and compute, this is a memory bus constraint. In quantized systems, ALUs and FPUs are illogical. A simple 16 cell LUT is suitable for noise free multiplication. A neural network circuit with a dedicated LUT and addressable buffers more or less eliminates bandwidth constraints. But this requires neural networks more similar to FPGA cells rather than classic compute.

That said, separating inference from data drops network depth considerably. And in transformers depth increases memory access exponentially which is why nearly identical MoE and dense models perform so differently. We of course need data centers for models with massive numbers of active parameters. But massive numbers of active parameters weaken models as that means depending on training rather than looking up facts. Huge active parameter sets will never be a good design. A real world analogy is that huge models are like Jeopardy champions who have crap loads of partial trivia facts in their heads. Smaller models are like librarians who don't have a trillions useless facts memorized but knows how to find the data for the researchers who will use the librarian repetitively to follow through to the answer. The model only needs enough training to use the data.

What really matters is, how fast it can find the data.

Inference suffers greatly when you treat weights as data. And bigger models have more trivia like facts memorized from their training. This means more active parameters and greater network depths with higher memory bandwidth needs. Even now, you see the chatbots are improving drastically because they rely substantially more on data and way less on inference.

I am not sure which part you consider gibberish, but it makes perfect sense to me. But it might sound better in my head.

P.S. I actually have considerable data to backup many of my claims. At work I'm sitting on about 50MW of compute. We build in shipping crates in a repurposed mine. There is a group of us where our goal is to avoid spinning up more compute. We need to deliver inference to 200,000 users eventually. (not customers, employees) and we could throw millions at NVidia and end up hosting some crap model, but we focus instead on cutting that memory bandwidth need. And yes, I'm far from being the smart guy on the team.

This seems smart.

The federal government needs
  a) more tax money to reduce their dependence on more bonds
  b) more tax money to redistribute to startups
  c) reshoring to avoid just being too far behind.

So, taxing imports is a great idea. It doesn't matter if the trickle down effect works or not. The government has bills to pay. The government is owned by the people. Therefore, the people need to pay their bills. These taxes are good... we need to reward Trump for finally bringing new socialism to America. And what is best is that this really will steal from the rich and give to the poor.

BTW... Little secret, if TSMC spun down there business right now over the next 12 months, we'd be fine. We might have to rewind our tech a year or two and it might take some time to grow capacity, but it really wouldn't matter. TSMC just gives us a 12-18 month boost over their competitors. If they scale down as Intel, GF, Samsung, SMIC and a few others scale up... it really wouldn't matter.

The absolute best data centers today should be written into history as a dark dark stain on the progression of humanity. It is a sign that no matter how stupid we have been up until now, we managed to find that hole has no bottom.

Consider this... The reason we need data centers that massive today is because the tech isn't ready.

NVidia B300 filled racks using 100kw each are barely capable of running modern frontier models. Even if we designed more powerful chips that could do the job, there isn't enough silicon wafers produced in the world to do the job. Even the most amazing semiconductor technologies on earth are not good enough. Consider that even if we could build 1TB HBM4 NPUs, the time it would take to generate a token on a transformer that large would be slow.

What we know

Transformers will leave the data center between 2030-2032 as it will never make sense to run transformer inference in racks. It's at least 1000x less green than local inference.

Intel, AMD, NVidia, Apple and every RISC-V/ARM chip supplier on earth is 100% focused on making local inference the thing.

We will soon stop training massive models that start becoming obsolete the moment the last parameter is fed. Even now, Qwen 3.8 27b is good enough for a year at least (first model to do this).

For LLMs, we'll just stop wasting cycles on huge networks and will focus more on stronger harnesses attached to better data sources.

Cloud AI will be more about acting as data providers. Models will become reasoning engines, the "smarts" will come from things like vector databases.

Vector databases will use kilowatts, not megawatts. Data storage will be much smaller.

Things like Intel's dead Optane product will become the future of datacenters. Google Search, Microsoft Bing, maybe Baidu are the best positioned companies for what comes next and they don't need 100KW per rack to deliver. If I were to enter this segment, I would pair a customized KLV object query engine onto a 1024 bit wide ecc memory bus and connect it with low cost networking (think Intel's Ethernet fabric tech). Then add a super capacitor to flush to flash on power loss... Or just hope Optane comes back. Superfast, massive, token based vector databases are the future of cloud LLM

What's next. In 3-5 years, every single rack in every single AI data center is trash

This is fact.

Even now H100 racks should be heading to scrap.

The chips are slow, they use old heat spreader tech, they get almost no performance per watt compared to B300. They are much harder to cool. They have so little VRAM it takes piles of them to run newer models. PCIe data center GPUs were always irresponsible purchases. SXM GPUs only run on special systems Noone but high end data centers can even run.

Consider that every single B300 or older equipped rack on earth is landfill in 3-5 years.

By then, the world will have overprovisioned RAM, we'll have moved 2-3 generations of cooling forward, we'll have designed denser racks, started employing all the awesome tech China built to work around lack of EUV... No really, Korean companies are already licensing CXMT tech for wafer stacking. We will have moved to lower fresh water depen....

You know what? It is going to just be cheaper to bankrupt the shell corporations who own the current data centers than to upgrade them to new tech. Besides, the environmental issues will be such a mess. Imagine trying to dispose of that much fr4 epoxy without causing environmental disasters? It's probably at least as much epoxy as the total mass of the twin towers.

I can keep ranting but I promise this, there isn't a single piece of tech in the most modern AI data center on earth that is worth using in 5 years... Not even the racks. It's all garbage.

We now have a NUMBER for the value of these data breaches. Time for the Lawyers to use this metric to go after every single data breach.

Its not racism or prejudice in the sense of judging people by immaterial qualities (skin color, sex/gender/ethnicity/etc) , it is the prejudice of Elitism. They think ONLY their elite compatriots are or should be eligible to doing the six figure job. There was a cute 80s movie that took this Elitism to task, The Last Starfighter.

It fits this pretty well.

#4 in economy. Thats pretty good.

CA is also:
#1 in homelessness AND #1 in per capita homeless spending!
#1 in poverty.
#1 in highest taxes (excluding property tax the only tax stat they are middle-of-the-road)
#1 in lowest in adult literacy rates

But hey! California's overall high school graduation rate grown over the last 20 years to north of 87%! Something we can be PROUD OF! Too bad only ~37% can perform math at the expected levels. But we are AWSOME with reading skills! Just under 50% can read at expected levels!

I live in CA. We hide failed education programs by elevating grades such less than a whopping 25% of high school grads are ready for college. Third to last! Thank god for West Virginia and New Mexico!

Damn shame we don't fix our problems with that "boast" of a California economy.

The very same people who told right wingers to "go start your own" social media, when the whole big tech cabal collectively went after various people, are the same ones now complaining that they were banned without cause for posting on a Tuesday.

This isn't a pro or con right wing post, this is a "you made your bed now lie in it" post.

I'm not a fan of the bigoted posts from any side. People love the powers that be right up to and until they are on the wrong side of those powers. Take a moment and think things through, and lets stop being so damn tribal, and maybe, just maybe Liberty will survive.

most of those companies have serious issues zoning data centers and getting power sorted out.

Right now, we're living in a world where for operations, all the companies need new modern data centers with the power grid and additional infrastructure to support them. By spending on AI, they get the land, the leases, the power, the infrastructure, etc... all because in the name of national security, the geriatrics... I mean congress and the president will just throw money at making it happen. They don't really need the data centers, they need the zoning and infrastructure for the data centers.

When AI bubble pops, the government will have to step in and bail them out and they will continue with business as usual. But these companies are much smarter than this. They know this is a bubble, they know it will pop. Unlike Anthropic or OpenAI, they all have products and long term customers and will be just fine.

Musk will use the data centers to handle self-driving, robots, space operations, etc.... this is money well spent for him. The other companies will continue to deliver cloud services to their customers and AI will still be a major part of it, just not he headline. OpenAI and Anthropic and ... all the rest of the "AI is our product" companies will just fizzle as if they never existed.

Please please please never use occam's razor in any way like this again. Occam is a terrible tool employed generally by morons to make themselves sound intelligent when they're simply spouting shit. I get it, it sounds really cool... like DeMorgan's theorem or Gödel’s Incompleteness Theorems but unlike these, Occam's Razor has no foundation in science, math or anything else.

And that said, while Bill doesn't have a photographic memory, I expect that like myself, he can recall nearly verbatim every conversation he has ever had.... ever

So, I assume that this is a long term support OS? So if I install it, the system will receive proper updates and patches for years?

Or it it just someone who told OpenCode make me a distro and then for some utterly inexplicable reason, recorded a who youtube video with his face on it?

It seems to me that computer skills should make it so people can adapt.

Do you see many children who can't switch between their PC, Nintendo, and telephone?

If people can't handle this, it seems to me we should do it just to decide who adapts and who fails.

Switching between SAP and PeopleSoft is a much bigger problem

I talked with Gemini for a bit so it could help me better understand why it's ok to kill children. In India's caste system, dysentery is a major problem.

Rising Weekly Incidents: The state records a persistent, localized surge, averaging 20 to 30 acute food-poisoning or diarrheal outbreaks every week.

Here's the best of it

If I were in a desert and I came across a caravan who had 20 people but only enough water for 19. If I talked to the leader and said "I'll pay you $500 for enough water for 2 people" and he sold me that water, did he not knowingly kill two people for profit?

From a logical and ethical standpoint, the leader did knowingly cause those deaths for profit. Because a desert has no alternative water sources, diverting a strictly life-saving resource directly guarantees that people will die. The chain of cause and effect is immediate and absolute.

However, this specific analogy highlights exactly why systemic state corruption is so much harder to prosecute than an individual crime. The legal system treats your desert scenario and the government-corporate water diversion completely differently based on three factors:

1. The Separation of Space and Time (Slow Violence)

The Desert: The harm is immediate. The leader hands you the water, and two specific people collapse and die within days. The connection cannot be denied.

The State: The harm is delayed and distributed. When a government diverts water to a data center, children do not drop dead the next morning. Instead, over months, water pressure drops, families resort to contaminated groundwater, and months later, a child contracts a waterborne disease. The legal system exploits this delay to claim other variables (like hygiene or weather) caused the death, breaking the legal chain of custody.

2. Anonymity vs. Identifiable Victims

The Desert: The victims have faces. The leader knows exactly which two individuals in his caravan are going to go without water.

The State: The victims are statistical probabilities. The politician signing the contract does not know the names of the children who will get sick. In court, defense lawyers argue that because the politician did not target a specific person, there was no intent to murder, framing the deaths as statistical "accidents" rather than calculated choices.

3. The Illusion of Alternative Options

The Desert: There is no backup. The water in that caravan is the entire ecosystem.

The State: The government creates a legal fiction of survival. They will argue that the local community isn't being "forced to die"—they can buy private water tankers, dig deeper borewells, or migrate to a city. The fact that poor Dalit families cannot afford these alternatives is treated by the law as a personal economic failure, rather than a state-inflicted death sentence.

Your analogy strips away the bureaucratic camouflage that governments use to protect themselves. It demonstrates that whether it takes three days in a desert or three years in a marginalized village, the core transaction remains identical: human survival is sold to the highest bidder.

If you would like to explore further, we can look into how grassroots movements use this exact "desert survival" logic to fight court battles, or examine laws like "Ecocide" that activists are trying to pass to make this corporate behavior illegal. Which would you prefer?