Forgot your password?
typodupeerror

Comment He is allowed to slow down (Score 1) 118

No one is forcing him to spend money he doesn't have to produce a product no one will pay for.

He has irresponsibly played a game and even helped make the rules. Now he realizes other people are better at it than he is, but he keeps spending borrowed money for 'pay to play' and all the other players except openai (his twin) are figuring out how to play for much less.

His biggest competition are the bastards at companies like Alibaba, Microsoft, and Google who actually have products, services, and paying customers who will stick with them even if their AI is a generation or three behind.

See, it's perfectly ok to slow down. The others have done it. But they cheated... They actually had business plans.

Comment Re:Yes but is it genocide? (Score 3, Informative) 61

People who have a much greater standing than me have said that Israel is carrying out genocide, eg: The International Association of Genocide Scholars, Holocaust scholar Amos Goldberg, Médecins Sans Frontières, Raz Segal "A Textbook Case of Genocide" to list but a few. The evidence is there for all to see but our political leaders in the West ignore it.

Then there is Jerusalem, the West Bank, South Lebanon - all with the aim of Greater Israel.

Comment Re: M5 Max MacBook Pro with 128GB (Score 2) 17

Qwen 3.8 27b runs with mtp and a 72k context window on two RTX 5060 Ti 16GB at about 35tps (in MTP, that compares to 75tps non mtp). And, yesterday, it nearly matched GLM 5.2 running on 8xH100 in every task I tossed it. (My glm rig is rate limited to 30rps). My comparison is "ability to handle long complex tasks".

So, the trick is to use memory, search, fetch, vector database.

The purpose of a model is to reason. It needs enough training to perform further research. Context is very-short term memory, vector databases are their long term memory, rag is the books they read, and search and fetch are their libraries.

I would LOVE to switch back to two 3090 cards for 48GB (they do image and video now), but when I run qwen 3.8 27b fp8 with 256k context, I find the quality is higher in some cases, but drops because the model then favors short term memory iver research.

Memory bandwidth is much more exciting. 500tps when running on an HPC is very very nice.

So, if it were me measuring, a single 64GB HBM3e GPU is really where we should aim as this should be consumer cost friendly by 2030. (now I have to try on an A100 later today)

Comment Just become legal in England (Score 4, Informative) 246

In the last few days plug-in solar became legal for Britons. How expensive are they ? One company will "sell 460W and 890W kits, starting at £599 and £999 respectively, and it plans to launch another four models in late September, with prices starting from £499". Expect them to repay their cost in 4 to 10 years. I expect the purchase price to come down as volume sold goes up.

Comment Re:Nobody is multitasking (Score 1) 185

Correct. When I am driving my attention should be looking at the car in front, not peering at a display trying to find the menu item to change air-con fan speed. These screens are dangerous, you think that you will find what you want with a quick glance and suddenly realise that you have had your eyes off the road for 10 seconds - which at 70mph is over 200 yards.

I guess that BMW is now struck off the list when I next look for a car to buy; along with Tesla and others which are just computer screens with a vehicle attached.

Comment Re:Doesn't matter, obsolete designs (Score 1) 160

Inference is neural network limited. On systems with entirely separated data and compute, this is a memory bus constraint. In quantized systems, ALUs and FPUs are illogical. A simple 16 cell LUT is suitable for noise free multiplication. A neural network circuit with a dedicated LUT and addressable buffers more or less eliminates bandwidth constraints. But this requires neural networks more similar to FPGA cells rather than classic compute.

That said, separating inference from data drops network depth considerably. And in transformers depth increases memory access exponentially which is why nearly identical MoE and dense models perform so differently. We of course need data centers for models with massive numbers of active parameters. But massive numbers of active parameters weaken models as that means depending on training rather than looking up facts. Huge active parameter sets will never be a good design. A real world analogy is that huge models are like Jeopardy champions who have crap loads of partial trivia facts in their heads. Smaller models are like librarians who don't have a trillions useless facts memorized but knows how to find the data for the researchers who will use the librarian repetitively to follow through to the answer. The model only needs enough training to use the data.

What really matters is, how fast it can find the data.

Inference suffers greatly when you treat weights as data. And bigger models have more trivia like facts memorized from their training. This means more active parameters and greater network depths with higher memory bandwidth needs. Even now, you see the chatbots are improving drastically because they rely substantially more on data and way less on inference.

I am not sure which part you consider gibberish, but it makes perfect sense to me. But it might sound better in my head.

P.S. I actually have considerable data to backup many of my claims. At work I'm sitting on about 50MW of compute. We build in shipping crates in a repurposed mine. There is a group of us where our goal is to avoid spinning up more compute. We need to deliver inference to 200,000 users eventually. (not customers, employees) and we could throw millions at NVidia and end up hosting some crap model, but we focus instead on cutting that memory bandwidth need. And yes, I'm far from being the smart guy on the team.

Slashdot Top Deals

You can fool all the people all of the time if the advertising is right and the budget is big enough. -- Joseph E. Levine

Working...