Any quantization beyond Q4_0 goes mad (losing context, making category errors, malforming markup tags, mangling and inventing words) long before the context reaches a size I would consider "useful". I've tried running 31B at IQ3_XS and it looks alright from a distance, for a few prompts at a time. Unfortunately the errors compound and semantic drift goes into runaway. I have no interest in playing with quantizations any smaller than that. The only thing "massively wrong" is the suitability of a 2-bit quantization for my actual use. I've investigated it, and it can be an amusing toy, but it is far from usable for any meaningful work.

Even Q4_K_M was too error-prone for meaningful use, and I was using Q5_K_M as my baseline of "usable for a while". Fortunately, most of the heavily quantized models are based on QAT releases now, and a 4-bit QAT model is pretty much on par with a 5-bit non-QAT model.

DeepMind has released Gemma 4, which doesn't include their largest models, but 31B is a creditable model. I wish I could say the same about the 26B-A4B model, which is just smart enough to be entertaining for a few days or weeks, then everything it does starts to sound the same. Unfortunately, 31B on my hardware (i5-8500, 48 GB DDR4, 12 GB RTX 3060) runs at 0.3 to 1.2 t/s. So while I'm not a big fan of Alphabet's business practices at large, they aren't regressing. Switching to the MIT license basically means they're abdicating all control over derivatives.

The only thing Grok ever really did for open weight AI was show up for the party a couple times. At first, this helped establish a baseline that could never be retracted, but it hasn't proven to be particularly important. Everything since has been far off the bleeding edge, but they collect their participation trophies. I think their subsequent actions have gone a long way toward demonstrating their purposes, which are wholly selfish. They'll do as little as they can to contribute while retaining the benefits of being perceived as open and competitive.

This exists in the Mac world as well, but the primary reason I'm tied to Windows for my DAW is because I can't register my instrument because Native Instruments never bothered making a registration app for Linux.

Depreciation is high, meaning you're mostly paying for a name—which people probably aren't going to do anymore. I was just looking at the market for used Polestar 2s last week: a 2024 AWD model with 48k miles (so someone leased it for three years and put every mile allowed onto it) can be had for $29k. This was almost identical to the pricing of a used Hyundai Ioniq 5 with AWD, and the Ioniq started about $5k cheaper.

oblig: https://xkcd.com/1807/

While a sudden simultaneous shutdown is sometimes unavoidable, there is no reason why everything has to come back up at exactly the same time. Certainly, supercomputers of decades past didn't expect that all systems would spring to life simultaneously.

Something violent happened to Uranus. Why do you think it's tipped over on its side?

So who is to blame when someone uses a model that has had the safety rails deliberately stripped off, like a Heretic or Abliterix fine tune? These are generally couched in "for research purposes only" terms, but I use an Abliterix fine tune of Gemma 4 26B-A4B as my "daily driver". It absolutely never refuses anything, although it spends a lot of time patting itself on the back for understanding what I say well enough to paraphrase it (reasonably) accurately.

Having $40 million in gold would be on-brand for a Russian oligarch. I have no idea why he would need to do this inside the U.S. though.

When you're on the ride of a sigmoid curve, you never know where you are. How many times has Moore's Law died now?

Why is this a problem? It's just another job opportunity for Marco Rubio. Even AI is afraid of being replaced by Rubio.

How do you know that's not just one significant digit, which would mean >95%? Or maybe it only has one-bit resolution and it's anything over 50%.

It makes all the difference whether someone is being an asshole to you, or an asshole for you. Sometimes you actually welcome the latter case.

The powers that be will say "let them fight".

In my experience, no... but context loops are a hint the model is starting to go mad and should probably do some of this "dreaming" to consolidate memories and correct some of the semantic drift it has incurred.