There are multiple good reasons to run local AI. Guard rails may be the most obvious, but perhaps not the most important. Other reasons include not putting your prompts in the cloud for glowies to collect, not having to deal with a model rugpull, never hitting usage limits (although they're replaced by the speed limit of your hardware), and being able to fine tune a model to not even have the guard rails in the first place. If you find something you love as much as some people loved ChatGPT 4o, you can run it forever and nobody can take away that access. You can talk about unethical schemes (as I frequently do, when writing fiction or worldbuilding an RPG campaign) and not worry about a knock on the door from a government official that thinks you *actually* want to dump ricin in the municipal water supply, or microwave your neighbor from the other side of the wall. Refusals are easier to work around in most cases than trying to anticipate cases where your efforts might be interpreted as something much more serious than they actually are.

And then there are always good-looking hosts in their early 20s reading scripts written by people in their 40s and 50s and filmed by people whose age just doesn't matter. Sometimes the 40s and 50s continue to host as well, if they're still good-looking enough (like SciShow).

I read almost all of this in Boris Tizengauzen's Ukrainian accent. I was even looking for the tell-tale "botox jellyfish" nickname for Putler.

There seem to be some critical complexity thresholds that must be met to achieve various types of emergent complexity. Just looking at Gemma 4 alone, going from E2B to E4B to 12B just seems to get confused a bit less and know a bit more. Then somehow, going to 26B-A4B gains a very clear ability to hold more complex associations and concepts in mind. 31B is better still, and enough so that I almost want to classify 26B-A4B with the toy models (E2B, E4B, 12B), but letting 26B-A4B do a lot of reasoning seems to help considerably. 26BB-A4B with Medium reasoning effort competes with 31B using no reasoning at all. However, 31B with reasoning enabled just seems qualitatively different again. It's no longer just a stochastic parrot and a knowledge base, but it actually seems to be able to work through many problems with multiple steps, where the best 26B-A4B can do is recite what the steps of the solution are. Just like an engine, there is no substitute for displacement (or parameters in this case) but you can bolt on turbos and the like to get around *some* of the shortcomings of a smaller cylinder such that it becomes less obvious.

True, if Fable is that size. I had been hearing Fable was between 3 and 5 trillion, in which case Kimi K3 at 2.8T is similarly classed rather than "considerably smaller". It's hard to say, if we never get to play with these proprietary models.

Kimi K3 is 2.8 trillion parameters. "Considerably smaller" would mean something like GLM 5.2, which is 753 billion.

Shrimp is bugs. You can ease into the land shrimp from there.

if you can "see" walls and gaps by clicking then you can presumably navigate a cave even if your light source goes out

It still won't stop you from being eaten by a grue.

They're like "How dare you object to our literal enshittification of the product. We bought those laws fair and square!"

When are users going to learn that any data they only have one copy of, they're just one error (human or electronic) from having zero copies of?

Would you turn your laptop over to an intern and expect they'll never fuck it up? Because we're basically doing the same thing when we hand the keys to an Agent. The only reasonable move is to never operate on the only copy of anything, if you can possibly avoid it.

The "unaware" problem is largely because the target is actively trying to anticipate and dodge, and it takes a fraction of a second to track and recalculate... by which time the target has already moved again. First they had to figure out how to target and attack. They did this. Then they had to figure out how to dodge and anticipate incoming attacks. They did this. Human combatants would (with practice) learn to compensate for this as well, which is when it really helps to anticipate what evasive maneuvers the opponent has at their disposal. The robots are still figuring this part out. But humans have the same problem, both physics and the speed of information processing mean our movements will always be "behind the curve". We deal with it by guessing where the curve is pointing, and sometimes being wrong, because going for the apparent target will *always* be wrong unless you're fighting a fence post.

A 16 GB 6800XT is going for under $500 these days, if you're willing to cross the CUDA/ROCm divide.

I think you are giving him too much credit. He has someone read it to him.

It could matter, if your planning requires operating at an international level, and the AI conveniently neglects to mention that outsiders practically never win court cases in China. Or even if it just has an allergy to talking about June 4, 1989 (which has showed up in the real world). The choice of using another culture's AI to avoid the biases of your own culture has merit, but it also means you won't have a lifetime of experience with the biases it *does* have.

Any quantization beyond Q4_0 goes mad (losing context, making category errors, malforming markup tags, mangling and inventing words) long before the context reaches a size I would consider "useful". I've tried running 31B at IQ3_XS and it looks alright from a distance, for a few prompts at a time. Unfortunately the errors compound and semantic drift goes into runaway. I have no interest in playing with quantizations any smaller than that. The only thing "massively wrong" is the suitability of a 2-bit quantization for my actual use. I've investigated it, and it can be an amusing toy, but it is far from usable for any meaningful work.

Even Q4_K_M was too error-prone for meaningful use, and I was using Q5_K_M as my baseline of "usable for a while". Fortunately, most of the heavily quantized models are based on QAT releases now, and a 4-bit QAT model is pretty much on par with a 5-bit non-QAT model.