"How did you get hired?"
"I was top at Fortnite. You?"
"Combat flight sims."

"Damn, two Boeings just crashed over an oil refinery on city limits and took out half the city."
"Did you remember to save your position beforehand?"
"Yeah."
"The reload and continue from there. No-one will notice."

I love computer games. I have XPlane 12 and many scenery packs. I rank well on Elite:Dangerous. From the sounds of it, the FAA would see me as over-qualified. In reality? There's no way in hell I'd be taking those kinds of risks with real lives. There's a huge difference between having good reflexes and a good eye, versus having the complex 4D spacetime relationship mental models needed for robust air traffic control.

1) 30B, dense. Optimized to fit Q4 quantized in a 24GB card with a speculative decoding model as well.

2) Because reporters don't know what weights are and assume you don't know either.

3) Way better than Gemma 4 on text tasks, slightly better on multimodal. Numbers below are all: Benchmark: Muse Glimmer score Gemma 4 31B score difference

Artificial Analysis Intelligence Index: Muse Glimmer: 35 30 +5
MCP Atlas (Public): 75.5 54.2 +21.3
DeepSearch QA: 74.6 61.7 +12.9
SWE-Bench Pro: 51.2 36.9 +14.3
SWE-Bench Verified: 76.0 66.6 +9.4
OSWorld-Verified: 65.9 58.5 +7.4
GAIA2: 43.3 36.4 +6.9
WildClawBench: 47.6 37.6 +10.0
TerminalBench 2.1: 51.7 43.4 +8.3
Tau3-Banking: 23.5 15.1 +8.4
MMMU Pro: 74.0 73.0 +1.0
Charxiv Reasoning: 78.8 77.7 +1.1
OmniDocBench v1.5: 75.8 72.5 +3.3
ScreenSpot Pro: 75.4 75.9 -0.4

4) 128k tokens

5) No, sadly.

What you want is a DGX Spark.

It costs $4000.

And yes, there are fundamental advantages to cloud services, such as large-scale batching, little idle downtime, high speed, and hardware optimized to the specific models / serving needs. That said, one can weigh that off against sovereign control over your server...

Define "tolerable".

The model in question - Muse Glimmer - with DFlash/speculative decoding - will probably get you ~60 to 124 tok/s on a 3090 (a quite dated GPU). For a 5090, it's said to clock in at 233,4 tok/s. On CPU you're looking at maybe 3-5 tok/s.

If you call that "tolerable", I guess you're more patient than me? And as mentioned, you're not just wasting time, but also wasting a lot of power too - CPU is a very power-inefficient way to run ML models.

If you insist on CPU, this isn't the right kind of model anyway. You want to take advantage of the fact that you probably have lots of (comparably cheap) RAM, and compensate for the fact that you have (comparably) terrible memory bandwidth, and for that, you want a MoE with a high total parameters but a low active parameters. Not a dense model like this.

Then I was basically aiming at Mac Minis

That's very much a special case which you didn't mention in your post that I responded to, but still the answer is "meh". You couldn't run it at all on a 16GB Mac Mini, and I think you'd struggle to run it at all on a 24GB (because you have to share the ram with the OS, the inference server, etc). For the base Mini you might get 10-12 tok/s, and for the M2 Pro / M4 Pro, maybe 25-35 tok/s. Still pretty far from a GPU, though.

This model is designed for >= 24GB GPUs.

MoEs don't save RAM (for a given quality), they increase it. You have to store all of the parameters in memory, not just the active params. But inference only uses a subset of the total params for each token, so it reduces the memory bandwidth requirements and improves token generation rate. But this comes at the cost of a higher total param count for a given quality.

Meta has always been releasing open models (Llama was famously the first powerful open model). The change has actually been in the opposite direction, with Muse Spark *not* being open.

Zuck descrbed his motivation way back when, about how they got burned with Facebook on app stores, in that Apple and Google could basically bully them however they wanted, on whatever extractive terms they wanted, and there was nothing Meta could do about it. He's now paranoid about "others controlling the platform", and wanted to make sure that doesn't happen with AI, that they have their own AI base to work with.

I mean, if you want your model to be terribly slow and terribly power inefficient, sure, go ahead and run it on CPU.

This really isn't a good model for CPU. For CPU, you want a MoE with a large number of total params but a tiny number of active params. Something like DeepSeek V4 Flash 0731 if you have at least 128GB of RAM - you might get 2-3 tok/s or so on that. The goal is to minimize the memory bandwidth requirements per token, at the cost of a greater total RAM footprint.

For GPU, you're highly VRAM limited but not bandwidth limited, so your best option is generally a dense model (non-MoE) with speculative decoding to make up for the performance limitations.

You can use it fine on 24GB with a good size context with KV cache compression. They've also designed it to use speculative decoding to boost performance instead of a MoE, so that they can pack as much capability as possible into a dense model. Seems like a winner.

You see, Linus has embraced transhumanism and is testing out the new kernel as a supplementary brain function, using MOSIX to offload all of the irritable comment generation at yet more nonsense on the mailing list to an Elizabot that he has written specifically to do this. This saves his actual brain for real work.

AI is not currently capable of performing any meaningful conceptual abstraction, it is only capable of very basic mechanical abstraction. Abstraction is itself multi-dimensional. Nor is there any indication that AI could ever perform multi-dimensional abstraction or multi-dimensional decomposition. Precious few humans are capable of it either, but some can.

No, AI would need humans because the best system is not a pure system but a hybrid system, and that means transhumanism with AI operating as an additional brain function rather than as a replacement for a brain.

Tool calling is, yes, but the current approach to it is (a) overt not transparent to the user, (b) not designed for the purpose I've outlined, and (c) not remotely good enough for the AI to be able to manage data through such tools.

Yes, you can connect an AI to a PostgreSQL database. Whoopee. Not even close to an AI detecting via classifiers that some of the data is relational in nature, transparently setting up its own PostgreSQL database in response, and using that proactively as an additional way to examine the data. You would need to explicitly set up the constructs, explicitly set up the databases, explicitly tell it when, where, and how to use the database, and if that particular usage conflicted with a way the AI actually worked, the AI would not be able to use it effectively. That's not remotely close to what I'm talking about.

What I'm talking about is much, much deeper than that. AIs lose focus when there is too much information currently in the system. Absolutely nothing stops an AI from using a document database as a virtual memory in which it can page in and out conceptual spaces so that the focus on any given step of a problem concerns just the problem. Other than such a concept not existing yet, which is kind of a limiting factor. But you need to know what connects to what. The AI could use an ontology reasoner for that, or a relational database, or an external graph. But you cannot provide those tools and you cannot configure them, for the simple reason that the AI is the only entity with any detailed map of how the underlying neural net is connecting those ideas up.

The AI itself has to select the tools, has to configure the tools, has to do all the work. Which means YOU cannot use any sort of API. YOU should have no involvement in the underlying mechanisms needed for the internal NN housekeeping.

And that's your other error. You're assuming this is about user data. No. It's about housekeeping operations within the NN itself to maximise functionality, it has nothing to do with the user side of things at all. If the user was capable of setting up dynamically structured databases that mutated with each step of a decomposed analysis, you'd have done everything the AI would be doing and you wouldn't need the AI to begin with.

You cannot do this work, you cannot even assist in this work, it has to be dynamic and it has to be utterly invisible to the operator.

We don't need bigger models, at this point. What we need is multi-dimensional decomposition, problem space transforms, and the ability for AI to use external tools (such as SQLite, memcached, etc) so that it can externalise static data that it needs to not corrupt accidentally but still keep in easy access.

If we had that, most of the things "bigger models" will do will actually end up being done better, quicker, with fewer compute resources.

To be useful in deep space, you're going to have to deal with very harsh radiation, far harder than the Vikings dealt with. You really want to have some large number of computers, where the number has to be odd and exceed 5. The reason it has to exceed 5 is that you need 5 in order to be able to use the Byzantine General's Problem to discern which computers are working correctly and which are radiation damaged. Since some will be damaged over time and you want 5 computers still operating around the time the power runs out, you have to start with more than that.

Massively radiation-hardened highly-robust low-power chips could be done. If you were clever enough, you could do this as a wafer-scale system where you marked parts as bad and networked around them. Free space optical communication across the wafer might be doable.

They might well still end up being low-density CMOS discrete logic.

Lead-lining the electronics isn't as much of a problem if you've launchers capable of handling heavy objects and don't mind burning through a lot of ion drive propellant for course corrections. This would improve the hardening beyond what can be done through the usual operations.

But, yeah, you'd need defect-free microelectronics. You really can't handle F00F bugs or defective instructions once you pass Earth orbit. And I can't think of anyone who knows how to do that.

Well, not just the solar system but the heliopause as well, and the near-Sol galactic winds. We can estimate the effects of space from just about any ancient near-Earth space junk, but I'm not sure how we'd go about calculating the impact of the galactic winds on something. The Voyager sensors won't be sending back nearly enough high-quality data to establish that.