Like this? https://www.pcmag.com/news/mic...

Personally I'd prefer that format but it didn't sell very well for MS.

This model won't ever be part of any of the subscriptions, it's going to be API only so no, probably not.

I wouldn't call 15-75W "A LOT of watts"

Sincerely,
Someone who actually read the summary.

Just saw the reported API pricing for those who are allowed access: $25/$125 per 1M tokens. To put that into perspective Opus 4.6 is $5/$25 per 1M tokens. Even Opus 4 was "only" $15/$75 per 1M. No way this one is coming to any plans. It will be enterprise only when they do open it up more.

Still cheaper than GPT Pro though ($30/$180)

You can't code rules into models themselves. Best you can do is try to train the behavior you want but that's never going to be 100% reliable. You can do it by watching the logits from the inference engine an try to redirect the model back on track or force a hard stop. Some are doing this today. The problem is that next word low probabilities are not always the source of this problem. You also run into high probability wrong results, so it's a bit more complicated. The other issue is not all of the APIs expose logprobs, or don't by default (openAI lets you turn them on). So if you don't own the inference engine and your LLM provider doesn't support it, it's not even possible to do it yourself.

And it actually is very much in their best interest. Hallucinations are a huge issue and kill many enterprise projects in the planning or demo stage. Solving it, even if that means returning "I don't know" or a signal in the response would drive more business for them, not less.

That Mac Studio with a 2tb ssd is $7900 not $10K. The old 512GB was a little over $10K but they dropped that option. As for price as far as I know it hasn't gone up. The new M5 Max 128 didn't get a price increase over the M4 Max (with the same size SSD configured) so hopefully the next studio will follow the same pattern.

But yea if you want to run large models for a reasonable price it's the only game in town right now.

If you want to use it async that's fine as long as async means 10's of minutes or more between turns for development work. And it's not just tokens per second. Prefill is compute bound and is going to be very slow even compared to a low end GPU. Larger contexts are going to pressure the KV cache reads which will also impact tps, and coding generally uses lots of context each turn. It all adds up.

AFAIK there are no unified RAM systems that are good to run AI on that don't use soldered RAM. AMD tried it but couldn't get it to work.

I hope you looked into benchmarks for running models in system RAM vs VRAM.

You joke but following up with "Are you sure?" will trigger a lot of models to go verify their output with external sources like web searches.

Funny enough, this is one place where the fix is easy but probably not cheap. They just need to build guardrails that automatically checks any case law references against something like LexisNexis and feeds back to the AI if it makes something up. Case law is extremely well documented and fairly structured in how it's indexed. You wouldn't even need to use AI for the lookup, a competent traditional search algo would work. Of course that's going to be expensive since it will require access to the case law data electronically and SOMEONE is going to make bank on that. It's not something you will get with a $20 ChatGPT subscription.

No. The entire model is loaded into memory but the feed-forward layers are split into subnetworks. A router picks a few of the best "experts" on a per-token basis to activate and those are the 3.8B that are activated. It's a way to increase inference performance so you get the speed of a dense 3.8B model with the output quality close to (but not equal to) a 26B dense model.

I'm not. Video generation is expensive, even by LLM standards. They needed a huge buy-in from industry to make it workable and all they got was the Disney deal. I suspect that the price they would need to charge to make it profitable is just too much to make it attractive to industry customers. Add in the issues with it in general and with getting it to work with long-form content and it's probably a non-starter right now.

Yes but do those keyboards send separate scan codes for the split spacebar, or allow remapping one of them?

Because they already have a product called EarPods: https://www.apple.com/shop/pro...