Forgot your password?
typodupeerror

Comment Re:To err is human... (Score 1, Funny) 106

Say it with me, now. As we all know, the infamous saying goes:

A COMPUTER CAN NEVER BE HELD ACCOUNTABLE

THEREFORE A COMPUTER MUST NEVER MAKE A MANAGEMENT DECISION

It's really incredible how marketing departments can radiate amnesia like this with such proficiency.

That's a misquote, the actual quote is:

UNLIKE EXECUTIVES A COMPUTER CAN NEVER BE HELD ACCOUNTABLE

THEREFORE A COMPUTER MUST MAKE EVERY MANAGEMENT DECISION

Abraham Lincoln, CEO
Union Carbide
July 12, 1856

Comment YIKES! API Price (Score 4, Interesting) 61

Just saw the reported API pricing for those who are allowed access: $25/$125 per 1M tokens. To put that into perspective Opus 4.6 is $5/$25 per 1M tokens. Even Opus 4 was "only" $15/$75 per 1M. No way this one is coming to any plans. It will be enterprise only when they do open it up more.

Still cheaper than GPT Pro though ($30/$180)

Comment Re:I use gemini (Score 1) 105

You can't code rules into models themselves. Best you can do is try to train the behavior you want but that's never going to be 100% reliable. You can do it by watching the logits from the inference engine an try to redirect the model back on track or force a hard stop. Some are doing this today. The problem is that next word low probabilities are not always the source of this problem. You also run into high probability wrong results, so it's a bit more complicated. The other issue is not all of the APIs expose logprobs, or don't by default (openAI lets you turn them on). So if you don't own the inference engine and your LLM provider doesn't support it, it's not even possible to do it yourself.

And it actually is very much in their best interest. Hallucinations are a huge issue and kill many enterprise projects in the planning or demo stage. Solving it, even if that means returning "I don't know" or a signal in the response would drive more business for them, not less.

Comment Re:Local LMs worth it? (Score 1) 46

That Mac Studio with a 2tb ssd is $7900 not $10K. The old 512GB was a little over $10K but they dropped that option. As for price as far as I know it hasn't gone up. The new M5 Max 128 didn't get a price increase over the M4 Max (with the same size SSD configured) so hopefully the next studio will follow the same pattern.

But yea if you want to run large models for a reasonable price it's the only game in town right now.

Comment Re: I already cancelled my subscription (Score 1) 46

If you want to use it async that's fine as long as async means 10's of minutes or more between turns for development work. And it's not just tokens per second. Prefill is compute bound and is going to be very slow even compared to a low end GPU. Larger contexts are going to pressure the KV cache reads which will also impact tps, and coding generally uses lots of context each turn. It all adds up.

Slashdot Top Deals

"Now here's something you're really going to like!" -- Rocket J. Squirrel

Working...