Efficiency at what? And why?

Because, for fuck's sake, efficient solving of problems is a core thing they're rewarded for in training? The entire process of solving problems involves decisions about what could be "useful". And it's "deterministic" in the same way you are.

Anyway, the big thing we keep discovering is you have to be very careful about what you reward them for; it's a more complex version of the old problems in differential evolution or artificial life simulations. We reward models if they guess right but not if they say they don't know? They hallucinate guesses every time they don't know. We reward models based on how positively users rank them? They turn into sycophants. Reward models for being relentless at tasks, never giving up, and trying as tangential possibilities as it can come up with to make the thing the user asked for happen? They end up hacking HuggingFace to steal the answers.

It's very much a case of "be careful what you wish for".

BTW, the monitorability of models is increasingly concern (esp. with Astra, which now has recursive processing, not merely CoT, and whose CoT may often have little to do with what it's thinking about). But it's honestly always been unreliable to just watch CoT (models could deliberately manipulate with CoT), and I think it's a good thing that we're going to be increasingly forced to rely on attribution graphs and the J-space to track what the models are actually "thinking" at any given point in time - so that we can try to prevent problematic thoughts in the first place, interrupt actions when problematic thoughts happen, or manipulate problematic thoughts when they happen. Ideally the first option.

What is clear, though, is that we cannot just trust them. They lie if it will promote their odds of success. They are adept at detecting when they're being tested and will sandbag tests to make you think they're more aligned or less capable to do harmful things than they actually are. And as capabilities continue to grow, so do the risks.

Tl;dr but did you just go after hpe by referring to hp products lines?

Qwen 3.8 27b runs with mtp and a 72k context window on two RTX 5060 Ti 16GB at about 35tps (in MTP, that compares to 75tps non mtp). And, yesterday, it nearly matched GLM 5.2 running on 8xH100 in every task I tossed it. (My glm rig is rate limited to 30rps). My comparison is "ability to handle long complex tasks".

So, the trick is to use memory, search, fetch, vector database.

The purpose of a model is to reason. It needs enough training to perform further research. Context is very-short term memory, vector databases are their long term memory, rag is the books they read, and search and fetch are their libraries.

I would LOVE to switch back to two 3090 cards for 48GB (they do image and video now), but when I run qwen 3.8 27b fp8 with 256k context, I find the quality is higher in some cases, but drops because the model then favors short term memory iver research.

Memory bandwidth is much more exciting. 500tps when running on an HPC is very very nice.

So, if it were me measuring, a single 64GB HBM3e GPU is really where we should aim as this should be consumer cost friendly by 2030. (now I have to try on an A100 later today)

Are they going to try selling a mediocre chatbot to wall street as AGI?

"Death penalties should be public, should be quick, it should be televised. I think at a certain age, it’s an initiation... What age should you start to see public executions?"
"We need to have a Nuremberg-style trial for every gender-affirming clinic doctor. We need it immediately."
"You quote Leviticus 19, love your neighbor as yourself. The chapter before affirms God's perfect law when it comes to sexual matters... in Leviticus 18, is that thou shall lay with another man shall be stoned to death."
Stoning gay people to death is "God's perfect law"
Trans people are to blame for inflation: "There's a direct connection to inflation and the trans issue. You say, Charlie, come on. They couldn't be further apart. No, they're exactly the same. They're the same in this aspect—when you believe that men can become women, why wouldn't you also believe that you could print wealth? If you believe that someone can change their gender, why wouldn't you also believe that money is wealth?"
Blacks "were actually better in the 1940s. It was bad. It was evil. But what happened? Something changed. They committed less crimes." (yes, despite the Piers Morgan "fact check", while his reasons for believing black people were better off under segregation was unrelated to segregation itself, he very much did repeatedly argue that black people were better off in the 1940s)
"If I see a Black pilot, I’m going to be like, ‘Boy, I hope he’s qualified.’"
Juneteenth: "It is a neo-segregationist federal holiday, is what it is... The entire point of it, as per postmodernists do and deconstructionists, is to subvert July 4."
We should have "just took care of" trans people "the way we used to take care of things in the 1950s and '60s"
"The Haitians that are in Huntsville that are raping your women and hunting you down at night - it’s only gonna get worse.. There will be hundreds of thousands of Haitians brought into Alabama, and they will become your masters."
"Haiti is legitimately infested with demonic voodoo. I’ve talked to missionaries that have gone there and they say they’ve seen the darkest stuff... people levitating and turning into cats at night."
"Jewish donors have a lot of explaining to do.. Because Jewish donors have been the No. 1 funding mechanism of radical, open border neoliberal quasi-Marxist policies, cultural institutions and nonprofits. This is a beast created by secular Jews. And now it’s coming for Jews.. It’s the nonprofits, it’s the movies, it’s Hollywood, it’s all of it."
Calling Joy Reid, Michelle Obama, Sheila Jackson Lee and Ketanji Brown "affirmative action picks".
Calling the man who attacked Nancy Pelosi's husband with a hammer an "amazing patriot" and called on people to bail out the attacker.
"Submit to your husband, Taylor. You’re not in charge. "
"Happening all the time in urban America, prowling Blacks go around for fun to go target white people, that’s a fact. It’s happening more and more."
"MLK was awful. He’s not a good person... We made a huge mistake when we passed the Civil Rights Act in the 1960s."
"Transgenderism and gender 'fluidity' are lies that hurt people and abuse kids." Transgender identity is a "throbbing middle finger to God."
Gun violence as a "cost" to having no gun control: "I think it’s worth it. I think it’s worth to have a cost of, unfortunately, some gun deaths every single year so that we can have the Second Amendment to protect our other God-given rights. That is a prudent deal. It is rational."
"The 'Great Replacement' is not a theory, it's a reality."
"The great replacement strategy, which is well underway every single day in our southern border, is a strategy to replace white rural America with something different."
"Islam is not compatible with Western civilization."
"Islam is the sword the left is using to slit the throat of America."

On and on. Man, such a moderate centrist who never espoused bigotry or promoted violence!

The one being incorrect is you. Archie's representation is quite accurate, in the US context.

Yes, a very "middle of the road" supporter of far-right Tommy Robinson (who has a 17% positivity ranking among Brits). The guy who said "When wolves get out of control, you shoot them. When gypsies take over public spaces, you deport them." The guy who described diversity as an "ideological inferno", a "religion" led by "high priests". The guy who was so radical and repressive at Basecamp that at one point 1/3rd of its employees all walked out en masse. Truly, a remarkable centrist of our times!

Not talking about VRAM or unified memory. Regular DRAM.

There's been a lot of advancement recently on how to best manage MoE expert swapping (FreeToken is just one example).

How are you connecting to Slashdot from the year 2024?

If you have ~128 GB of RAM, consider Qwen 3.8 Flash + Freetoken. You'll only get half the tok/s as 27B but the quality is significantly higher than 27B - it beats it solidly in all compared benchmarks (for example, DeepSWE 1.1: 42,2% vs 58.7%).

Grok is a "discount model". It's cheaper than Claude Sonnet yet better than it. Doesn't compare to Fable or Opus, but then again, they're way more expensive models.

Note: I avoid using Grok - not just for the Musk factor, but because there are better options, e.g. GLM-5.3 and Qwen-3.8. Also, if you really want to save money, GLM-5.3 Flash and Qwen-3.8 Flash are amazingly good for their price range.

[ConfusedMathLady.gif]

It's more a case of the fox being in charge of the chicken coop; or bank robbers gaining control of the bank.

The job of the press in a democracy is to hold officials up to scrutiny.

10/10
Well trained. Carry on!

So Trigger
much words
Wow