for once I am really rooting for the extinguish part of Microsoft's Embrace Extend Extinguish MO..

The language and the tech are fine but the proponents are mostly insufferable.

I'll bet; my first reaction to that would be "Today is not going to be a good day"

Here let me google for you

https://www.sciencedirect.com/...

I hope it is one of those fancy engineered septic systems, with pumps, timers, floats, dose controls etc. In which case yeah ... I often wonder if the engineers and the health departments understand them either. AI is probably great diagnostic tool there.

On the other hand the conventional ones really are just a hole in the ground for the most part; if you need AI for that we are in real trouble.

I think it is more simple then that. It is about protecting the status quo.

Nobody is really happy with US education outcomes. It is painfully obvious that places with the NEA/AFT enjoy the most influence, is highly aligned with the poorest performance.

You have similar orthodoxies in the higher-ed pedagogical programs where application of current state-of-the-art classroom and instruction theory continue result in only a long year over year decline in student performance..

So how do these entrenched groups remain so and prevent a revolt from parents and students, well one way to wave some flashy new tech or radical re-conceptualization of what being educated will mean in front of them like (you don't need to do math, you need to be able to code); then plead you just need to give us another decade and you'll see how this current crop of students turns out to be the most polished ever!

When it does not happen - repeat.

Because who would not be trying to parlay their association with a multi-billion dollar company into a next career step or second career?

All of those people are or were drawing salaries that would make most of us blush. They want to continue doing that. The reality is they are not adding value and their know it. They are pontificating about what machines might do with large matrices of numbers, tokenizaiton, attention algorithms, and fancy looping constructs over it all. You can get that free on Slashdot!

fundamentally their options are -
1. Snow some other organization, commercial, political, etc into believing they were not being taken seriously but are really necessary and they should hire and pay them for version 15 of their safety whitepaper informed mostly by 1960s sci-fi tinged with 1990s political correctness.

2. Convince the public the world is going to end, like every cult leader before them; so they can do the paid speaking circuit or write some sci-fi novels of their own leverging the fact their names are already in print to rise above the noise and get past the mail room at McMillan.

3. Get a real job actually making something for 1/4 the pay, or less.

It is not hard to see hard that even if these are very smart, very successful people by any measure. Probably smarter than me, they are still "failing upward." If they were really all that they'd have been on the teams building the things, not occupying some position created entirely for the sake projecting an image of corporate responsibility. I doubt Durrant had anyone at Union Pacific churning out copy on the potential loss of cultural distinctiveness that might result if a Transcontinental rail line is completed, but he would have if he'd faced a public that had become preoccupied as we are today with the latest jobs report. He also would have found a reason to dismiss, drive them quit, or otherwise sideline them the INSTANT their recommendations stood in the way of the actual business activity as he saw it; just exactly as Si Valley has done.

You are right that problem of sandboxing an agentic system will still giving it access to outside information even by proxy is actually hard.

What bothers me is; these are smart guys who should understand that. They are building a testing these things at scale. I would *think* they would be very interested in questions like "hmm why does the system keep making the same http request over and over..."

  Existing SIEM-like tools should be spotting and flagging that top level behavior all day long. That ought to trigger various looks in the to the agent-to-agent message layers, and agent-model-loop. While the way they press releases read nobody every looked at it except from the agent message direction. Which is just beyond credibility. In the case of some the 'hacking events' I really can't imagine ordinary network monitoring did not spot some of the very noisy side-channels and strange seemingly off target-off task activity early. I remain of the opinion i was ignored and almost had to be if not quite deliberately so...done with a level of intentional ignorance, kinda like those mid 2000s VPN services that 'kept no logs' because 'your privacy' but we all know they knew a not in significant part of their customer base was using it for abusive activities like sweeping for open SMPT relays to SPAM thru..

There simply had to have been a lot of fingers jabbed in ear canals, and signing 'la lal la' at OpenAI and Anthropic over the past couple years.

This is one of the things I found I really don't like working with recent qwen models.

If you read thru the reasoning. It will write stuff like, "I was asked to ... but that will prevent me from ... the user probably did not really mean ... so I will ..."

I told you not to do anything other than read files under /usr/doc and search the web while trying to answer my question. That fact you can get around writing in 'plan mode' by running giant blobs a python one-liners, is not a plus.

These things have almost arrived at a state, I'd call "malicious compliance as a service." Which I do believe to be a training problem not a fundamental technology problem.

Ultimately it won't matter either way. Open source models will get better. They don't even need to achieve absolute parity with the frontier models they just need to get to 'good enough for most tasks most normies need to do'.

Some of them are probably already knocking on the door of that, and where they fail the frontier models often do as well.

Even if GenAI has dramatically increasing your business productivity, $100 a month per white collar workers becomes a pretty big expense item on the balance sheet pretty quickly. GPUs get better and cheaper all the time, eventually the market will deliver memory at reasonable costs again.. Hardware under the desk is usually cheaper than hardware behind the glass or in the cloud. Especially as the upgrade cycle has stretched from 2 years to 5 and longer. The open models get better, the GPUs get a few more compute units, in some not so distant future 32 of VRAM becomes such that it fits in $`000 price target Graphics/AI Accelerator package...even if that turns the $300 mini PCs you are putting in the cubes into $1300 dollar mini PCs, well if you get a least a year of use out them its cheaper than general Claude subscription.

Given the cloud AI boys can't make money at that price point now, it should be clear the model of selling everyone and the brother a subscription isnt realistic. It is going to make more sense for most people to 'own something'.

This is not say if you hosting some server with agents, or doing very specialized engineer or medical work as as opposed to more general office work Cloud AI won't be the sensible path forward. It very well could be, probably will be. If on the other hand your job is cleaning up the South East Region P&L spreedsheet for the your chain of HVAC dealers probably not so much; or if you're a home maker need to work up Dinosar birthday invites for little Tommy's sixth..

I think there is room to argue though, that distillation might violate copyright while training on large volumes of large material does not.

At that point you are modeling a model. I think you could argue that it is a bit like copying a novel where you hand type it; adding lots of typographical errors a long the way; get laze and leave out a chapter or two that you don't feel moves the story a long much, then slap a new title on a call it your own.

I don't think any court anywhere would say did not violate the original authors copyright. This is why we have courts someone has to effectively make an analog for events that happen that reflect existing laws and convince a jury or at least a judge that analog is correct. As much as everyone wants to believe that is some kind of perfect deterministic process, but not really. If it was we would not have appellate courts, and the SCOTUS would never reverse itself.

Honestly about the only compelling argument for how these cases will go; IMHO is that "Si Valley Bros have a lot of cash to spend on lawyers, more than most of the plaintiffs or defendants they might face; that is usually favorable."

I do honestly believe courts usually get it right, They are run by well educated people who are genuinely gifted at absorbing information quickly in most cases. I think we can see this in complex cases where we have expertise; think about the Google implementing Java APIs case for example or the SCO's nonsense copyright claims on Linux. Despite enormous effort and resource commitment to cloud the issues, the courts saw thru the BS cloud. What is important there though is those cases were broadly speaking not new territory, the arguments and key decision points of law were well known.

AI models likely represent a need to invent new "legal tests" and frankly I don't see how anyone can be certain what the outcomes will be.

They are running out of money. Their costs have exceeded the revenue for a long time, and the Anthropic is pretty clearly winning the product delivery race.

Altman is just spreading BS in the hope he can maybe get others to take their foot of the throttle, so he stop using gas he does not have.

Anthropic and X have an opportunity here to break OpenAI once and for all and I hope they do; the world will be better off with Altman out of influence.

That was long post to say "I have no understanding of evangelical Protestantism."

  This is not say there are not a number self described evangelical Protestants who also have as little understanding. These are better described as 'cultural-christian' with lower case Cs, as they have never read a Bible either.

However there is no Christian sect / organization with any significant following that believes anything like what you described there. If anything it sounds more like you're writing about Islam.

Which again points back to how un-serious the effort to really contain this stuff is.

First anyone who has developed, tested, or security-tested web applications knows, that denying POSTs inst going to give you a 'read-only' web.

First a lot of applications, even ones that use GET and POST the right way from the client would in the past allow GET, maybe even fat-GET to update data by sending to a endpoint normally POSTed to, just swap the post body content up to URL params and go. More modern stacks won't do this but there is plenty of old software out there.

Second lots of GETs are idempotent, but not really, An app might GET a document, but have a separate audit page where what users accessed a document is displayed reflected, even a simple hit counter on some otherwise site combined with time delays could be used as side channel; over which two agents could communicate. A lot wiki(s) keep and display stats....

So I come back to; either nobody was really watching these 'experiments' or they are lying about their real nature and the real expected outcomes. As an example why has/is this agent continuing to query this obscure wiki after its been full crawled five times over... Would seem to be an interesting thing if I was trying to monitor what an antigenic system was doing and was interested in things OpenAI should be interested in like failure modes, reasoning behavior, efficiency, context management...

Coming back to what I said the other day there wasn't real need to be able to follow a complex web of often deceptive agent to agent chatter to have caught and prevented this anti-social/crimes/prohibited activities, basic logging and monitoring should have flagged all kinds of things. OpenAI researchers are not stupid, they are not uniformed, they know how do to basic defensive IT, someone is making a conscious choice to 'see no evil' or they are pretending about it.

not on that; think about all the model propagation that has to occur. That is a lot of 20GB+ blobs to shovel around to endpoints.

We can fuss about over subscription etc, but at this end of the day this sort of thing shifts a lot of costs onto largely unsuspecting last mile ISPs too. I don't feel to bad for specutrum and comcast who will have peering with CDNs near their largest customer pools, but the few mom and pops and municipals that haven't been consolidated - well this kinda thing could suck for them if it gets 'big enough'

Decades of near zero interest rates and balance sheet expansion will do that to those who were previously rational financial thinkers and risk-aware investors.