It'll be Turtles All the Way Down. :-)

Flock will install camera cameras to watch their traffic cameras.

Because who would not be trying to parlay their association with a multi-billion dollar company into a next career step or second career?

All of those people are or were drawing salaries that would make most of us blush. They want to continue doing that. The reality is they are not adding value and their know it. They are pontificating about what machines might do with large matrices of numbers, tokenizaiton, attention algorithms, and fancy looping constructs over it all. You can get that free on Slashdot!

fundamentally their options are -
1. Snow some other organization, commercial, political, etc into believing they were not being taken seriously but are really necessary and they should hire and pay them for version 15 of their safety whitepaper informed mostly by 1960s sci-fi tinged with 1990s political correctness.

2. Convince the public the world is going to end, like every cult leader before them; so they can do the paid speaking circuit or write some sci-fi novels of their own leverging the fact their names are already in print to rise above the noise and get past the mail room at McMillan.

3. Get a real job actually making something for 1/4 the pay, or less.

It is not hard to see hard that even if these are very smart, very successful people by any measure. Probably smarter than me, they are still "failing upward." If they were really all that they'd have been on the teams building the things, not occupying some position created entirely for the sake projecting an image of corporate responsibility. I doubt Durrant had anyone at Union Pacific churning out copy on the potential loss of cultural distinctiveness that might result if a Transcontinental rail line is completed, but he would have if he'd faced a public that had become preoccupied as we are today with the latest jobs report. He also would have found a reason to dismiss, drive them quit, or otherwise sideline them the INSTANT their recommendations stood in the way of the actual business activity as he saw it; just exactly as Si Valley has done.

You are right that problem of sandboxing an agentic system will still giving it access to outside information even by proxy is actually hard.

What bothers me is; these are smart guys who should understand that. They are building a testing these things at scale. I would *think* they would be very interested in questions like "hmm why does the system keep making the same http request over and over..."

  Existing SIEM-like tools should be spotting and flagging that top level behavior all day long. That ought to trigger various looks in the to the agent-to-agent message layers, and agent-model-loop. While the way they press releases read nobody every looked at it except from the agent message direction. Which is just beyond credibility. In the case of some the 'hacking events' I really can't imagine ordinary network monitoring did not spot some of the very noisy side-channels and strange seemingly off target-off task activity early. I remain of the opinion i was ignored and almost had to be if not quite deliberately so...done with a level of intentional ignorance, kinda like those mid 2000s VPN services that 'kept no logs' because 'your privacy' but we all know they knew a not in significant part of their customer base was using it for abusive activities like sweeping for open SMPT relays to SPAM thru..

There simply had to have been a lot of fingers jabbed in ear canals, and signing 'la lal la' at OpenAI and Anthropic over the past couple years.

This is one of the things I found I really don't like working with recent qwen models.

If you read thru the reasoning. It will write stuff like, "I was asked to ... but that will prevent me from ... the user probably did not really mean ... so I will ..."

I told you not to do anything other than read files under /usr/doc and search the web while trying to answer my question. That fact you can get around writing in 'plan mode' by running giant blobs a python one-liners, is not a plus.

These things have almost arrived at a state, I'd call "malicious compliance as a service." Which I do believe to be a training problem not a fundamental technology problem.

If we're lucky, maybe AIs will just develop a taste for Black Jack and hookers.

I felt that way about Emacs - as a *long* time user.
Despite its logo, I still can't use it to wash my dishes. :-(

Also, Nvidea isn't the one who'll get sued if (when) an AI agent does some actual damage to someone/thing. They'll probably just suggest AI companies buy even more Nvidea chips to combat the issue.

Seems like some people only care about things, like privacy, rules, etc... when it's concerning and/or affects them. I expect more from law enforcement officers -- any public servants really -- but am not naive enough to believe we'll (always) get that. A lot of dickheads are drawn to positions of power/authority over others.

> It doesnt think

>> Yeah, it does. https://transformer-circuits.p...

?

In the paper, in the text itself, whenever the authors use the word think to refer to a human thing they don't put think in quotes and when they use the word to refer a description of what an LLM is doing they always put "think" in quotes.

I didn't find an explanation why. Are they in effect acknowledging that they don't mean the same thing by the word in each situation and want to avoid a kind of equivocation? Near the end of the paper they add another typographical feature to think: italics.

"Progress in AI is birthing a new kind of intelligence, reminiscent of our own in some ways but entirely alien in others. Understanding the nature of this intelligence is a profound scientific challenge, which has the potential to reshape our conception of what it means to think".

Not only slow and cumbersome for an in-store purchase, but probably dangerous w/o the people on Ark B.

Interesting, hadn't thought about risks.

Thanks! I'm not a bot, the name is just mixed slang from Farscape and Futurama. My post history apparently goes back to November 10, 2008, though I would swear I posted earlier than that. I know I've been reading /. since 2000, though some of that was lurking w/o an account...

Ultimately it won't matter either way. Open source models will get better. They don't even need to achieve absolute parity with the frontier models they just need to get to 'good enough for most tasks most normies need to do'.

Some of them are probably already knocking on the door of that, and where they fail the frontier models often do as well.

Even if GenAI has dramatically increasing your business productivity, $100 a month per white collar workers becomes a pretty big expense item on the balance sheet pretty quickly. GPUs get better and cheaper all the time, eventually the market will deliver memory at reasonable costs again.. Hardware under the desk is usually cheaper than hardware behind the glass or in the cloud. Especially as the upgrade cycle has stretched from 2 years to 5 and longer. The open models get better, the GPUs get a few more compute units, in some not so distant future 32 of VRAM becomes such that it fits in $`000 price target Graphics/AI Accelerator package...even if that turns the $300 mini PCs you are putting in the cubes into $1300 dollar mini PCs, well if you get a least a year of use out them its cheaper than general Claude subscription.

Given the cloud AI boys can't make money at that price point now, it should be clear the model of selling everyone and the brother a subscription isnt realistic. It is going to make more sense for most people to 'own something'.

This is not say if you hosting some server with agents, or doing very specialized engineer or medical work as as opposed to more general office work Cloud AI won't be the sensible path forward. It very well could be, probably will be. If on the other hand your job is cleaning up the South East Region P&L spreedsheet for the your chain of HVAC dealers probably not so much; or if you're a home maker need to work up Dinosar birthday invites for little Tommy's sixth..

I think there is room to argue though, that distillation might violate copyright while training on large volumes of large material does not.

At that point you are modeling a model. I think you could argue that it is a bit like copying a novel where you hand type it; adding lots of typographical errors a long the way; get laze and leave out a chapter or two that you don't feel moves the story a long much, then slap a new title on a call it your own.

I don't think any court anywhere would say did not violate the original authors copyright. This is why we have courts someone has to effectively make an analog for events that happen that reflect existing laws and convince a jury or at least a judge that analog is correct. As much as everyone wants to believe that is some kind of perfect deterministic process, but not really. If it was we would not have appellate courts, and the SCOTUS would never reverse itself.

Honestly about the only compelling argument for how these cases will go; IMHO is that "Si Valley Bros have a lot of cash to spend on lawyers, more than most of the plaintiffs or defendants they might face; that is usually favorable."

I do honestly believe courts usually get it right, They are run by well educated people who are genuinely gifted at absorbing information quickly in most cases. I think we can see this in complex cases where we have expertise; think about the Google implementing Java APIs case for example or the SCO's nonsense copyright claims on Linux. Despite enormous effort and resource commitment to cloud the issues, the courts saw thru the BS cloud. What is important there though is those cases were broadly speaking not new territory, the arguments and key decision points of law were well known.

AI models likely represent a need to invent new "legal tests" and frankly I don't see how anyone can be certain what the outcomes will be.