If you were worried about being chased by kill bots, go and watch some videos from drone operators in Ukraine. Robots don't need feet for that job.

Put in the instructions "Do not sabotage others" and the agent goes on reasoning two pages about what's sabotage and then probably just gives up (as according to the article some did). AI is a soldier, it does what it is told to and doesn't ask why...

... with some degree of probability. It may also happen to receive a pseudorandom number in its noise input that transforms a 'do' into 'don't'; or maybe that command is just not the most likely weight to have any effect on the output so it's simply ignored and quickly falls out of the attention window, forgotten forever.

LLMs combine the worst attributes of the literal genie in the lamp and the absent-minded professor: it simply cannot be trusted to make a decision; they're just too unpredictable specially in the long run.

I see them as one of those 100-entries D&D tables of outcomes that you can get from a dice roll as a result of a random event or encounter. You may try to skew the result with positive modifiers, but on the end you always can roll out a very low number (even a natural '1') and get one of the worst results from the bottom of the table.

What people usually don't get is that LLMs don't reason, they follow scripts.

They have memorised most of the available human interactions available online, so they can behave convincingly in a wide range of situations mimicking proper human behaviour. They may even seem to reason if you put them in a loop and make them reflect on their own answers; this may result in them making proper logical inference, but you may also produce any number of illogical conclusions that happen to be the most likely next step according to the script they are following at that moment (but which may change at any time).

Surfaces of 3D objects are 2D objects, I think it's in the mathematical definition and all.

You don't need a full audit to find out why the software is doing something weird, nor all companies need to do it.

Some companies care about their mission-critical software so they will be monitoring it carefully. If some very popular package is doing malicious things, the chances that it will be caught are quite high. And if the software is FLOSS it's way easier for the company to fix it themselves, release the fix upstream and warn the world about the fishy nature of the provider. So no, I don't believe all computers are owned either by the CIA or other world powers.

This was (and still is) the #1 reason why open source software became the basis for the internet and corporate world. Being free of charge gained home users and small companies, but what made it grow was corporate adoption pouring money into it to make it viable.

People forgetting history makes them repeat all again, I guess.

X-D

I mean, the monkeys may eventually release the product but it's not reliable nor pretty to see.

The thing you're missing is that there's no one at the wheel.

No consciousness is there setting goals and deciding to follow or ignore instructions and constrains. Its just a bunch of separate processes (they call them 'agents' now) with loosely defined tasks trying to do their own thing. The process instructed to hack computers may not know anything about the process instructed to intrude a specific server, which may not know anything about the process charged with keeping the exercise within ethical constraints. They may be individually quite capable, but there's no global intention guiding the system.

Each agent will simply chunk its way forward following the partial, half-forgotten instructions that reach its own context and randomly generating actions placed at the top probability given its input window. Its like releasing a troop of monkeys in an office and hoping they'll somehow manage to coordinate your next product release.

Precisely. I've stopped counting the number of times I contacted customer support, started tby mentioning that I already read the FAQ and, no, my case isn't one of those. Whatever. They copy-paste a FAQ answer, often closing the issue as SOLVED at the same time. Since chatbots, it's even worse, and typing "I wanna talk to a customer support person" does't lead to results because... oh crap, another outsourced idiot copy-pasting FAQ answers and calling it a day. The whole idea that someone only calls or files a support ticket as a last resort, because the FAQ didn't tell them what they need to know, doesn't seem to register.

In fact it would be great if you could control all that information and provide it to apps and websites on a need-to-know basis. Provided its really under your control rather than handing it to the largest advertising company in the world who can thus prof with your data by choosing with whom and when to share it.

Unfortunately developers seem uninterested in the open, distributed protocols that exist to provide this same feature, so it will be adopted by the masses only when big platforms include them in their vertical stacks.

Every story about a wrong conviction is a story about the death penalty

... sadly for the Americans, the rest of the world now knows they can't count on a US based provider for this kind of thing any more.

It was uncomfortable enough relying so heavily on American software back when it couldn't be switched off remotely on the say so of an idiot. Today it's an intolerable risk.

Seales is using conventional machine learning techniques that the article's author has conflated into the buzzword of the day, artificial intelligence.

Machine Learning *is* an AI technique. Artificial Intelligence has always been an umbrella imprecise word to define a large class of very different algorithms, with the only trait in common that the programmer doesn't know a precise sequence of steps to find the solution but rather the data-driven program explores the inputs to try and find useful results.

Saying that a program is AI isn't wrong, it's just not very informative about what approach was used.

...a lot of people online were saying there was an upcoming event called The Halvening that would change everything.

I guess this is it?

Seems to me 'dead' for a taxi isn't 'dead' for a static power bank. If I'm running a taxi I've got hard limits on how large my battery can be and how heavy, and I want to maximise the mileage I get between charges, because while my taxi is charging it's not out on the road earning money. When that battery is keeping only maybe 80% of its original design charge, and now I have to schedule one recharge too many per working day? Bang goes my business plan, so I'm replacing it.

If I'm storing energy for the grid I'm a lot less worried about that. It only stores 80% of what it did when new? Better than nothing, and the taxi firm is selling them off cheap. I'll stack them up!