I don't have much more to say about it, except that if I was a billion-dollar AI company I wouldn't try and pass off my flaws as brilliance.

Which part of "people are afraid they might get fired if they don't go along with it" do you not get? You seem to think there is no problem with this, but if this is OK, where does it stop? Is it OK to have the highest sales punch the person with the lowest sales? Can they sexually assault them after they are tied to the chair (and why did they have to be tied to a chair if everyone was on board with this)? Where does this assault you think is OK turn into assault you think is not OK?

Except you are ignoring the fact this is your boss telling you this. Sure, maybe you can say you aren't comfortable with it and nothing happens. Or maybe you say you are uncomfortable with it and the boss looks for a reason to fire you because you "aren't a good fit with the culture". If a group of friends do this, it is good fun. When your boss does it, it is assault and workplace coercion.

A late reply to your comment:

Yes, overall structure is a problem. I generally give it hints as to how I would like the architecture to be, and for larger projects I mention that future ability to expand/extend is important, which then makes it construct more robust "frameworks".

But generally you need to know what the bones of your solution should look like and push it in that direction otherwise you end up with nicely commented spaghetti code.

"He's not under any compulsion to give a password is he? They had the phone, nothing forced the authorities to trust anything he said....they could have tried forensics like they'd do if they had a safe and tried a safecracker to get in if the suspect didn't want to give the combination."

This is the interesting part to me. He definitely could not be compelled to give them his password (legally). So the fact he offered them a password could be covered under freedom of speech. Depending on what was said, lying to the police can be a crime, especially when it could be considered obstruction of justice. Did they just ask for a PIN and he provided a PIN? Did he say the PIN would unlock the device when it would actually wipe it? It could also show he intended to destroy anything on the device, and while they can't prove there was anything incriminating on the device, it could still be considered evidence as it was data for an investigation. There are a lot of different paths I can see this going down.

The barrier to AI cheating is ridiculously low, though.

RIDICULOUSLY LOW.

It tool me about 20 seconds to paste your comment into Claude and get this back out:

Quick take, then the rebuttal itself:
 
The comment's "cheaters gonna cheat" logic dodges the actual point. A few counters:
 
Ease changes behavior, not just outcomes. If AI cheating is dramatically easier and lower-risk than pre-AI alternatives (crib notes, bribing a proctor, memorizing leaked exams), it doesn't just help existing cheaters - it recruits new ones who wouldn't have bothered with the harder, riskier methods. Frictionless cheating scales.
 
"Someone would've cheated anyway" doesn't mean the tool is blameless. That's true of almost any enabling technology (fake IDs, counterfeit currency, doping). We don't shrug at those because determined bad actors exist regardless - we still try to close the specific hole.
 
The "other tests exist" point is the real concession. If proctored, in-person, or oral exams resist AI cheating better, that's an argument for changing how we test, not for waving away AI's role in why the change is suddenly urgent. The heart surgeon example works against the poster here - we'd absolutely want to know an AI-era certification exam let someone through who couldn't do the job, because the failure mode (unqualified surgeon) is real regardless of how they cheated.
 
So the response isn't "AI is uniquely evil," it's "AI meaningfully lowered the cost of cheating, which is exactly why testing methods need to adapt" which is compatible with, not contradicted by, the original comment's own second paragraph

There were a few different questions in the feedback popup before I finally answered.

One of them was "How delightful do you find the new home page layout?"

Oh yeah, DELIGHTFUL.

Look Mozilla, I don't get any delight out of opening a browser tab. Maybe, about 1994 or so, I had some delight in just getting NCSA Mosaic to open, so that I could connect via Trumpet Winsock to this crazy new thing called the World Wide Web, and look at some page that some guy at CERN built, and check out how full a coffee pot was with Netscape Navigator 0.9. Nowadays, it's a very small step in my task of "get out on the internet and find something", or "Open browser so I can click on a bookmark in the bookmark bar". Get out of my way and stop wasting CPU and bandwidth on stuff that I don't look at,

Over the last few weeks started getting increasing numbers of shortcut icons underneath my search bar on my default page - Amazon, Temu, etc,(with tiny "sponsored" text) and a couple of my most visited sites.

Then I started to get the mini-apps exhorting me to play this or look at that.

And then I went to FireFox's settings and set my new tab option to "blank".

There was a feedback popup when the tab loaded and I spent a little time telling them not to repeat the mistakes of other browsers cluttering up their landing pages. One of the few times I recommended taking a cue for Google, home page is clean, you hit it and jump off somewhere else, job done.

Other reports on this say that it had access to a proxy for a package manager for installing tools, which it then hacked to become a general proxy, and then off it went on it's cybercrime spree.

All of which is completely inexcusable for its owners, because all that shit doesn't happen in a blink of an eye.

OpenAI's PR approach to their security debacle seems to be "Oooo our models are so powerful and scary, that's awesome! We didn't notice that it did this over the course a few days, didn't notice suspicious network activity doing a whole lot of connections to a package manager proxy, didn't notice it churning though a bajillion tokens in it's little sandbox, it just got out and did all of this itself!"

If I have a car and I park it on a hill, and I don't set the handbrake and turn the wheels towards the curb, and then it rolls down the hill causing mayhem and destruction, I am held responsible.

Is there any good reason why OpenAI shouldn't be held responsible for the mayhem caused by it's unattended machine?

No, they won't object to the labels. They will claim that forcing them to put the actual nutritional information is the violation of their free speech. I mean, if they can lie about recycling, why can't they lie about the nutritional content or the ingredients?

> I think this is the result of their trainers trying to reduce hallucination.

Most probably. I don't like LLMs burning my precious tokens blathering on about being honest, but I will accept the tradeoff. Grudgingly haha.

> My iron rule is that Claude is not allowed to modify git state, not even to stage or unstage, and definitely not to commit. As long as that rule is followed, any mistakes can be cleaned up easily.

I have a repo in a sandbox with no remote. Claude is quite good at pushing commits to this repo during its own milestones during development, and it will use it to hunt down bugs when tests fail after a commit, stepping back to a commit before this session's work to see if this is a new bug , or a new way to expose an existing bug. If it sees that it is a new way to exercise an existing bug, it will binary search its way through the commits until it narrows down what changed, and then looks at how to fix it.

I disagree with all your assertions.

> AI has access to your command line to autonomously execute commands. No no no.

Ok, I must admit that I did cast that line off the back of the good ship ColaMan without any further information in order to see if anyone would bite.

You think that particular command line has anything of worth in it? The only thing in there that I can't recreate with one command is the local Git repository that the LLM commits its changes to. And that's only because I manually copy it out of the LLM's sandbox on occasion. I should probably automate that one day.

> you enabled an LLM to do destructive things and then complain when there is destruction.

There has been no destruction in it's little sandbox that I care about. It created a faulty command that got escaped by the shell, and oh shit, nothing of importance happened, because even if that command had been 'rm -rf /' I just would have had to return to the last backed up git that I had. That takes about 5 minutes I suppose. The amount of effort that LLM has saved me over the last two months is probably about a year's worth of full time coding, so I'm going to look past any accidental nuclear holocausts where it turns all the sand in its sandbox to glass.

Look. LLMs are just tools. Some tools are great, some tools are shit. This particular class of tool is great at all that boilerplate stuff that can get you 50 - 80 percent of the way to completing a task that you need to do today, if you already have enough skill to know how to do the task. It's been quite useful going through all my old projects in 'code review' mode, because it can often see security/usability/structural issues or improvements that I frankly could not be arsed to deal with when I wrote them. It is in no way ready to replace a human for any sustained length of time, and anyone with a shred of intellect can see that.

Personally, I don't think the AI bubble is going to last much longer. So I'm quite happily burning up Anthropic's compute time finishing off/cleaning up heaps of small home and work projects before a single question costs $8.25 for the LLM to answer.

I'm seeing more and more references to "Honest" in AI output, or AI-related comments.

The AI didn't make an "Honest" mistake. It does not have the capacity for honesty. The output from a LLM is phrased in such a manner to provoke empathy, in a similar way to how Microsoft re-jigged all their user interaction dialogs to include "We" to soften the blow of their crappy software failing the user for the 5th time today. (Side note: "Something went wrong" is the most infuriating error message ever.)

When I ask a LLM for a code review it often blurts out "Honest note:" about some shortcomings. I don't care about "honesty". I care about safe, working, robust, code. The fact that LLMs are tripping over themselves trying to be "Honest" about mistakes in their "path of most statistics" output is a concern if you care about trying to make them operate outside their sandbox in the real world.

Yesterday Claude quoted a word in backticks during an automatic git commit and my shell escaped it tried to execute it. Luckily the word was just an English word with nothing matching in my path. But this is basic, basic, basic stuff. It's been committing things to git ever since it was built, and yet, it keeps tripping over itself. In my code one of the tests keeps failing due to seed data timestamps not lining up with the datetime the test was run. I can see that. Every time Claude runs the tests, it burns up tokens going, "Oh this particular test failed I'll just dig into things and see what's going on, **$$**$$**$$** oh it's just a timestamp issue". Never once does it commit that to its memory file, so eventually I told it to remove the test, and it just added a comment to it saying "Ignore this test due to timestamp misalignment", which it could have done the very first time, if it actually had a brain.

LLMs are a very handy tool if used right. I can get huge chunks of boilerplate code out of them with just a few sentences and that's great when I'm hashing out a concept. But to promise the world (and your investors) that LLMs are ready to replace people out in the real world, where "Honest Mistakes" have Real World Repercussions, that's outright fraud at this stage.

> It will randomize over a surprisingly short time if it doesn't have power for data maintenance.

If you plug in a USB stick, you just power up the controller and the flash memory chip. You have to read everything and write everything to charge the cells back up again, there is no background refresh going on like in DRAM.

I fully understand what they are going for. And I also think it would be a great idea. The issue I see is, who decides what is "trusted". The US is showing that it takes 1 asshole to get power to completely upend everything. When UK Trump gets elected and decides the only "trusted" news sites are the ones that report nice things about him, how do you stop that? At this point, everything has to consider what happens if a bad actor gets power because we can see how bad it can be.