For a small model (think handwriting recognition, say, 6x8 pixels), it is actually possible to construct a formal proof that the model will correctly identify digits and characters correctly. However, it is usually more expedient just to test with a large sample of typical inputs. In such cases, an error ratio smaller than what a human would produce, and the low risk of failure makes this approach acceptable.

However, for LLMs and models of the size typically used in self driving, it is practically impossible to verify the model (think heat death of the universe) using formal methods. The best that one can do is identify the desired outcome and perform statistical tests to determine the error percentage. When it comes to applications such as criminal sentencing, self driving, and applicant selection where mistakes expose the model user to substantial liability, AI may be more appropriate in an assistive role where a human must be in the decision loop. Even so, there are yet a large number of AI applications where mediocre performance (i.e. good enough, with occasional errors) is in fact good enough to justify the cost savings.

It is programs like the one proposed here which represent a structural danger to society: because of doctrines like sovereign immunity, a state which certifies a model as "safe" does not actually have any responsibility for the harm caused by such a representation. That is, a company which relies on the state's safety certification can avoid liability for harm caused by the model, and the state has neither the responsibility nor the incentive to test the model in an intellectually honest and formally rigorous way. Thus, the cost of testing the model is borne by the state, the liability for mistakes is borne by the public at large, and the disproportionate profit such an arrangement facilitates accrues to the benefit of private corporations.

Guaranteeing that AI won't cause harm is a matter of making sure it can't control anything that could cause harm. Think air gap. Think novelties, jokes, etc... There are many good uses for AI that literally can't harm someone, but I strongly suspect NY intends something a bit more nefarious and needs the imprimatur of an official state agency to secure the trust of the public.

We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."

I know it's old hat to talk about traditional software engineering techniques, especially when there's the taxpayer's purse to loot, but there already exist solutions to these problems. I get it - these folks would rather have someone else pay for their ignorance, but the beauty of AI is that it can be counted on to break in new and novel ways. It's funny when AI swears when autocorrecting texts, but if you should find yourself blown to bits because you let AI control your air defense network, it's hard to blame anyone except yourself.

AI is not some novel type of technology outside the realm of traditional formal proofs and testing regimes. The problem is that even testing conventional systems is expensive, and AI systems even moreso. So, the solution - in the case of New York - is to get the taxpayer to foot the bill (thereby socializing the losses) so that the profits can be privatized. The canard of "public safety" is hung like a sword of Damocles over the public at large so that those who would loot the public purse can do so without arousing the suspicions that they are doing just that.

It is remarkable how much harm can be prevented by autonomous systems when the politically-well connected are held responsible for their failures. Should Black Rock executives find themselves in prison because one of their companies' AI systems accidentally microwaves a kitten (hypothetical example), the necessary corrections will be made post haste. But until that day, we can safely predict that the rich will never miss an opportunity to transfer responsibility from themselves to the public at large, and money from the public at large to themselves.

How much has your government money grown in that time?

Illegal doesnâ(TM)t mean I donâ(TM)t have the right to record, tho. Government doesnâ(TM)t give rights.

We're running a business here. We need real software, not toys created as a fun weekend experiment with a chatbot or burning money to run an agent and get 50% (or more likely 5%) of the way there.

The real world is more complicated than that. For example, we still have a Windows 7 machine in my office. It has some very expensive but still very useful software locked to it that has no direct replacement available on either Linux or Windows 11. Obviously we have to be careful about security and controlling access to a legacy OS like that.

Blunt statements about "forcing an upgrade" are worthless in this kind of situation. Chances are anyone who is still using a system like that has a good reason for it, at least if it's in use professionally. We buy these assets because they solve problems. Those problems might or might not have a useful and cost-effective solution available on a different platform. If there is no modern solution available and the original still works, your statement that "the old ways are obsolete" is by definition false.

What you do need to do is rationally assess the risks and the threats you face and provide other ways of mitigating the risks and protecting against the threats if you can't rely on, for example, a legacy OS that no longer gets security updates to be secure by itself. But then ideally you shouldn't be relying on any single device or piece of software being secure as your only line of defence anyway, and assuming you already have some degree of defence in depth, accommodating unusual requirements is more a redistribution of security responsibilities than a fundamental change of approach.

Yes, it does look like we broadly agree here.

I suppose the main thing I'd have said very differently is the terminology, in that I wouldn't regard the kind of incompetent senior as a senior in the first place. But then I'm pretty old school about this: to me, a junior developer needs quite close supervision to produce useful results, a regular/mid developer can perform routine tasks with minimal supervision but might need more help with larger or more complex tasks, while a senior is someone you can give more or less any task on your project to and they'll figure it out, and senior is also the level of someone who would be capable of supervising and training a junior.

We've had a wave of people, particularly since the tech boom during COVID, who have obviously been over-promoted due to the employee's market and title inflation. They might now have titles like "senior" or even "lead" or "staff", but they don't have the skills and experience to operate successfully at those levels. I agree the acceleration of everything due to increasing AI use tends to show these people up and I agree that a good mid-level developer using AI might well outperform them. But then I suspect in reality a good mid-level developer was probably outperforming them before AI as well, it just wasn't so obvious.

That is wildly optimistic IME. Frontier models do a great job of sounding authoritative on any given subject. This is not the same as actually having expert-level knowledge and understanding, and the choices the models make when you ask them to implement something can and often do reflect that. There are several domains I've worked in over the past few years where AIs lack even basic knowledge an expert practitioner would have, probably because these fields are too specialised to have a lot of books available or papers published, so presumably there hasn't been much subject-specific content to train the models on.

I don't think it's usually about what AI "did really badly on" at this point. It's more about whether the overheads of adding all the guard rails and then reviewing and correcting all the output end up costing at least a similar amount to just writing the code yourself.

For a junior developer, I'd say frontier models passed their standard some time ago. Today's LLMs routinely write code at the level of a competent junior-mid but with the advantage of an inhumanly broad knowledge base.

For a good senior, I'm much less convinced. The frontier models are hit-and-miss when it comes to writing good test suites, which matters a lot when they rely on tests so heavily to reach acceptable quality. IME, they are still mostly miss when it comes to the business insight that seniors will take into account and design/architecture decisions that keep the code manageable as it evolves. While they have become much better at recognising existing patterns in the code, and I would say they follow them reasonably well the majority of the time now, there are still bizarre anomalies that creep in, and they aren't very good at extending those patterns when necessary.

I think AI advocates also tend to underestimate just how much time it takes to properly review what they are doing. Take a half-day Claude session to implement some new feature. It might take half an hour to give it the initial brief and then review its plan. Make a few edits and it'll propose a new plan that is also long and detailed and has no edit markings to show what it changed so you have to review the whole thing, carefully and in detail, again to pick up the minor changes it also made for no obvious reason. Then you let it start doing its thing. You can't really leave it unattended for long because it'll stop to ask questions. Some of those questions might be whether it can run a screenful of ad-hoc Bash script to make some edits to your source files, because for some reason it decided not to show the actual edits as it usually would, so now you have to figure out what all the sed incantations are actually going to do while looking at those files in another editor or just trust it and then review all the changes properly later anyway. Sometimes it decides to change its plan mid-flight, but not always in a good way, and if you interrupt it to apply corrections or redirect, you break the neat plan flow it was in. Then you get to the end and you can finally see how everything it has done fits together, and you still need to review everything with the same care as you would with a junior developer's code and probably apply a few fixes where it's come up with weird names or not quite followed a pattern or imagined some instructions that were never there and made other changes in other parts of the system that should be out of scope.

If you read to the end of that last paragraph, you know how I typically feel at the end of a half day working with an agentic UI. And the thing is, I do those sessions as experiments fairly often to keep up with progress, but a lot of what I do is still written by hand, which gives me simple, objective data to compare the two ways of working and see whether the AI tools are really making me more productive once you take everything into account.

Currently, based on objective data comparing similar fully completed development tasks like for like, I still outperform agentic workflows using frontier models by about 25-50% on average, and I don't have to pay anyone extra fees to do that, and subjectively I also find it more engaging to work that way. So I don't really buy the hype about AIs making us all orders of magnitude more productive unless you're happy to YOLO everything forever and take the much-discussed risks that comes with.

Personally, I think the optimum use of LLMs right now is probably not to use them for the whole coding workflow like that or rely on agentic tools, but to use them for tasks like answering quick questions about the existing code, doing research like a fancy search engine, writing a first draft of some boilerplate, and writing quick and dirty throwaway code for experiments, one-shot tools and the like, where you don't really care about the quality as long as it gets the job done and you will see immediately if it has in fact got the job done. For these things, the LLM can read and type faster than me, and that's useful.

Finding a hotel that can accommodate a family of 4 is not that difficult. Finding one that can accommodate a family of 8 is a bit more challenging. Finding one which can accommodate 8 adult siblings, their children, spouses, etc... is nearly impossible. The short-term rental market is the only realistic option for those people who want to meet with the rest of their family for an extended weekend.

Hotels are set up for small gatherings (an individual room or two), large gatherings (hundreds to thousands in a conference room), but seldom in between.

The last time my in-laws got together, we numbered almost 40 people. Airbnb rentals provide accommodations which are much more practical for those people who want to spend a holiday together with their geographically distributed relatives. Yes, the cleaning can be substantial. But making guests both clean the place, and then pay an additional cleaning fee (in the hundreds to thousands of dollars) seems ridiculous.

I fly 200ish flights a year so figure 70-80 nights away.

2025 was 100% hotels but 2019 was about 80% Airbnbs.

Airbnb will never get my business back. Ever. The professional scammer hosts are so good at finding a way to nail me for a fee. Iâ(TM)ve taken perfect check out video and still had to spend hours fighting fees.

Airbnb allowed making guests do it.

They lost so much money to hotels theyâ(TM)re banning hosts from requiring it.

It was probably the other O'Neil, with one L. He has no sense of humour at all.

I could understand bots and mailing lists. The thing I never understood with LinkedIn was how many real people (usually recruitment agents) had apparently spent the time to read my profile and send me a personalised message, yet still thought I'd be interested in a job paying half the market rate for someone with my skills and experience.

I don't know what you mean. Look at the improvement in hiring when you can advertise an open position on LinkedIn and get maybe 100x as many applicants as before! OK, probably 99% of them will be completely unqualified for the role and barely capable of pushing the instant application button, but how is that a problem, really? Surely hiring managers can easily filter them out and no qualified applicant will ever get lost in the noise?