Forgot your password?
typodupeerror

Comment Re:Rent seeking (Score 2) 20

For a small model (think handwriting recognition, say, 6x8 pixels), it is actually possible to construct a formal proof that the model will correctly identify digits and characters correctly. However, it is usually more expedient just to test with a large sample of typical inputs. In such cases, an error ratio smaller than what a human would produce, and the low risk of failure makes this approach acceptable.

However, for LLMs and models of the size typically used in self driving, it is practically impossible to verify the model (think heat death of the universe) using formal methods. The best that one can do is identify the desired outcome and perform statistical tests to determine the error percentage. When it comes to applications such as criminal sentencing, self driving, and applicant selection where mistakes expose the model user to substantial liability, AI may be more appropriate in an assistive role where a human must be in the decision loop. Even so, there are yet a large number of AI applications where mediocre performance (i.e. good enough, with occasional errors) is in fact good enough to justify the cost savings.

It is programs like the one proposed here which represent a structural danger to society: because of doctrines like sovereign immunity, a state which certifies a model as "safe" does not actually have any responsibility for the harm caused by such a representation. That is, a company which relies on the state's safety certification can avoid liability for harm caused by the model, and the state has neither the responsibility nor the incentive to test the model in an intellectually honest and formally rigorous way. Thus, the cost of testing the model is borne by the state, the liability for mistakes is borne by the public at large, and the disproportionate profit such an arrangement facilitates accrues to the benefit of private corporations.

Comment Re:Rent seeking (Score 1) 20

Guaranteeing that AI won't cause harm is a matter of making sure it can't control anything that could cause harm. Think air gap. Think novelties, jokes, etc... There are many good uses for AI that literally can't harm someone, but I strongly suspect NY intends something a bit more nefarious and needs the imprimatur of an official state agency to secure the trust of the public.

Comment Rent seeking (Score 3, Interesting) 20

We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."

I know it's old hat to talk about traditional software engineering techniques, especially when there's the taxpayer's purse to loot, but there already exist solutions to these problems. I get it - these folks would rather have someone else pay for their ignorance, but the beauty of AI is that it can be counted on to break in new and novel ways. It's funny when AI swears when autocorrecting texts, but if you should find yourself blown to bits because you let AI control your air defense network, it's hard to blame anyone except yourself.

AI is not some novel type of technology outside the realm of traditional formal proofs and testing regimes. The problem is that even testing conventional systems is expensive, and AI systems even moreso. So, the solution - in the case of New York - is to get the taxpayer to foot the bill (thereby socializing the losses) so that the profits can be privatized. The canard of "public safety" is hung like a sword of Damocles over the public at large so that those who would loot the public purse can do so without arousing the suspicions that they are doing just that.

It is remarkable how much harm can be prevented by autonomous systems when the politically-well connected are held responsible for their failures. Should Black Rock executives find themselves in prison because one of their companies' AI systems accidentally microwaves a kitten (hypothetical example), the necessary corrections will be made post haste. But until that day, we can safely predict that the rich will never miss an opportunity to transfer responsibility from themselves to the public at large, and money from the public at large to themselves.

Slashdot Top Deals

Old programmers never die, they just become managers.

Working...