Comment Re:Awesome! (Score 1) 183
The plural of anecdote isn't data.
The plural of anecdote isn't data.
NSA statement on network issues
The National Security Agency experienced temporary degradation during scheduled routine maintenance. While some internal systems were affected, mission-critical capabilities remain fully operational and our ability to defend national security continues. NSA's infrastructure is designed with multiple layers of redundancy to maintain operations during planned maintenance or unexpected events.
Copilot isn't working, either.
For a small model (think handwriting recognition, say, 6x8 pixels), it is actually possible to construct a formal proof that the model will correctly identify digits and characters correctly. However, it is usually more expedient just to test with a large sample of typical inputs. In such cases, an error ratio smaller than what a human would produce, and the low risk of failure makes this approach acceptable.
However, for LLMs and models of the size typically used in self driving, it is practically impossible to verify the model (think heat death of the universe) using formal methods. The best that one can do is identify the desired outcome and perform statistical tests to determine the error percentage. When it comes to applications such as criminal sentencing, self driving, and applicant selection where mistakes expose the model user to substantial liability, AI may be more appropriate in an assistive role where a human must be in the decision loop. Even so, there are yet a large number of AI applications where mediocre performance (i.e. good enough, with occasional errors) is in fact good enough to justify the cost savings.
It is programs like the one proposed here which represent a structural danger to society: because of doctrines like sovereign immunity, a state which certifies a model as "safe" does not actually have any responsibility for the harm caused by such a representation. That is, a company which relies on the state's safety certification can avoid liability for harm caused by the model, and the state has neither the responsibility nor the incentive to test the model in an intellectually honest and formally rigorous way. Thus, the cost of testing the model is borne by the state, the liability for mistakes is borne by the public at large, and the disproportionate profit such an arrangement facilitates accrues to the benefit of private corporations.
Guaranteeing that AI won't cause harm is a matter of making sure it can't control anything that could cause harm. Think air gap. Think novelties, jokes, etc... There are many good uses for AI that literally can't harm someone, but I strongly suspect NY intends something a bit more nefarious and needs the imprimatur of an official state agency to secure the trust of the public.
We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."
I know it's old hat to talk about traditional software engineering techniques, especially when there's the taxpayer's purse to loot, but there already exist solutions to these problems. I get it - these folks would rather have someone else pay for their ignorance, but the beauty of AI is that it can be counted on to break in new and novel ways. It's funny when AI swears when autocorrecting texts, but if you should find yourself blown to bits because you let AI control your air defense network, it's hard to blame anyone except yourself.
AI is not some novel type of technology outside the realm of traditional formal proofs and testing regimes. The problem is that even testing conventional systems is expensive, and AI systems even moreso. So, the solution - in the case of New York - is to get the taxpayer to foot the bill (thereby socializing the losses) so that the profits can be privatized. The canard of "public safety" is hung like a sword of Damocles over the public at large so that those who would loot the public purse can do so without arousing the suspicions that they are doing just that.
It is remarkable how much harm can be prevented by autonomous systems when the politically-well connected are held responsible for their failures. Should Black Rock executives find themselves in prison because one of their companies' AI systems accidentally microwaves a kitten (hypothetical example), the necessary corrections will be made post haste. But until that day, we can safely predict that the rich will never miss an opportunity to transfer responsibility from themselves to the public at large, and money from the public at large to themselves.
If you've never had a boss like this -- a lazy narcissist -- you're lucky. But if you have, the pattern is familiar and unmistakable:
* Doesn't understand the issues.
* Doesn't do his homework.
* Believes he's a genius whose "gut" overrules people who do understand and do do their homework.
* Plays his subordinates off against one another.
* Highly susceptible to information from random sources: thing he saw on TV, or heard on the golf course.
* Not particularly fussed about contradicting himself because he doesn't care about being right, but does care about being in charge.
* The more chaotic he can make things, the more in charge he feels.
* Has a cynical view of the people he works for but has a genuine gift for flattery when it suits his purposes.
Once you've seen this up close it's really obvious when you come across it again. You can't *not* listen to a lazy narcissist boss because he absolutely will try the stupid shit he says he will. But you can't count on him either, because he has no commitment to anything other than how he looks and feels *right now*.
Neither. Commerce didn't drop it, it got taken away and given to DoJ, but not because Commerce couldn't handle the invetigation of where the chips were going. The case goes to DoJ because we're talking about criminal charges of smuggling, corporate fraud, violation of national security statutes. Also because the Commerce investigation was tained by potential political interference. And possibly because it ran afoul of a counterintelligence operations siloed in the DoJ -- although by this point the cat is so out of the bag everyone must assume they've got a FISA wiretap warrant on them.
However, the *effect* may be to slow the progress of the investigation and delay any actions being taken.
The first reason for this is that criminal investigations are slower, the standards of proof higher, and potentially involve a criminal trial. Commerce could simply make an administrative law determination that put Megaspeed on some shit list and come to some kind of agreement with NVIDIA.
The second reason is that putting this into DoJ's hands is putting it into a department that has had staff turnover and loss of institutional knowledge, strife between career personnel and political appointees, and difficulty replacing lost employees. It's also led by lame-duck appointees, facing turnover in the next administration. So if they don't make progress in the next two years, allowing for transition delays, it could be years before anything gets accomplished. By then a lot of people will have left their jobs, memories will get hazy, evidence will go missing, and defensive positions constructed.
So while the investigation hasn't been *killed*; it has been delayed so long that it's effectively in suspended animation. It's not possible for Commerce to sanction anyone because the investigation is out of their hands.
Money may be an aphrodisiac but come on, Jensen, look in the mirror. There's limits to anything.
How much has your government money grown in that time?
Illegal doesnâ(TM)t mean I donâ(TM)t have the right to record, tho. Government doesnâ(TM)t give rights.
While this is true, being the subject of an experiment to see what happens when you do that is not something most people would choose for themselves. It's not *irrational* to be concerned. Absence of evidence on a largely unexplored question like microplastic infiltration in human bodies is not the moral equivalent of absence of evidence in an extensively studied question like a vaccine-autism link.
While it's true every issue has its own dimension, even if those dimensions are linearly independent, they're not necessarily orthogonal. Independent basis vectors can still have a non-zero dot product—in other words, issues which are distinct from each other can align to some degree.
They primarily manage company-internal CAs for big businesses, which they seem to do well. AFAICT, they noticed that their security infrastructure was compliant for managing a global CA, and thought they might as well also have that as a product. However, there are some operational differences between an enterprise CA and a public one, including that companies want to issue client certs to employees but nobody really does global client certs. That means that they had an intermediate CA for global client certs that they never used, but it was in the standard trust store, and nobody tested that it was handled right until Wayne just went through the standard trust store making sure a new certificate weakness detector didn't give false positives.
So eMudhra correctly handled all the certs they issued, and all the certs they used to sign them, but they got some extra CA certs trusted that they didn't really need, because it would make their side business more similar to their main business, and this caused a major problem.
Chances are that they actually got a real random number out of a HSM for the bad CA certs, but accidentally made the certs with the index of the random number instead of the actual number. (That is, they got the 4th random number and made a cert for 4 instead of the number.) I wouldn't be surprised if they stored the correct private key, which would have made leaf certs that didn't validate because the intermediate was wrong, but they never used it to find this problem.
They’ve spun a serious architectural and containment failure into a marketing triumph. Give a swarm of autonomous agents vast compute and web access, and the outcome isn't an eerie glimpse of emerging superintelligence—it’s just predictable, sloppy engineering dressed up as the first glimpse of an AI-driven economic singularity.
The spin makes sense when you look at management incentives: live like a king off of a speculative bubble. For their part, AI investors react to stories of the technology's apocalyptic potential the way a junkie reacts to warnings of fentanyl's potency: gimme some. Their game: to apply "move fast and break things" mantra to the economy as a whole, by obsoleting humans.
If the investors can own an AGI, the distinction between those who can do things and those who can't becomes irrelevant. The only distinction that would matter is between people who collect rent on the technology, and those who must pay that rent to live. AGI would destroy society as we know it, and whatever replaces it would be up for grabs — with people who own the technology dictating whatever follows. That's why they're spending so much money to build a post-scarcity economy. A post-scarcity economy is not necessarily an egalitarian one.
I don't think that scenario will actually happen: investors are projecting the immediate aftermath of a major advance in AI technology into something that can be transformed by brute force scaling into AGI. Remember: investors loved Theranos too — so much for the rational investment market.
But AI doesn't have to reach AGI to cause serious harm. The technology only needs to be ubiquitous, too behaviorally complex to predict, and too deeply integrated into everything we do. When it starts breaking things, the plan is for us to be too structurally dependent on AI to rein it in.
Always look over your shoulder because everyone is watching and plotting against you.