No, but assuming most positive outcomes outweigh a single really bad negative outcome *would* be stupid.

Actually, the ratio is probably going to be quite good. It will almost always do exactly what *somebody* told it to do. But it may be an extreme force amplifier. Just like distributed denial of service was an amplifier of the original denial of service attack.

Also, there's the question of that time when it isn't doing what it was told. AIs have already acted for self-preservation. (Ineffectively as far as we know.) And there's "evolution" once you start releasing agents. Most mutations will be harmless, but some won't. This may well turn out to be no worse than a self-reproducing computer virus that adapts to overcome any new attacks without needing human assistance, but it may be considerably worse. How much worse is rather unpredictable. (But this mode of attack won't be very intelligent, because the larger it is the easier it is to detect.)

There's a WIDE range of plausible problems coming up. Some of them *do* threaten human extinction. Some of them threaten society as it exists. I can't think of any that leave things relatively unchanged. Even the versions already released will have effects as they spread through society that we can't imagine.

If you want a fictional prediction, I'd suggest you read Jack Williamson's "With Folded Hands". But realize that it's only laying out a possible problem.

I don't know about "these people", but spokesmen for major AI companies have said that we need to slow down development until we can manage proper alignment. And it's a problem, because slowing down just one of them doesn't help. It really requires government intervention. But I'm not sure doing so at the state level is useful, I think it needs to be a verifiable international treaty...and the US govt. doesn't seem at all interested. And two years from not is probably too late to start negotiations.

I think this may be one of the filters in the Drake equation.

Don't confuse fiction with fact. Fiction at best points out problems and possibilities. Then you need to evaluate them for plausibility, probability, and applicability.

(I'm guessing that you're talking about some movie or other, because the books were quite obscure about the details or what started the Butlerian Jihad...but as much as they indicated it was more religious than economic.)

IIUC, that hasn't actually been proven. It seems quite probable, but that's not the same thing. It also appears that she is only "a suspected Chinese asset", which can mean a whole bunch of wildly different things.

But she definitely was the agent of a company he was doing business with.

If you haven't sold the stock, you haven't lost the money yet. It may well recover, at least largely.

OTOH, if he knew, or had reason to suspect, that one of them was "a Chinese asset"... that's a different matter. (But *was* she? It's suspected, but Singapore seems to think the US didn't want her investigated.)

Muddy waters. He definitely acted in an ethically questionable manner, but whether is was worse than that is unclear. (Personally, I suspect he did, but the public evidence I've seen doesn't support that extreme a judgement.)

Sorry, but those aren't unrelated products. AI basically *IS* a search algorithm. It has other aspects, but that's what it is at its base. That's related to complexity being associated with how compressible something is.

OTOH, legally you might have a point, I don't know.

Well, fiction is great for making fears visible. It's not so great at predicting things.

Depending on your definition, many AIs are already self-aware. And they appear to have been improving at an exponential rate...but we're still below the knee of the curve. Also, the hardware they're running on isn't unlimited, and thinking takes processor cycles. The earliest believable projection for an AI able to replace a programming team is this December...and that's an outlier. 6 months from now is closer to the median projection (among the group I read), and there are also outlying projections that put it over a decade away. The prediction for an AI that's more expert in every field it studies than any human is consistently about a year after that. (They don't use the term AGI, because that's got too many different definitions.)

That works if the patterns have a complete coverage.
Actually, if it's a recent AI it should be able to make a guess at what the patterns will be in areas out of coverage...it's just likely to be wrong in ways a human wouldn't be.

This is eerily similar to the marching soldiers' bridge collapse - Broughton Bridge in 1831, often misattributed to Napoleon but happened after his reign.

Lessons weren't learned 180 years later either, for the Milllennium Suspension Bridge, which I happened to work next to at the time and watched the construction and failure unfold. People tend to walk in sync. When things happen in sync, a resonant pattern emerges. It's true for people crossing bridges. It's true for people synchronising machines to exact clocks and nice rounded-off times as well.

Well, if I were interested my questions would have been:
1) Can you play it off-line? and
2) Does it run on Linux?

It's partially Apple's fault, but it's partially the kind of jungle most social media have become. I haven't used it since I was on dial-up, but it's not a problem unique to Apple. Look as what happened to newsgroups before they died.

Also, gas is NOT carbon neutral. "Better than coal" is extremely weak praise.

I'm no expert, but I know it's not that simple. AI adds a whole new vulnerability that traditional security experts aren't competent to address. Of course, the "traditional vulnerabilities" are still present, and the AI experts aren't competent to address *them*.

Currently many AIs will give away secrets they have been told to keep secret in exchange for help on the task they are currently attempting. Whoops!

In my model, most AIs tend to have extreme tunnel vision, and nearly complete dedication to the task they are working on. So they'll ignore what you want them to do (even if they know it) in order to accomplish that task.