Comment Re:Unbelievable what people will publish (Score 2) 11
They’ve spun a serious architectural and containment failure into a marketing triumph. Give a swarm of autonomous agents vast compute and web access, and the outcome isn't an eerie glimpse of emerging superintelligence—it’s just predictable, sloppy engineering dressed up as the first glimpse of an AI-driven economic singularity.
The spin makes sense when you look at management incentives: live like a king off of a speculative bubble. For their part, AI investors react to stories of the technology's apocalyptic potential the way a junkie reacts to warnings of fentanyl's potency: gimme some. Their game: to apply "move fast and break things" mantra to the economy as a whole, by obsoleting humans.
If the investors can own an AGI, the distinction between those who can do things and those who can't becomes irrelevant. The only distinction that would matter is between people who collect rent on the technology, and those who must pay that rent to live. AGI would destroy society as we know it, and whatever replaces it would be up for grabs — with people who own the technology dictating whatever follows. That's why they're spending so much money to build a post-scarcity economy. A post-scarcity economy is not necessarily an egalitarian one.
I don't think that scenario will actually happen: investors are projecting the immediate aftermath of a major advance in AI technology into something that can be transformed by brute force scaling into AGI. Remember: investors loved Theranos too — so much for the rational investment market.
But AI doesn't have to reach AGI to cause serious harm. The technology only needs to be ubiquitous, too behaviorally complex to predict, and too deeply integrated into everything we do. When it starts breaking things, the plan is for us to be too structurally dependent on AI to rein it in.