I think what's unsettling is that AIs are initiating an arms race: AI attack was analyzed (and probably patched) via AI.
Evolutionary pressure: long term (few years?), agents at various places will evolve a kind-of immune system solely to defend against other agents from taking advantage of them... Humans will just be observers by that point, powerless to even understand what's going on.
...now AI is one of the most destructive
The profit motive is making it non-economical to do anything other than create mediocre code with LLMs. Exceptional generation defining leaps in tech (like DOOM and Quake) simply won't happen if everyone is just generating their way to the next paycheck.
Think about it, could AI have come up with Fast inverse square root... I'm sure it could have with enough prompting and someone who knew what they were looking for, feeding it prompts detailed enough that they contained the answer.... but on its own, there's no way AI would've coded it from scratch into a game engine.
I recently had to deal with some xray images on CDs, and discovered to my surprise that I didn't have a reader... I tossed away all CDs like a decade ago, and old hardware just got replaced, all without drives. Nothing came with cd-rom drives in at least a decade, no? So genuine question, how many folks still have a cd-rom drive to be able to read those CDs?
(I ended up buying a cheap drive just to read those xray images... but haven't used it for anything since).
So... everyone talks about vibe coding these days... yet nobody within the administration comes along and says "oh, instead of paying MS for 500k licenses... how about we just vibe code something similar with an open local model"...
(e.g. do they *really* need the latest frontier model for routine administrative tasks... or would a 7gig local model do just as well?).
Yep! With I had points for this... but not just EU... why would there be a need to have chatbots in WhatsApp at all? We already got other ways to chat with bots... no need to push them into primarily person-to-person medium.
What's they're saying is that they're generating more and more code with less and less human eyes to look at it... recipe for success!
but but but... this time it is different!
Same reason why someone would need a toothbrush with Bluetooth connection
What kind of university would not have a physics department?
The kind that has "Stamp-Collecting" as a major
Then you're welcome to convince China, Iran, etc., to use AI systems built in US... and vice versa. Like it or not, national boundaries are often a pretty good reason to "do things differently", if for no other reason than "national security". And this is good... if AI designed vaccine kills off the population, then there's a pretty good chance that the species will survive anyway because other countries "did it differently".
Small nations often fall in line with whatever the regional superpower does...
...how is that a solution? Do you think Russia, China, Iran, or you name a hundred other countries, are going to follow your suggested limits
Each country can and must do whatever they want... the segmentation of systems can happen at country borders. As far as extinction is concerned, a world where US, Russia, China, Iran, India, etc., each have THEIR OWN (home-built-from-scratch) system is better than one-system-for-everyone.
Each country should develop their own vaccines, medicines, AI systems, etc. To ensure species survival in case the improbable happens.
print culture
Oh, really??? Many burned/hung/drowned "witches" would disagree.
even a 1% chance of catastrophic events like extinction or the destruction of democracies is unacceptable
...it is impossible that the improbable won't happen.
The only solution is to limit individual systems... define a sort of kelly criterion for AI, where a single big failure does not mean extinction. e.g. for military robots, mandate any model/manufacturer/dataset/etc., be limited to say 5% of the entire robot fleet... don't let them share code, or collaborate. We *want* them to have different bugs. That way if there's a glitch/feature/emergence someplace, it's limited.
Same for medicine/treatments synthesized with the help of AI... only let 5% o the population to benefit from any individual "solution". That way if there's an extinction gene-editing virus, only 5% of the population is impacted, etc.
social media and the scrolling feature---tech savvy teens slowly winning the darwin award
...wait until people start spending a few years dead... for tax purposes.
If it has syntax, it isn't user friendly.
