> What stops a 15 year old script kiddie that hacks a battalion of these things and sends them on a rampage?
I wanted to make a joke, because this is a great setup to one, but it frankly won't be funny in a few years.
The parts of the world that seem automated, are already exploited, no exceptions. The biggest and glaring sign of this is SWATting. Hackers identify that the government is responding with overwhelming terror force to phone calls. Hackers "hack" this system by simply spoofing a dangerous scenario. Bad things ensue.
This is a fully automated system- but every element of the automation is humans. From getting the call to parsing it to responding with force to deploying officers in full military gear to apartments and houses, everyone is doing their job. And it's broken. What's the piece of automation needed to exploit this, what's the real hack? It's the telephone or other comms device. This narrows the bandwidth enough that the barrier to entry drops from "trained actor" to "rando asshole". It also removes the needed "now escape" part of the plan. Just that ONE piece of tech- the phone- is enough to break the system, and because it wasn't happening when the police weren't geared for unseating nests of aliens or whatever the fuck all that shit is for, they built the house of cards on bad tech.
And this will keep happening. Every piece of automation, every thing that removes a chain of humans from the equation in the name of efficiency, dramatically ups these risks.