Machine learning is, by its very nature, unreliable and ethically ignorant. It is the apotheosis of increasing laziness and impatience in software development circles, where people have literally thrown their hands up, tossed all the data at the machine, and said they donâ(TM)t care HOW a problem gets solved, as long as the machine can do SOMETHING to the data to make it pass a very limited set of tests. As long as it can step over that extremely low bar, it will be considered good enough. When edge cases are inevitably found, theyâ(TM)ll just get added to the test case set, the model will be retrained, and entirely new ways for the ML algorithm to fuck up will be created, continuing a never-ending game of whack-a-mole that can literally never perfect itself over any decently sized problem space.
In practice, this means that ML should never be the determining factor when the potential consequences of an incorrect decision are high, because incorrect decisions are GUARANTEED, and its decision making process is almost completely opaque. It shouldnâ(TM)t drive a car or fly a plane, control a military device, perform surgery, design potentially dangerous chemicals or viruses, or try to teach people important things. It shouldnâ(TM)t process crime scene evidence, decide court cases, or filter which resumes are considered for a job. Itâ(TM)s perfectly fine for low-stakes applications where no one gets hurt when it fucks up, but we all know it will get used for everything else anyway.
Imagine what happens once you have dozens of layers of ML bullshit making bad decisions with life-altering or life-ending consequences. Depending on how far we allow the rabbit hole to go down, that could very well lead to an apocalyptic result, and it could come from almost any angle. A auto-deployed water purification chemical with unintended side effects. Incorrect treatment for a new pandemic pathogen. Autonomous military devices going rogue. All things are possible with the institutionalization of artificial stupidity by people who donâ(TM)t understand its limitations.
Of course we should start regulating the shit out of this right now. And, of course, we will obviously NOT do that until the first hugely damaging ML fuckup happens.