I called it cheating because they violated both one of the prime rules of AI: train on a data set that is more or less representative of the data set you will test with, and one of the prime rules of statistics: do not apply a priori statistical analysis when you iterate with feedback based on the thing you estimated. Their test images are intentionally much different from the training images, which is one of the first things an undergraduate course on AI will talk about. They also use what are essentially a priori estimates after they repeatedly tweak the inputs to push those estimates to extremes, which is identified as taboo in decent undergraduate courses on statistics. Both of those are intentional violations of good practices that make the results look worse for the neural networks.

I can't tell from their paper what they mean by "99% confidence". Unless the DNN has max-pooling layers very near the output, none or many of the output units might have high activation levels for a given input. (It sounds like they had classes with low typical activation levels, and did not try to evolve fooling images for those classes.) If that happens -- say, "wheel" gets a score of 0.99, "lizard" gets 0.90, "dog" gets 0.80, and everything else is near zero -- then it is inappropriate to say that the network decided it was a wheel with 99% certainty. You would usually say that the network recognized the image as a wheel, but note it as an ambiguous result.

Why was my characterization of their approach "hardly fair"? Someone -- either the researchers or their press people -- decided to hype it as finding a general failing in DNNs (or "AI" as a whole). The failure mode is known, and their particular failure modes are tailored to one particular network (rather than even just one training set). I think the "hardly fair" part is the original hyperbole, and my response is perfectly appropriate to that. The research is not at all what it is sold as.

Don't multi-class identification networks typically have independent output ANNs, so that several can have high scores? I assumed, perhaps incorrectly, that the 99+% measures they cited were cases where only one output class had a high score, and the rest were low. If they were effectively using single-class identifiers, either in fact or by considering only the maximum score in a multi-class identifier, that makes their findings even less notable.

The neural networks in question were absolutely trained with supervision. Unsupervised learning is a quite different thing.

The researchers also basically cheated by "training" their distractor images on a fixed neural network. People have known for decades that a fixed/known neural network is easy to fool; what varies is exactly how you can fool it. The only novel finding here is their method for finding images that fool DNNs in practice -- but the chances are overwhelmingly high that a different DNN, trained on the same training set, would not make the same mistake (and perhaps not make any mistake, by assigning a low probability for all classes). It is a useful reminder for some security analyses, but not a useful indictment of AI or DNNs as a whole.

The people choosing the training sets are not morons at all. This "research" is almost exactly analogous to finding that this year's SAT can be passed by feeding it a fixed pattern of A, C, D, A, B, and so forth -- and then declaring that this means standardized testing is easy to fake out. They are exploiting the particular structure of a particular instance of a DNN. It is not surprising that they can find odd images that make a DNN answer "yes" when the only question it knows how to answer is "is this a rotary phone dial?"

The last time I frequently used a US bus system -- about 15 years ago, in Pittsburgh -- they used a zone system, with the fare based on your origin and destination zones, and most bus routes crossing at least one zone boundary. The last time I used a public bus -- about 5 years ago, in Japan (Yamanouchi, Nagano Prefecture) -- riders took a numbered ticket when they boarded, and their exact fare to the next stop was displayed on an LCD panel. I would guess that US cities avoid exact metering in order to subsidize their passengers who live far away from their workplace; these are the most frequent riders, and tend to be lower-income.

Yes, and you have effectively described Uber's concept for driverless transport. It won't look anything like today's buses, and the only real similarity is that strangers share road vehicles.

If you go from ten single-occupancy cars to a ten-passenger bus, you've eliminated 90% of the vehicles at the (relatively low) cost of adding one more driver. Eliminating the bus driver gets you from eleven people in the bus to ten, which is probably not as important as other efficiency improvements. Also, buses are awful unless you have quite high population density -- lots of areas don't have enough prospective trip endpoints to justify mass transit.

And in your world, how do you deal with unicorn overpopulation?

When you have one district that breaks down mostly into two significantly different constituencies -- whether they are based on race, class, rural versus suburban, or whatever else -- the way politicians react in reality is that they focus their appeal on one of the constituencies, try to increase turnout for that constituency, and discourage turnout for the other one. It's just gerrymandering of a different kind.