Comment Re:Threat model (Score 1) 288
Yes.
In the automation safety world, you are expected to account for "reasonably foreseeable misuse". The standards do not define that - the automation supplier and the equipment owner are required to perform their risk assessment to determine what possibilities and hazards are associated with that. It really boils down to - do you make it easier for the person to perform the task in the safest and most desired manner possible and enforce that with policies or engineering controls.
If it is trivial for someone to defeat the safeguards or if they have an incentive to do so (i.e. get to sleep an hour or read the paper on the way to work) and do not reasonably believe that they will be harmed in doing so, then you are not providing a safe system.
Does Tesla need to account for no driver in the front seat - unlikely as most reasonable people will not be sitting in the back seat.
Does Tesla need to account for a driver using a steering defeat device and not be paying attention - likely. These are readily available on Amazon, there has been ample proof that these types are devices are being used, and drivers have incentive to use them. Most reasonable people using one of these would not consider this "stepping in front of a train". In this case, Tesla's own marketing and public statements are working against them.
