So the argument is someone steals my password, steals my money, gives it to a money mule... then I get my money back from the bank, and someone that doesn't cost me in the end? Even disregarding the fact that those costs are going to get passed on to me somehow... The inconvenience of having to deal with identity theft is not always minor (and there's probably collateral damage here as well).
My biggest beef with banking is that I don't, but should, have the ability to send money with end-to-end authorization, by way public key crypto. If, say, Amazon could verify that I authorized a purchase using my public key, then network security, and banking security, is irrelevant. Bitcoins have offered a very secure example of how this could work, assuming that you have good local security (your private keys are safe).