You are sort-of right, but you are looking at the wrong 'tax'. The real benefit of cashless, is that central banks can drive interest rates negative in a deflationary environment.
Since around the 1990s, automation and competition from low labour centers basically destroyed the utility value of the working class. We still give them jobs (which are basically funded by welfare), but most of these are just a sop to our consciences so that we can enjoy our lattes and craft beer without having to stare at slum dwelling children.
The real horror for us, however, is that through the 2000s this same effect has extended well into the middle class. Some of us have quite obviously been directly affected, but these are not a majority yet so nobody cares about them. So why has the middle class not collapsed? The reason is because of the banking sector. What the banking sector has been doing since 2000s is engaging in a giant UBI process for anyone in the middle class who owned a house. Central banks have maintained broadly negative real interest rates (when you've adjusted for inflation) since then, which has ensure that anyone with access to credit (ie middle class home owners) can get connected to a perpetual money spiggot that supplements their falling income from their increasingly unnecessary jobs. This has sustained the middle class in most western countries.
Essentially the problem that occurred is that central banks let a fast bubble develop which blew open the nature of the ponzi scheme and those middle class homeowners freaked out. They then hit the zero lower bound on interest rates, and could not keep the bubble inflated. This is why they are desperate for inflation (just watch Mark Carney do nothing as UK inflation rockets) as this would force real interest rates negative, but demographics, automation and a continuing lack of confidence means that they cannot get the mild 3-4% inflation that would allow them to keep the debt ponzi scheme going.
Because of this, what they would really like to be able to do is drive nominal interest rates deeply negative. If they could do this they can get the debt bubble going again even under deflationary conditions. Just imagine how many people will rush out to buy a house for even more stupid prices if people started getting paid to have mortgages. The middle class economy would take off again as the homeowner UBI comes back on line (well, for anyone lucky enough to have gotten onto the housing bandwagon).
The biggest impediment to negative rates is cash. This is why the swiss national bank does not allow you to store francs in safety deposit boxes, and why many countries have introduced controls on cash (under the guises of preventing terrorism). Central banks want this as a tool for the next crash, and they are slowing setting things up to ensure this is possible.
If you have lots of money, then you'll know how to protect it (buy a house, basically). If you have lots of debt then the central banks have your back. If you have a moderate amount of savings, then you will be screwed. Indeed, watch Mark Carney's recent talk on the post crash recovery and he admits that the only people who have suffered since the crash are those who had savings but not assets, but he justifies letting this happen by saying they in the minority.