The trend towards cashless troubles me because it takes a transaction between two parties and adds a third party - the bank. As we've seen in the past few decades, they're not exactly trustworthy or competent.
Retailers are already charged for every transaction made by card, which is fine for Walmart but not expensive for small traders, further biasing the market towards already huge entities. It adds a monetary cost as well as a knowledge barrier to small traders, flea market stall holders and self employed tradespeople.
Furthermore, every purchase becomes a public act with a permanent record. If there is information on you, you can bet your ass someone will pay for it - social media has made it a business model to collect it. The panopticon wasn't built around government, but advertising. This is not a hypothetical future but instead our reality, today.
Do you really wish to give up more privacy? As America lurches and burps and farts its way towards authoritarianism with King Baby at the helm, don't you think the Intelligence Services will be delighted to pour through your purchase records for signs of being an Enemy of the People?
Simply, it's another centralising trend at a time when people should be keeping well away from it. In a way, its a sign of how centralised we have become - the lie of the tech utopians was that we would rid ourselves of middle men. Instead, the middle men have become ubiquitous.