I generally have only the phone, no cards or cash on me. Since PayWave (a contact-less payment PoS) is supported by next to all terminals I very very rarely run into issues.
From what I know plenty of people do the same -- phone only is enough for pretty much everything.