For one thing, you're only counting the middle steps.
First, you have to get your wallet, get your card from wallet, swipe it, wait, then either sign or enter a PIN, put card away, put wallet away. You're ignoring the setup and teardown steps.
With Apple, its pull out phone, hold it near device, tap, put away phone.
I am not familiar with the Google Wallet version of NFC-compatible phones but I assume it is quite similar (since Apple Pay is mostly a NFC device except how its setup and the arrangement with the banks to secure the transactions)
More importantly though, the regular credit card use leaves that number everywhere you use it, just begging to be stolen which has happened repeatedly of late.
With Apple Pay, the credit card is not stored on the device, instead its a per-device number arranged with your bank when you set it up -- and when it transmits it also transmits a dynamic authentication code and that pair can only be used once. (I don't know what it uses to generate that code but I suspect its something like a software time based token).
They don't get your name, your credit card number is not vulnerable, they don't get any of your personal details. They just get paid.