But the merchants eat it. If you use a debit card for a $100 purchase they eat 2-3%. If you use a credit card for the same purchase, they might eat 3-4% and if you use a "rewards" credit card, the merchant must eat 4-6% of the transaction. Maybe their retail markup was only %12 on that $100 item, and now instead of making $12, they only made $6 because you got "rewards".
The big box stores can negotiate lower rates, so use your rewards card there, or at a merchant you don't care about, but if you can, pay with cash (or at most, debit) at the mom & pop stores.