I don't know about OP, but most of the utilities I get service from do not take credit cards. Instead, you can either mail them a check/money order, pay via EFT online from your checking account, or pay in person at a place that does Western Union. If I recall, the only one that'll take a credit card is Comcast... though my natural gas place might as well. I know the electric company and the water department don't accept credit cards at all. For excise tax and at the DMV, I have to pay with cash, check, or money order (they accept credit cards for licenses at the DMV, but not for titles, registrations, or sales tax). My landlord doesn't take credit cards either - he requires a check or cash.
For online purchases, one can rely on picking up gift cards at a local brick and mortar or go without. Personally, I have a debit card attached to a checking account for such things with a limited amount held inside, but it's perfectly reasonable to go without one. Indeed, I don't have any credit cards, just that one debit card. Yes, it can be a pain getting cash out for purchases, but I find I spend less if I'm using actual money.