More like, Store A is charging $20 for a loaf of bread, I'll go to store B where I can get it for $5.
Lets at least get the metaphors slightly more accurate.
Store A is charging $20 for a loaf of bread, but provides an awesome atmosphere, chairs, clean eating space, nice employees, free coffee, and massages while you eat your loaf of bread. Store B sells the same bread for $5, but you can't eat your bread there. So you buy your bread from Store B, and then expect Store A to let you stay in Store A to eat your bread.
Companies pay taxes to pay for the externalities that they take advantage of while doing business in a country.