You pay for channels you don't want so you can watch the few channels you do want.
The communications director at a local cable service provider once told me the problem with ESPN: it's the most expensive channel in their entire cable lineup. They would love to separate it out and treat it a-la-carte like HBO, but their agreements don't allow for it. Either everyone gets it, or no one does. And he said everyone gets it, because whenever the feed goes out for that channel, their switchboards light up like a Christmas tree. (He also mentioned that the other channel that customers most hate to lose is Lifetime, though that's not nearly as expensive.)
It's extortion, plain and simple. Though ESPN is only partly to blame...the NFL, NBA, and NCAA are also guilty for making game broadcasting rights so pricy.