The "if you don't like it, switch to another provider" argument is ridiculous if you look at the reality of the situation. There is no true commodity market for internet access, major markets have 2, maybe 3 options, smaller markets only have 1. Of those 2, they will often rely on different technology, so only one may actually meet your needs. Furthermore the costs to switch can be very large, especially for large companies. Furthermore, it will be mostly invisible to the average end user. The costs are going to be born by the websites that want to get their traffic to the customer (using the connection the customer has paid for). If you are Google, having your search results artifically delayed to be slower then Bing results will cost you Money. If you are trying to run a voip service, and the QoS applied by the ISP artificially slows your packets while giving another service priority, your service will be spotty and drop lots of calls, while the other service will work great, even though you both are using the exact same infrastructure.
Allowing ISP's to treat packets differently is giving them a license for legal extortion. They can abuse the fact that to the end-user slowness caused by their ISP and slowness caused by the site itself is indistinguishable to extort money. Furthermore, they can give sites they have a financial interest in priority, without spending a dime on increasing capacity. Furthermore, it creates a barrier to entry to new players that do not have the funds to pay off the ISP's to carry their traffic.
Neutrality to traffic is a fundamental aspect of the internet that is part of why it has been so successful. Allowing protocols, services, and sites, to live and die on their merits without artificial limitation is what has led the boom of internet development. Imagine if in those early days of servers in cases made of LEGO, Google had to negotiate an agreement with every ISP to carry their traffic to their users before they could offer services. And at any time, someone with a bigger warchest could have offered more money to keep them off.
Congress and the FCC have created these monsters, constantly pouring government funds, preferential treatment, monopoly agreements, etc. into them to keep any real competition from occurring. If they don't have the right to place limits on their abuses of the oligopoly position they gave them, who does?