Could we do a better job if we could cache intelligently and do p2p
That would help, but ultimately the question that's being asked is, "who will pay for this bandwidth?"
The answer of course is the ISP users are already paying for it. The ISPs just don't have the bandwidth they've claimed to have sold their clients. And I would call this bordering on fraud.
Say an ISP has 1,000 clients, and sells them all 1.5Mbps DSL connections. But if 500 people go and try to stream video at the same time, and the infrastructure can't handle it, the ISP has sold you a product it actually didn't have. If a store tried to oversell the latest Harry Potter book, and asked customers to "share" the books because they didn't actually have enough to go around, there'd be lawsuits flying.
Now I know in reality having 1,000 x 1.5Mbps infrastructure probably would never happen, and there would be some bandwidth sharing, that's the point of packet switched networks. But scaling up to meet the needs of customers for what the ISP claimed the customer was buy, is ultimately the ISPs responsibility. Net neutrality should not be used as an excuse to not provide the minimum infrastructure needed for the service ISPs are collecting money for.