The downside is channels like SciFi will no longer exist. Of course, neither will the Golf Channel, or EWTN and that might be a good thing.
The far, far bigger downside will be with consumers paying directly for programming they will be faced with a decision every month (at least!) of "is this channel worth it?" Today, these costs are hidden whereas with this model it will be clear and obvious - and most channels will fail the "worth it" test. End result? There will be far, far fewer channels if any are left at all. HBO will remain because it is already passing the "worth it" test every month for every viewer that gets it. But virtually nothing else is going to make it.
Sure, you might be willing to subsidize the SciFi channel, but just about everyone else will not. Same thing goes for the Golf Channel - where there might be hundreds of people that would easily plunk down $20 a month to receive it, that isn't enough people by far. So it dies. ESPN? They might survive, but it will be a near thing.
A far bigger question is sports in general. Today what pays for the entire college sports program, everything from golf to swimming to polo? Football television revenue. Wipe out the advertising there and there is no more television revenue or certainly not anywhere near what it is today. So most colleges will fold up their sports programs, all of them, simply for lack of funds. Was someone thinking of building a billion-dollar sports palace for a pro sports team? Ha. Without television revenue, which all comes from ads, there will be no more huge sports palaces. There will not be million-dollar contracts for NFL players either. So ESPN might survive, but there will be virtually nothing to show on it. Bye-bye ESPN.
No, without advertising revenue you are looking at a signficant remaking of American society in ways you cannot imagine. I can't either - it is too sweeping a change with television ad revenue sponsoring or supporting so many different aspects. Were this to change - and I really doubt it will - the changes will affect every single person in the country, probably in a negative manner.