1. Peak demand. In car-culture areas there's a peak demand. *Someone* has to own the rush hour fleet. But no business is going to want to invest in a fleet that has 21 hours of downtime during non-peak loads.
This isn't the comparison which makes the most sense. The question is, "Can fleet ownership result in greater value for consumers significant enough to make a profit?". Most of the trips that residential vehicles make are commuter trips, but the vehicles making these trips are very often compromise vehicles, capable of doing a larger variety of tasks. If the surge is commuters, the commuter fleet can mostly be made of much smaller, task-oriented vehicles, reducing fleet costs. Many of these vehicles will be able to service multiple users sequentially, since the starting and ending times for work surge over a two-three hour period, increasing utility rates. A non-insignificant portion of this surge fleet will still be in use by people throughout the day. The reduced nuisance of parking could increase useage for things like lunch trips. The potential for there to be economies of scale is here and benefits from fleet ownership, so this cannot be simply dismissed.
The logic of where the savings exist can be thought of in terms of, "Are there savings that self-driving vehicles could accomplish with smaller modifications to the system?". If I had a way of making my self-driving vehicle available to people to pay me to use during the day, that would result in an overall increase in economic efficiency. If it was more convenient for people to rent specialty vehicles, utilization would increase and rental costs would decrease such that my every-day car could be optimized for commuting tasks and thus less expensive, more than offsetting the occasional rental costs of specialty rentals, increasing economic efficiency. These efficiencies are easier to realize with fleet ownership and thus there is room for cost savings for consumers and profits for a fleet owner.
2. Consumers want reliability and 100% availability. Consider Uber and Lyft that promise this, except during surge pricing periods. People hate this. It's economically correct in the case of Uber and Lyft, and an obvious idea, but surge pricing during rush hour isn't going to work. People will still own their own cars.
The problem with Uber, Lyft, car sharing, and taxies is that pockets of high useage have a lasting decrease in service availability in those areas. With self-driving vehicles, there is a very small cost to shifting resources to fill in empty pockets of the map. Yes, this becomes problematic the more rural one gets, but in urban and suburban environments, populations densities are high enough that with high levels of utilization, it could easily be economical to make it a very rare occurrence to not have a vehicle able to be at any address within a small number of minutes after being summoned.
3. Personalization and customization. Hey, I like my cars stock, but I still have my stuff in the center console, my presets on the stereo (yes, 760 am in the morning, I'm a dying breed), and my iPhone paired to Sync. A different car every day isn't going to cut it. And think about comfort, especially on a commute. If it's hit or miss as far as comfort, people are willing to pay for 100% access to a Fusion versus an Elantra (or choose an Elantra versus a smaller B-sized car).
Storing crap in one's car is probably the best argument for what will bother people about changing to this system, as having to lug crap out to the car while you are on the clock is a stress people will likely often buckle under; however, this seems like a problem which could be solved by some interesting lateral thinking. Radio presets are an easily solvable problem and luxury models are a method of price discrimination which would likely very quickly enter the marketplace, although it would be strains on my previous arguments. Much of the other elements of vehicle size and status are an extension of driving and how it feels to do so and they diminish greatly when one is a passenger. A self-driving sports car would lose a lot of its mystique. Another 'comfort' issue would be managing vehicle cleanliness between users, which might be a difficult problem as well. Of course, airlines have shown that people will tolerate an awful lot of discomfort in order to save a buck, so this is all a big open question.
4. Toy haulers. You're not going to call Uber or Lyft to tow your trailer to a state park or tow your boat to a launch. And this isn't 99%'er speaking, this is blue collar worker in my part of the country.
I really don't understand the objection here. In fact, I think owning trailer toys would be easier in a self-driving world as it can be expensive to own, operate, and maintain a huge vehicle for these purposes when you only use the towing feature a few times per year and many are not comfortable hauling a trailer at all. A self-driving towing vehicle available on your intermittent schedule and which would then be used by other people on the next weekend seems like an easy win.
All of this is going to depend on balancing a lot of variables and developing logistics algorithms and a hell of a lot of accounting, but this is not incredibly difficult stuff, just a bit tricky to get right. It is not immediately obvious that fleet ownership of residential vehicles is uneconomical and plenty of reasons for smart people to sit down and crunch numbers with the potential for them to make money and for consumers to save money under the new system.