I have taken around 180 Uber rides, in London, San Francisco, New York, Paris, Moscow, Munich and Boston. In London, every Uber driver appears to treat it as a full time job. I ask them and they tell me the long hours they work, often complaining if I let the conversation go that way, how Uber has reduced their remuneration to where they claim it is almost not worth it anymore. (note, they always say "almost":)
Addisson Lee had to re-scramble their business model, to account for all of their drivers leaving for Uber. They had bought up and consolidated mostly all the "mini-cab" i.e. limo driver companies. These were generally full time roles.
In Moscow, I got the same driver for every single trip, for the entire week. He was in a suit and tie with sunglasses... I think I must have been the only Uber customer in town so he just hung around my hotel and workplace.
In New York, one of my latest drivers talked of how he quit his trucking job for Uber - remarking how he is treated better by Uber customers than by his contacts in the shipping industry.
The only outlier I can think of that supports your observation is Boston, where I was picked up, at 4AM by a guy who claims his primary job is writing. I had to wonder, who gets out at 4AM to do side work? Well, for a creative person you never know.
None of my or your observations "change" the concept, but very very few of my observations support the concept that your observations are supporting. So, at a minimum I would have to say those Uber drivers you know, and I, are not driving and riding via Uber in the same localities, at the same time.