And the rest of us will be encouraged to remove ourselves from their world, as we will then be worse than useless to them, mere objects taking resources away from them.
Marshall Brain wrote a short science-fiction story called Manna. It's about a dystopia where machines and automation reduce most people to poverty -- and this is exactly what happens in the story. Most people must live outside society in high-density housing called "terrafoam". It's a great read, and it's online at http://marshallbrain.com/manna1.htm .