I think you've misinterpreted this notion of a more robotic labor force as some sort of idealism, altruism, prosperity, instead of simple economics. He seems to want to say that robot labor will be cheaper than human labor; and here, your thesis is correct. Humans very often seem to value short term gain over long term gain, and more importantly personal gain over utilitarian gain. If a manufacturing company recognizes that it can make a bundle of profit off of laying off 90% of its human workforce in order to 'employ' machines, it very well might ignore the fact that this will put a lot of human beings out of work. This already occurs in outsourcing human labor in one country to cheaper human labor in another country.
This is not necessarily a Utopian idea to say that the undeniable rise in machine intelligence could possibly result in an incurable rise in human unemployment. There won't necessarily always be something for uneducated, unskilled workers to do in order to scrape by and make a living. We see more and more that higher qualifications are typically required for not so difficult work. College education instead of high school education is becoming a norm these days; how long until one must receive a master's degree in order to be considered economically competitive? This isn't to say that I know for sure that machine intelligence will entirely make obsolete human labor, but it seems rather plausible that if there is a cheaper (more consistent, safer) alternative to human labor, then companies that are admittedly not altruistic entities will not hesitate to make changes that will negatively affect the humans that depend on them for employment.
The question then would become, yes, what will the world look like at that point? What if we really do see consistent 25% unemployment? How do we support those people who could not help being replaced? Will we be expected to and will we even desire to rise to that new challenge? Will it be necessary, or will abundance of resources support that burden?
I'm not sure its moronic to ask these questions in a serious and critical way. In fact, I think there is every reason to. Worst case scenario, we're wrong. Best case scenario, we've had thoughtful discussion on a particularly meaningful and potent topic in development of human economics and the capitalistic concept of earning one's keep.
What sin has not been committed in the name of efficiency?