Strictly speaking it is not the energy "potential" that is the the problem with solar and wind power. Plenty of potential, but the land area requirements to practically collect the energy are very big so there are some very real environmental downsides.
But the real problem, as Bill Gates has properly identified is the "baseload" requirement, so if you can only rely on solar and wind 50% to 60% of the time then you need to be able to store the energy efficiently and you need a "smart grid" to distribute the unpredictable energy output. Put it all together and you really can't get to a mostly solar and wind powered grid without huge drops in efficiency through conversion and storage which further increase the deployment costs and land use requirements. Such a system would cost so much that it would have a seriously negative economic impact. Right now solar and wind can get a free ride (efficiency boost) as long as most of the grid is powered via other sources that can provide a consistent baseload
So, I'd say that the physics can be made to work at great cost, but agree that the economics can't work with current or currently foreseeable technologies.
That said, something around 20% solar, wind on the grid would be a good policy goal. But the 80% goal needs to come from nuclear and natural gas and whatever hydro-electric already exists in the US. With the other goals to reduce coal and oil as much as possible and to increase efficiency in energy usage wherever possible.