It's there, but we have to put an absurd amount of relatively rare resources into photovoltaic cells to make *use* of that energy.
You have peculiar notions about what solar cells are made of.
By weight, a finished monocrystalline solar panel is primarily silicon, 2nd most abundant element in Earth's crust, followed by aluminum, 3rd most abundant element in Earth's crust. The n-dopant that is half of what makes the silicon a semiconductor is phosphorous, 11th most abundant element in Earth's crust. The p-dopant that is the other half of what makes the silicon a semiconductor is boron, 41st most abundant element in Earth's crust, considerably more common than beryllium or tin, somewhat less common than lead. There is a tiny bit of silver on the upper surface, as a conductor, 65th most common element in Earth's crust, and the only remotely rare element in the list. Fortunately it is used in proportion to its rarity, so it's not a limiting factor to speak of.
In short, solar panel production is not in any way material limited. The Earth is made of the necessary elements, almost in the proportions they are used. There is enough of the necessary materials available to plate the entire surface of the Earth in solar cells, oceans included.
Fortunately no such excess is necessary. A population of 20 billion could enjoy the average per-person energy availability of an American without covering an appreciable fraction of the Earth's surface in solar panels, and you couldn't tell the difference in terms of the material used.