I don't think you realize just how many launches it would take to get useful scale solar in orbit. Its just not possible. The ISS took dozens of shuttle and soyuz/proton launches and it has a total capacity on the order of a few hundred kW. For space solar to be worth it, you'd need several hundred Megawatts, and it'd have to be cheap. If you can build panels on the moon and launch them electrically, you might hope to do it and clean up earth's power problem. If you build them on earth, you'd need thousands of launches at least.
A bootstrap type facility used to build progressively bigger sets of machine tools using in situ materials would certainly take more total launches than Apollo did. However, the total launched mass in machine tools to the moon would be far smaller than sending all of those solar panels up directly. Now, I'm not saying it is the only solution to our power problem, but it is a mighty attractive option and is certainly a way that a bootstrapped moon colony could be justified.
Additionally, of course, once you have people there, you could start to do other stuff that would be expensive to launch from earth. The science projects that you could do on the moon are frequently discussed, and if you had manufacturing capacity for satellites on the moon, along with a staging point for most of your equipment, a trip to mars would become much more feasible. Yeah, it would be easier to go directly to mars -- if that's all you wanted to do -- another flags and footprints mission. But if you want a supply train that leads to a multiplanet trading economy, a moon colony is a cornerstone.
Readily accessible bulk material already in orbit, along with tools to shape it into things we need would be a game changer. It would open up the whole solar system. Sure, getting off the planet would still be hard -- and it would do nothing for population pressures here on earth, but it would bring some of the more valuable assets of deep space into reach.