the supposed mineral riches are mostly high priced specialty materials and not the boring metals like iron and aluminum
Huh? The moon has gigatons of common metals and many of the areas are especially aluminum rich (and titanium to a lesser degree).
The idea is to refine those and launch them to a useful orbit for much less cost than trying to throw them out of Earth's gravity well. Whether that's Earth geosync or a LaGrange point, or something else, the challenges are along the lines of having enough utility stock (for doing the Bayer Process, e.g.) - there's plenty of sunlight to make the needed electricity.
The moon process will likely be different than the Earth process, in the end. For instance, all that oxygen that needs to be liberated from AL(2)0(3) can be stored for useful purposes. Even if at first you're storing the separated aluminum for future use, it might still be the best way to make oxygen for those fussy humans to breathe, if you look at the long-term cost/benefit. I seem to recall the University of AZ had a whole system worked out in the 90's and demonstrated using near-analogues to moon rock in big vacuum chamber. Undoubtedly a 2015 plan involves several generations of 3D printers that can eventually be used to print more moon base.
But, yeah, having a H3-H2 reactor would make it much more efficient, but solar or RTG will have to do in the short-term. It's quite the bootstrapping problem. Once you have a major factory complex up and running for safe habitation and flexible manufacturing, you're ready to set the AI's out building more structures night and day, and that's when you get polynomial expansion.
Barring government interference, I fully expect to see moon habitation visible from Earth in my lifetime (the NIMBYs will complain that it should be on the dark side).