We can't build a sustainable habitat in Antarctica or in the middle of a desert
Can't or won't? I would have thought that it would be possible to create a habitat in either that would require nothing incoming. Not easy, but not impossible. It would just cost a fuck of a lot to build and would probably require a very large area (either above or below ground) to support just a few people.
Both locations, however, have the immeasurable benefit of being on a planet with a breathable atmosphere and getting the huge amount of resources required to set up this habitat to the location would not involve climbing a gravity well.
It's definitely a "won't" for anywhere on Earth and likely a "can't" for anywhere else in the solar system.