First, building such a fab, if you know what you are doing and have respective experience, is a 10 year or longer thing. Second, you need people with very specific and very rare skills to build and then operate it. I would be surprised if Musk can even hire those. He would have to hire them away from TMCS or Samsung and these people are likely not even interested in moving to the US, let along working for Musk. Third, you need the equipment. This is exceptionally expensive and there are long waiting times. And then, you need to get things like wavers and these do not grow on trees and the global supply is rather limited and long-term allocated.
I guess this is of the quality of the tech-claims made by DOGE, i.e. completely disconnected nonsense by somebody who has no insight into how technology works.