Look at what happened (and is still happening) with furniture: the value is moved on the creative side, because robots can build all the parts required, assemble it, and ship it to IKEA. If you really need custom furniture, it's the same. A designer can design it in a CAD software, split it up into separate parts, send the file to a factory that will build the parts and ship it to the people who will assemble it. Done. He can reuse designs and just change the sizes, the colors, the software adjusts what needs to, and bam, it's shipped.
I don't see why this can't be scaled to entire buildings? Yes, once you have the complete CAD of the entire building, and I do mean everything, humans assisted by robots that have the schematics uploaded can do the work of hundreds of men.
I don't think it's about destroying all jobs, it's just about from a team of 100 people, you keep 10 people that operate/supervise a few robots. I don't think anybody will risk having AI building entire buildings, but saving 90% in salary costs? That's achievable, and there are new and more advances assist-type machines used every day in constructions.
It's like a guy with 1 jackhammer can do the work of 20 people working with manual hammers. (I don't know the numbers exactly, but it's an example).