Not to mention the hellishly expensive wafer handling equipment. It's already difficult to carry a boat of 300mm wafers, larger wafers (and containers) are even heavier. More clean space is required, more robotics, larger machines, more power! A 300mm fab is astoundingly expensive, a 400mm fab will only be available to the really, really big players. Contract fabs will be the norm for all but a few, in fact they are pretty much already the norm.
And the probe cards used for testing these wafers get bigger. Full wafer test 300MM cards are too heavy for our techs to easily carry and set up so we have mini-forklifts to move these things around. It takes two or three people to level and planarize them.
So where you used to hire a couple cleanroom techs to easily carry boats of wafers from process A to process B, you now have to buy robots and track systems...which need expensive maintenance engineers, more intricate fab design, etc. People forget how size gets expensive sometimes.