That's the theory, but it's not the practice. They are taking just as long as large reactors. In some senses longer, because none has actually been put into commercial operation, whereas new large reactors have come online. And they turn out to have very little actual modularity and scalability, because there's more than 150 designs out there, so no-one is winning enough of the market for repeatable learnings, and anyway, you'll never get to volume manufacturing for the parts involved, because the quantities are just too small, and everything still requires literal nuclear-grade engineering.
Take a motor-operated isolator valve that you'd find in a cooling system. You find plenty of these outside SMR applications, often 4 to 6 inch sizes, and they might cost something like 3 to 10k USD in a typical demanding industrial application. But they will cost more like 30 to 100k USD in an SMR, because they need to be made from nuclear-grade materials with documented material traceability, need seismic qualification, need nuclear-specific quality assurance, inspection and documentation, and need highly secure supply chains (no-one wants a counterfeit component showing up in a nuclear safety system). And they won't get much cheaper, because of strict configuration control, which limits the extent of any changes to components, materials and manufacturing processes.
It's also important to recognise that the economies of learning that apply afe fundamentally different and more limited for construction projects, which is what every SMR is, than for component manufacturing. That reflects the facts that (1) different places have different ground conditions, seismic risk, regulatory requirements, humidity, water access, etc, etc, and (2) huge amounts of stuff still have to be constructed on site by humans (even if assisted by machines), compared with a factory line that can be automated. This is still a problem even for solar, where the panels themselves can be cheap compared to the install costs, which is why there are various attempts being made to turn more of the deployment into a manufactured process rather than a construction process. Eg https://5b.co/en/5b-maverick