Forgive me for not recapitulating the entire world's history for you. I will, however, focus on just one civilization to make my point. Take Egypt, now a desert. Archaeological evidence points to it being a lush fertile land in the time of the pharaohs. It needed vast amounts of wood, for instance, for burning to cook food, make tools and buildings, for building things like pyramids, etc. Agriculture flourished in places were native vegetation grew. And as cities grew, they cut down an expanding circle of trees until it was no longer possible to transport the necessary supplies the long distance. Without trees, the soil washed away. And now it's desert. As in, in recent times. And shortly after the empire declined.
Same basic thing happened in southwestern US (Anasazi), Mexico (Aztecs), Argentina (Incas), Gobi desert (Chinese empires), and so on. They were all fertile and lush, which led to empires using the natural resources, which led to desert. Now, if you have evidence that these empires were built on, and thrived in, desert, please present it.