I worked for a 3PL (third party logistics) company. Years ago, they'd decided they were going to make $$$ with SaaS, basically selling our services to others. A huge undertaking had been embarked upon to make our system usable for other companies. They got a grand total of one client.
A few years later I was working there, and we got a second client! Bad news was, literally no one was still working there that had been when the first SaaS client had been set up. So there was a lot of guesswork trying to recreate it. I was a Junior Developer at the time, and was tracking down why some data loading wasn't working right. I knew the issue was almost definitely a trigger in the database, so that day I made some changes, loaded the days's data import into the Test DB, and checked if my fixed worked. It didn't, so I cleared out the load, made another change, and did it again. OK, now it was kind of fixed, but there was a problem somewhere else. Wash, rinse repeat.
I'm sure you see where things went wrong.
About the sixth or seventh time I did this, I accidentally ran it against production. I distinctly remember the panic that gripped me the moment I hit the F5 key to execute that SQL statement - I realized what I'd done immediately. The drivers (this was a logistics company, remember?) had been out on the road for about two hours at this point, and all the sudden all their handheld devices just stopped working. Where's the next stop? As far as their handheld was concerned they didn't even have a route, much less anything on the truck. This happened for all of the Office Depot drivers in Florida. And we couldn't just reload the day either. After the initial import happened at around 1:00 am a lot of virtual paperwork was done by humans to optimize routes and such, work that couldn't be easily duplicated.
I spun around in my cubicle and told him what I'd done immediately (I was told later I looked white as a sheet) and he assured me it'd be OK. An hourly snapshot was taken by the database. We'd lose a bit of data, but it wasn't the end of the world. He went to talk to the DB Admin.
Those snapshots? It turned out six weeks ago they'd just stopped running. Why? I don't think we ever figured out for sure, but either way they weren't there. Now everyone was panicking a bit. This was a new client we'd just picked up and we didn't want to screw the pooch. In the end, they ended up doing an emergency purchase of some software that allowed them to roll the database back using the transaction logs. Fun times.