Comment Fix was simple (Score 1) 15
Fix was simple, go into safe mode, run some dos commands to remove the update, boot up.
The problem was related to having to touch each server.
Running VMs, or servers with IPMI you could handle them remotely and quickly. if you have monitoring you know which servers got affected pretty quickly.
Took me an hour from recognizing the problem to fixing the few servers affected.
Luck and proper systems in place can reduce many outages.
I have to question those that took a long time, unless there are blockers in place like no remote hands, no IPMI, bad P&Ps, and/or software issues that arise from environments half going down.
Feels like a lot of problems should have been realized with this outage.
The problem was related to having to touch each server.
Running VMs, or servers with IPMI you could handle them remotely and quickly. if you have monitoring you know which servers got affected pretty quickly.
Took me an hour from recognizing the problem to fixing the few servers affected.
Luck and proper systems in place can reduce many outages.
I have to question those that took a long time, unless there are blockers in place like no remote hands, no IPMI, bad P&Ps, and/or software issues that arise from environments half going down.
Feels like a lot of problems should have been realized with this outage.