reaper's Journal: Anatomy of rebuilding a Windows 2003 server

So, I get a gig trying to rebuild a windows 2003 server that has recently had problems with such trivial tasks as adding machines to the network, and booting. It's interesting how the progression of events leads to a better, and better understanding of what's going on, and how things actually work. I'd like to share this experience:

• Find that the system fails to boot reliably, and cannot do any functions involving COM applications. Booting into Linux to diagnose the hardware shows a clear problem with the SATA-RAID card losing interrupts. Card interoperability problems. Booting to a recovery console, and watching it blue screen in the driver every time it hits a certain spot in the chkdsk confirms this.
• Breaking the mirror, and dumping the disk to a regular IDE disk shows that it has tons of bad sectors. Card firmware has poor handling of partial disk loss.
• Using the other disk in the mirror to create the PATA disk works like a charm. Booting off of this disk brings the system up reliably, but still has the COM errors. Disk corruption probably led to the COM errors.
• Using MS's instructions to rebuild a COM catalog (using the less destructive way)causes the system to blue screen on boot. COM is used in the boot process by something .
• I used the upgrade in place facility (AKA Repair) of 2K3 server to fix the COM error, to no avail on the phase 2 install. This resulted in more blue screening. The upgrade is not thorough in fixing COM, probably because it would have to reinstall all the previously installed apps. I'd also like to add that this added an hour to the rebuild time.
• You now have to boot to a recovery console in order to go the full-out, destructive COM wipe route in hopes of making the system boot, which could have probably been done before you tried re-installing windows. Windows makes me cry, sometimes
• After going to a recovery console, and renaming the c:\windows\registration directory, the system continues to the stage 2 of upgrade/repair.
• "Approximately 12 minutes left" means suck my left testicle, luser!.
• You get real punchy at 5am

Anyway, this should get finished up now. There's no way I'm shooting for any more than "the way it was before", but I think it'll come up OK, once it's done registering components.

Ya, 12 minutes.... my ass.