Re:Make it fail (Score 1)

Damn right. Look at failure scenarios in depth. Despite generally good experiences with Cxxxo gear, they often did not explain failure modes.

For networks, Layer 2 versus Layer 3 load-sharing or failover. Time to restore service. I once saw a web farm for one of the top news sites fail for 45 minutes because some junior sysadmin configured a Sun box with the switch management IP address in a large Layer 2 domain, and VTP went down.

Little things, like not having enough RAM on a supervisor module caused the box to be down for 2 hours while some poor guy baby-sat an xmodem serial transfer from a laptop.

Note, some things like redundant switch fabrics are really only the cost if you're writing 5 9's SLAs for paying customers.

Ultimately, it takes experienced, senior engineers to know what to buy. Not some $40K MS cert d00d who thinks he can do the job.

-RJ