1) Don't leave them turned on, especially not in the original system. If the system gets hit by lightning/surge/etc you don't want it to take your backup with it.
2) Rotate your backups. Assuming you have more than one removable media, swap them out in a rotating fashion.. That gives you multiple restore points in case you have one of the backup media volumes go bad.
3) If you use an internal drive dock, get one that encloses the drive so it has basic physical protection from handling. Store them in the plastic (usually antistatic) bag they came in somewhere away from the main system.
4) Instead of using an internal dock with multiple internal drives that all need their own dock sled, consider an external USB/Firewire/eSATA drive instead. You can even get ethernet based drives too, so they can be used on your home network. You can buy an external drive enclosure for pretty cheap and use the same internal drives saving a bit of money, making it about the same cost as using an internal drive docking solution with the bonus that you can then access your backup from ANY system, not just the one that has the dock.