I remember initially setting up our little site with 3 servers and a "managed" loadbalancer/failover solution from our hosting provider. Our domain name pointed to the IP address of the loadbalancer.
I learned that "managed" is actually a hosting company euphemism for "shared" and performance was seriously degraded during "prime time" everyday.
We eventually overcame our network latency issues by ditching the provider's loadbalancer and using round-robin DNS to point our domain name at all three of the 3 servers.
I was using Apache + JBoss + MySQL, and on each server I configured Apache's mod_jk loadbalancer to failover using AJP over stunnel to the JBoss instances on the other 2 servers. I also chose to configure each JBoss instance to talk to a MySQL instance on each box, these being configured in a replication cycle with the other MySQL instances for hot data backup.
For our load, we've never had any problems with this. The biggest component with downtime was JBoss (usually administrative updates), but Apache would seamlessly switch over to use use a different JBoss instance.
One of the servers was hosted with a different provider in a different site.
