How do browsers behave when 1 of the server goes down?
Half the DNS lookups will still point at the failed server. Since most browsers cache the dns lookup, they will not re-request the IP address, and will just assume the site is down.
If you rely on DNS round-robin records, you need to either ensure that they are always up (i.e. each one is an HA cluster) or that you can remove them quickly enough to cause your users as little pain as possible. (TTL should probably be 1 minute, and you'll want an automated method of removing the bad entry from the zone file)