...or try to use them but fail and fallback to IPv4 a millisecond later.
Almost right. If the browser thinks it has IPv6 connectivity it will try, and it will fallback to IPv4. The fallback to IPv4 will take something like 30-60 seconds though!
That's exactly the problem Yahoo faces. People with broken IPv6 connectivity will experience serious delays when visiting their site. This has withheld other large sites, such as Google, from running dual-stack before. A test done by the large German website Heise.de shows the reality is not that rough. During one day they enabled dual-stack access for their website. Among the approximately one million visitors, only five experienced problems due to broken IPv6. After this experiment they decided to simply keep IPv4 + IPv6 enabled. See http://www.heise.de/netze/meldung/IPv6-Tag-bei-heise-de-Erste-Ergebnisse-1081201.html
Of course this percentage will vary according to the demographic of a site so, as Heise has shown, the best way is to test it. On 8 June, 2011, Google, Facebook, Yahoo!, Akamai and Limelight Networks will be amongst some of the major organisations that will offer their content over IPv6 for a 24-hour "test drive". See http://isoc.org/wp/worldipv6day/