Firefox itself seems pretty decent these days. The biggest and ugliest problem is the extreme sluggishness of its most popular add-ons. I don't think I'd want to live without Firebug and AdBlock, but these two are huge performance hogs and I almost think they should be absorbed into the main codebase, rather than being sandboxed and crippled in their current incarnation.
A clean install of Firefox loads instantly, just like Chrome and IE9. As soon as I load those two add-ons, it starts taking 2-3 seconds to launch, pages often freeze up due to the repetitive and redundant DOM swizzling. This over-reliance on Javascript-based functionality leads to really sloppy performance and sometimes massive memory leaks. Right this second, with only two tabs open, Firefox is guzzling 450mb of memory. Chrome uses 1/10th of that to display the same content, with the same add-on functionality.
I've been holding out for a long time, but Chrome is starting to lure me over. I don't like being at the mercy of Google's totalitarian whims, but Firefox' idealism is wearing thin unless some real programmers get in there and clean things up. For the average user, Chrome is a clear winner simply because it's faster.