Firefox itself seems pretty decent these days. The biggest and ugliest problem is the extreme sluggishness of its most popular add-ons. I don't think I'd want to live without Firebug and AdBlock, but these two are huge performance hogs and I almost think they should be absorbed into the main codebase, rather than being sandboxed and crippled in their current incarnation.
I've been holding out for a long time, but Chrome is starting to lure me over. I don't like being at the mercy of Google's totalitarian whims, but Firefox' idealism is wearing thin unless some real programmers get in there and clean things up. For the average user, Chrome is a clear winner simply because it's faster.