Both IE and Chrome offer sandboxing, JIT hardening and ways to make vulnerable plug-ins less easy to exploit and gain access to system. Firefox offers none of these.
On the other hand only Firefox is checked with static analysis tools before released, meaning that there are very, very few actual flaws in the browser (IE might be, Chrome certainly isn't). For instance when Chrome added a very basic memory checker to their test servers they caught dozens of bugs -- and that's just from the most basic of runtime checks. When people have run their commercial static analyzers on Chrome they've found several hundreds of potential flaws.
What does this mean in practice? The inner sandboxed code in Chrome is wide open to attack. They aren't even using serious methods to try to protect that code and are instead relying completely on the sandbox. This is the reason why you'll get random crashes in Chrome, and why they purposely try to keep you from using too many tabs (if a process is rendering more than one tab then when it crashes more of your tabs have to reload). On the flip side, this is the reason why in a years of running Firefox nightly it has never crashed once. Yes, there are errors in Firefox, but they are complex ones not the simple mistakes that crash Chrome left and right.
Personally I've never had a malware in dozens of years, so browser stability matters a whole lot more to me than security. A sandbox would be nice, but one that is relied on and causes random page crashes is worse than not having one but having far fewer crashes.