Agreed, it's the one thing that I keep using Firefox for. What popular Firefox extension features have been ported for Safari generally don't work nearly as well. For example, I don't want to use a system-wide proxy for ad blocking, and as nice as the web inspector is it's not as good as he Web Developer + Firebug extensions for Firefox... not to mention that most Safari tweaks require input manager hacks which are sketchy at best and prone to breaking between Safari releases.
If Safari had a system in place for writing extensions I wouldn't even have Firefox installed on my computer, it's generally painful to use for any period of time in OSX (it's a bad OSX "citizen" and feels like a ported Windows app). Page load performance I really don't care about, what I care about is the 1+ seconds it takes to open a new Firefox window once I have a bunch (~5-10) already open. Safari is instantaneous no matter how many windows I already have open, as it should be on a quad core system with 8 gigs of ram.