If you live under the domain of a more enlightened electric utility company (or, if you prefer, a more regulated utility), there may be subsidized bulbs or rebates available for your CFL lamp and fixture needs. http://www.efi.org/ offers limited quantities at subsidized prices, primarily in the New England area. Even if you're not covered by the subsidy, EFI offers retail pricing and honors manufacturers' warrantees -- if your 10,000 hour CFL goes out a few years too soon, it will be replaced with minimal hassle.
Brand can be king and you get what you pay for. If you've had a bad experience with a particular brand but like the concept of CFLs, try another. There are some really shitty CFL manufacturers, to be sure. If you don't like the light it gives off, try a different color temperature (higher is whiter/"bluer", 2700k is "standard," about as close as they get to an incandescent temp) and wattage.
Mercury content is fairly negligible and is offset by reduction in coal-burning plant pollution. They can be recycled with many local recycling programs. Magnetic ballasts in CFL fixtures have been replaced by more efficient electronic ballasts that cut down on intereference, hum, and slow start times.
In addition to CFL subsidies, rebates are offered on Energy Star appliances. Check http://www.energystar.gov/ if you're in the market and take the time to do the math in terms of overall price and energy payback.
Call your utilities and see what else they might have to offer. There are low-interest loan programs out there for more efficient heating/cooling equipment. Replace your windows. Get an energy audit. Take advantage of federal tax credits. Learn how to regulate solar heat gain. There are any number of ways to cut costs and bring energy demand down regardless, if CFLs aren't your bag.