The choice of protocol is going to be your first decision. I picked Z-wave because there are many different manufacturers and a wide variety of devices. Every so often, someone will have devices on clearance or closeout. And you can find them in lots of places: Home Depot, Staples, and a wide variety of online merchants carry them. If you go with a more proprietary system like Insteon, you'll pay more per device and be more locked in.
The hub cost should be less important to you than the per-device cost. You need to buy only one hub, but you'll end up buying a lot of devices. Be prepared to pay about $40/device (list), or $10-20/device (clearance).
Unlike the protocol, the controller isn't something you have to be stuck with forever. You can upgrade them. Z-wave controllers are available in a USB stick form factor, which means you can build a home controller out of any computer you want. I chose to buy a Vera because I wanted an open system that didn't have a monthly fee, and I wanted the whole controller as a turnkey system. The Vera has a really good UI and a lot of mobile phone clients that connect to it, but there are several other Z-wave controller options, including HomeGenie (completely open source, designed around a small platform like the RasPi or the Beagle Board), and OpenHAB (platform agnostic Java, completely open source, but very weak UI.)
People have recently started rooting the Wink, which is a really cheap controller with a lot of connectivity options (including Z-wave), but it's not an open source device. And the Staples Connect Hub (made for them by D-Link) has lots of connectivity options for only $49. My Vera2 is currently at the limits of capacity for all my devices, so I'm considering options besides migrating to their Vera 3, including building my own HomeGenie system. The Vera 3 is about $299, but I think I could build a very capable HomeGenie box for under $100.