I haven't used their VPS but you can look into Dreamhost. I've been very happy with my shared web account.
For what it's worth, I used to think I needed a VPS as well. Then I found myself with limited funds for awhile so I transferred my site to Dreamhost on their shared hosting plan. At the time they had a deal that it was around $48 to pre-pay for 2 years. It averages out to about $1.95/month. In addition to the unlimited domains/disk/bandwidth and mysql database, the account comes with 1 free domain registration, easy setup to have email hosted either through Dreamhost themselves or through Google Apps [which I do]. And it comes with a shell account, which is on the webserver itself running Debian. They have all kinds of one-click installs for various blogs and ecommerce options and whatnot.
As for stuff like git, I'm not sure. I see that there are some git commands on the server, but I don't know if it's possible to run a git server.
At first I thought I would hate it, and would be transferring back to a VPS/Dedicated server first chance I got. Well, it's been 2.5 years now and I have no desire to switch back. I haven't really missed having root access, and it's nice not to have to worry so much about managing everything. Admittedly, this is just personal sites of fairly low importance, but I've been happy with the setup so far. And they don't seem to mind me sitting on irssi on the shell account all day. The only thing it does is it will automatically kill any single process which has been running for a week.
Anyway, it's worth a look and if you don't like it, I think they have a relatively painless process to transfer to a VPS.
As an added bonus, they've been against SOPA since November, according to their official blog.