I've been involved in the bookkeeping end of several small businesses. My suggestions .... as to payroll, do not do this in house. Its a PITA, and you will wind up either making mistakes you will come to regret, or spend more time on it that you might be able to use more wisely somewhere else. Keeping up with local tax districts, who lives where, when the govt(s) (assuming you are US) decide to make some subtle change to tax regulations. Penny wise, pound foolish to do this in house for most small businesses.
As to basic accounting and booking, I am not aware of anything that does the sheer number of things that QuickBooks can do, or as well. But I haven't tried everything. This sounds to me like the kind of thing that is hard for free / open source projects to compete with. Maybe your needs are very simple? I would suggest some of the SaS products like FreshBooks. These may not meet your critieria of open source projects per se, but many of these have open API's which is a step in the right direction. Also, these are likely built with open source projects, eg Apache, Ruby, PHP, MySQL, Python and so on, so there is some second hand support of open source that way. They also free you from the local installation situation, and can be accessed via mobile or remote locations (ie work from home), with zero hassle. I have used FreshBooks for accts receivable situations, and found it very nice. Compared to QuickBooks, I would opt for FreshBooks for billing and invoicing easily. I have not looked at with acct payables or general ledger in mind though.