I work for a company that does a lot of Open Source stuff. Here is how we manage it: We have core toolkits that are open source, and custom applications that are closed source, made for specific customers. When ever a customer needs new functionality, we try to generalize it and put it into the toolkits, which we then release. We tell the customer that we have this open source toolkit which we use for the project, and which we keep improving. But we don't specify how much of the work goes into the toolkit, and how much on the custom side.
Those toolkits have been our main marketing effort, and have certainly paid off. Within our very narrow field we are world famous, and our toolkits almost dominate the market. Nobody can afford to build a competing one, when ours is free. Although anyone may use our tools, we happen to know them best and have most experience with them, so we can often do any given job faster than others. The company has survived over a decade, and has expanded internationally, and is now all of 15 people.