Industrial Strength Open Source Code? 68
dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"
ISO 9000 (Score:3, Insightful)
ISO 9000 just means the people that "certify" this have all their procedures documented.
I'd be much more interested in a company that delivered software that worked, and stood behind it, rather than a company that puts all it's time and effort into ISO 9000. Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.
Man, oh man.... The next thing people will be saying is that the company mission statements actually make a difference.
Open Source and ISo 9000 (Score:2)
I mean, you can *see* when the code was changed, you can *see* when bugs were fixed, you can see any QA that was done (if any)...
Shouldn't any Open Source product be able to get ISO 9000 certification, if they do the paperwork? I know that paperwork is *massive*, but
Re:Open Source and ISO 9000 (Score:2, Informative)
Re: (Score:1)
Re: (Score:3, Interesting)
Re:ISO 9000 (Score:4, Insightful)
IMHO, certification really means repeatability (do you always follow some established process). Which is certainly important, especially when it comes to very expensive/complex programs, in order to mitigate risk. But it's not the same as quality. One can repeatably implement a lousy process or procedure. One can also be repeatably bad at implementing a good process (you can have the best design process in the world, but a bad designer will still give you a bad product).
I think the whole certification buzz comes about in large part because it's difficult to evaluate the other big factors that impact quality, which are largely human factors - talent, workmanship, management acumen, etc. Because customers have little visibility into those factors, they instead evaluate the factor most amenable to measurement - process compliance.
People with some common sense recognize the certification as a useful but limited tool for evaluating a company. It's the other folks that you seem to have encountered - the ones who think process compliance is the Holy Grail for business success. Those are the ones I try to avoid
Re:ISO 9000 (Score:4, Interesting)
We were 9k certified.
Re: (Score:2)
"Extreme" methodologies, which are rapidly morphing into "Slash development time, skip the testing" methodologies.
ISO9000 used to be 'Know what you are doing and how to do it' before it became "document everything, including your documentation procedures, especially the procedures for documenting how you document the docuemntation".
Re: (Score:2)
In a small company I worked before (medical field), the documentation and process were not quite as nonexistent as you describe, but they were thin to put it politely. We were ISO 9000 certified, too.
Re: (Score:2)
Or, in my company's case, is in the medical fields, and thus is required by law to be ISO compliant (as well as CSA-compliant, etc.) in order to do business. Being and staying ISO9000-compliant isn't a cakewalk - you have to have documented procedures for everything you do, and your employees have to know (or know how to find) those procedures. I'd wager that most companies that become compliant
Re:ISO 9000 (Score:5, Interesting)
One of the engineers at work keeps a photograph of the Firestone plant that produced all the deadly, defective tires for Ford.
The photo shows clearly:
- The plant was shut down and closed.
- The plant has a large "ISO 9000 Certified" sign on the entrance sign.
ISO 9000 just means that you documented your procedures and you can verify and prove that you followed them. It does not tell you whether or not they're smart, safe, profitable, or anything else about your business.
In other words: ISO 9000 forces a company to document what they're doing, but can't save the idiots from doing the wrong things.
Re: (Score:1)
"No animal nor tree was injured in the making of this program."
Laff (Score:3, Funny)
Re: (Score:3, Insightful)
And I don't understand why the parent is modded troll.
Import it into your own code base, and review it. (Score:5, Interesting)
Mod Parent Up ISO9000 Compliant (Score:5, Informative)
You shouldn't even need to go to the lengths of merging the code bases.
Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.
Re:Import it into your own code base, and review i (Score:2)
I think that's exactly what he's trying to avoid: it is probably cheaper to build your own from scratch, than to integrate, adapt, understand, test and review foreign code. After all, actual coding takes negligible resources (20% ?) when writing mission critical software.
I think what he wants, is to import open source code directly, without extra testing and reviewing, under the premise that it's popularity is a goo
Re:Import it into your own code base, and review i (Score:1)
Re:Import it into your own code base, and review i (Score:4, Interesting)
Don't forget that you do have to do all these checks. As a pharma manufacturer I tell the FDA that I rely on CoAs from my vendors, but I rely on them only by getting samples from them, cross-checking their work, and then also cross-checking that on the raw materials that are actually used in manufacturing. And I check during the manufacturing process and after manufacture as well just in case!
But you can't reasonably do all this for, say, a whole O/S. It's just too big and too complicated. This is why you'll see medical systems (or avionics) running on LynxOS or Green Hills' OS rather than standard Linux, ITRON or eCos (though eCos is small enough that you could probably review it yourself too). Regretfully, some are starting to ship with Windows which I am sure has not been subject to the equivalent review.
So why am I so confident in saying this?
In other words I'm probably the only person on the planet who's been under GMP, under software ISO9000 and also been a free software developer.
Mod parent informative (Score:2)
How can we certify such software?" (Score:2)
Own it (Score:2, Insightful)
Re: (Score:2)
FWIW, I have written FDA-approved software (LIMS), and I know I wouldn't (and didn't) use any third-party stuff in any of my systems if I could write it myself. It takes more time, but we were known as miracle workers at my company, because we actually delivered wo
The starving programmers (Score:4, Insightful)
I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.
In this case, the budget to get a piece of software properly certified.
There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants)
It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.
But the problem is that few actually get paid to do this, it is done in spare time.
In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?
Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?
The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".
Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.
It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.
Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".
This doesn't seem right. It seems totally backwards to me.
It reminds me of a quote from "V for Vendetta": "People shouldn't be afraid of their governments, governments should be afraid of their people."
In other words, "Harware manufacturers shouldn't be beholden to the software companies, software companies should be beholden to the hardware."
The fact of the matter is that I just wish that after I pay $2,000 for my nice new system that it would run right out of the box. Macintosh is close - but I would have the added stipulation that the software on any hardware system be standards based so that I don't have a Dell OS, IBM OS, HP OS, Alienware OS...etc.
These are just thoughts, and I understand that there are many counterpoints.
You're right, it's budget (Score:2)
It's the fact that the license cost is zero that works against open source.
Huh?! Yeah, I know, that's hard to understand, but stay with me.
Think about it from the POV of whatever the "head of IT" is called in your company.
What's going to work for him come new job time? What will most qualify him for a "step up" in his career?
In one sentence: the size of the projects he's run. How is that size measured? Budget.
So there is zero incentive for "heads of IT" to go o
Re: (Score:2)
More relevant is the *benefit* of a project. If your project solves a problem worth a million bucks, while costing 100K to run, it's a *MUCH* better project than the 900K project that also saves (or earns) a million.
It's called ROI, and
Re: (Score:2)
Re: (Score:3, Interesting)
Candidate 3 Interview Excerpt: "I put in a new system X that would have cost $20 million, except I used Free Software and did it for only $1 million instead." "How successfull was it?" "Oh, extremely -- and I saved the company $19 million!"
In other words, calculate the cost of doing it the stupid way and then frame the discussion in terms of the amount of money you saved rather than the amount of money you spent.
Re: (Score:2)
Candidate 3: "Yes, but..."
Interviewer: "No, no, thanks, that's quite sufficient." *makes mark on clipboard* "Thanks for coming in. We'll call you."
Again, refusal to see the point I'm making doesn't change the point I'm making!
Of _course_ in an ideal environment all aspects of a person's experience would be taken into account. However, we're dealing with the real world, and the very real "pissing con
Re: (Score:2)
When someone asks what kind of projects you where responsible for, what is stopping you from saying; I was responsible for a large project that earned over $20 million for the company. You yourself has to consciously choose to use the cost of a project as a primary parameter. And that's just dumb.
The starving programmer? A myth. (Score:3, Interesting)
Philanthropically? Um. no. Free and Open Source software is written because the author needs it. They requre the use of it. By allowing others to use it, the author loses nothing, they are not losing their time because they needed the software in the first place and the effort to copy it is neg
Re: (Score:2)
With a commercial product they can at least afford to document the silly thing.
Re: (Score:2)
Probably because "Accountant" is not a creative process. There are many "Open Source Writers", "Open Source Poets", "Open Source Sculptors", and "Open Source Painters", etc.
"...
Re: (Score:2)
Problem is documentation (Score:3, Insightful)
Namely, that there isn't any. And I'm not talking about end-user documentation here, I mean process documentation. Specification documents and all that. The kind of stuff that normally gets developed alongside code, in any commercial/industrial development methodology.
There is an unspoken assumption behind the OSS ideals, and this is that the program's source code is the only documentation that anyone should ever need. This, frankly, is not true. It
Re: (Score:1)
Re: (Score:2)
Okay - who's standards?
You have to bare in mind interoperability. It is all well and good you want to standardize on 'Docbook' format for your documents - but might be a problem if you attempt to share these documents with your Microsoft Office-centric partners who have no desire to change.
The FDA's One Thing; Your Customers Are Another (Score:5, Informative)
The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?
It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.
If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).
Good luck!
Re: (Score:1)
Fork it? (Score:3, Interesting)
Re: (Score:2)
Frankly, if the developer has any desires to not produ
Re:Fork it? The Perfect leach. (Score:1)
even if it's only with funding and supplying the man power for the certification of a
certain piece of OSS?
Why not guide your boss's thinking down that avenue and let the corp,
which is probably benefiting from OSS with a couple of hundred Linux servers,
give a little something back to "the community" ??!
Why can't you just certify the source code?? (Score:2)
1. We have to review all of the code that is written, making sure that everything is traceable to a design specification.
2. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source.
Since it is "open source" wouldn't just #1 apply? I think #2 would only be if you can't review the source, or do you guys get that privledge at that level?
What ISO9000 really means (Score:4, Interesting)
ISO9000 means one thing:
Re: (Score:2)
Every organization struggles with doing what they say/think they're doing. How often have you heard the joke "do we do it the way we're supposed to do it, or the way we actually do it?" At the manufacturer where I worked, this was a common refrain.
The first step in eliminating the gap, and the point of ISO certification, is establishing the discipline of figuring out how things
Re: (Score:2)
This isn't as easy as it sounds. (Score:4, Interesting)
So, don't imagine that this is going to be an easy one. Open Source projects by IBM might be easier to get past the Great And All-Seeing (but definitely not all-knowing) Pointy Haired Bosses - at least some will be familiar with the phrase "nobody ever got sacked for buying IBM" and may even still believe that. Some of the Apache projects might be workable, provided you use the line of reasoning that since Apache is listed on IBM's website as a project they are working on, it is covered by the "nobody ever got sacked for buying IBM". You don't have to tell them that IBM is only one member of a large consortium, and it might be better not to.
Some projects were connected to IBM and other major corporations but are now independent. Postfix is an example of that. I believe evlog (Enterprise-grade event logging for Linux) is also such a project. Speaking of evlog, I would DEFINITELY suggest using it in any commercial or Government setting. It's not that good and Linux has plenty of other security, but "Enterprise-grade logging" is mandatory in many cases and this provides it. It ticks the right box, even if it doesn't do a whole lot more for you. It's a pure CYA and nothing more.
ISO 9000 (or later) compliance is probably the toughest requirement, as it stipulates documenting the process and activities, where the level of documentation depends on how critical the project is, and Open Source projects have neither that type of documentation or any real concept of criticalness as components are freely reusable. Your best bet is to work through vendors that are themselves ISO 900x certified AND supply either the Open Source OR the links to those projects, then argue that by documenting the use of a project that comes from an ISO 900x certified source, you inherit the certification indirectly. Some bosses will buy that easier than others and depending on the structure of the organization, you may have flexibility on who you present the case to. If so, shop.
It's jst a tool. (Score:2)
But must all the tools? Is your compiler? Is your IDE? I'm not sure that all the supporting infrastructure needs to be held to the same standard as code that goes into the final build. IF a tool finds a flaw, is that flaw somehow not a flaw if the tool is not certified?
Do it like a programmer (Score:1)
Open source use not prohibited for FDA approved... (Score:1)
Re:Open source use not prohibited for FDA approved (Score:1)
And quoting from the document 'General Principles of Software Validation; Final Guidance for Industry and FDA Staff' at http://www.fda.gov/cdrh/comp/guidance/938.html#_To c517237928 [fda.gov]
The Scope section of the document refers to just this type of situation:-
'Where the software is developed by someone other than the device manufacturer (e.g., off-the-shelf software) the software developer may not be directly responsible for compliance with FDA regulations. In that case, the
Cheat as everybody does (Score:2)
If you want any OSS project be ISO9000 certified all you have to do is employ somebody who creates these docs. He doesn't have to actually develop all he has to do is create all the docs. Yet none of the actual developers have to change anything since what counts are
Certifying open-source code (Score:2)
It's like 'how do you certify a new drug'. Taxol is an extract from yew trees; in principle an 'open-source' pharmaceutical, you discover it growing, rather than synthesise it from first principles. It's rumoured to be useful in treating some forms of human cancer.
It doesn't save everyone; but it does extend the lives of some.
If you go through the steps of how that got FDA approval, then you'll understand a credible way of arranging approval for "open source code wher
FDA regulations (Score:3, Informative)
Don't blame the individual inspectors, BTW. They are just doing the job handed to them.
The trick here is to find a way to satisfy the regulations. Get your quality department on side here, because they are the ones who actually have to justify it to the inspector, and its their jobs on the line if they don't succeed.
As for open source, you need to get it considered under the same rules as other "COTS" software. I don't recall the details, but basically you have to come up with convincing evidence that some kind of process is properly followed. ISO9000 is the cheat sheet for proprietary software because its evidence that something of the right sort is done and audited. Open source doesn't have that, so you have to do it yourself.
Start by writing down what a good open source project looks like. Take as much as you can from ISO 9000 here. Do they have coding standards? Show they are adhered to. Configuration management? CVS. Code review? Contribution policy. Defect tracking? Bugzilla, and cite statistics showing that reported defects get fixed. Most large successful open source projects can ace this stuff anyway: if they couldn't they wouldn't be any good.
For requirements you have to get a bit more creative. Is it documented at the user level (e.g. API)? If so then you can call that documentation the requirements document and show that the source complies with it. The fact that this documentation was written after the software is irrelevant from your point of view because you only need to demonstrate that the software is fit for purpose, and that the documentation shows this. Write this argument down and it pretty much covers you.
You might also run a code review of a sample of the software to demonstrate that it meets your normal quality criteria.
Obviously this costs more than simply saying "supplier has ISO9000". But if its cheaper than buying ISO9000 then its still the right thing.
Incidentally, when I was doing this (a couple of years ago) some FDA staffer talked to our Quality Dept saying that open source was random, uncontrolled stuff that should never be allowed near real software. Basically they regurgitated a load of MS FUD. Be ready for this, and be ready for your QA dept to recite it back to you.
Paul.
Re: (Score:2)
self-certifying is very expensive (Score:1)
The problem with doing your own audits of open source projects is that typically the open source project will have lots of extra bells and whistles and generality that your application doesn't need. It may also depend on various libraries that your application otherwise doesn't need. It may be very difficult to untangle the 20% of functionality that you actually need from the rest of the open source package. In the end, it may be easier to just write a "good enough" solution with only 80% of the feature
CFR 21 Part 11 (Score:1)
I work in the Pharma industry as well, and have some experience with validated systems - along with the associated regulations. CFR 21 Part 11 is the pertinent regulation for any and all pharmaceutical data systems - whether used for clinical trials or other cGMP purposes.
In a nutshell, it says nothing about an ISO 9000 requirement - that's SPECIFICALLY your company's policy. I understand why, it's the same deal as Sarbanes-Oxley compliance - they use ISO 9000 standard as the benchmark on which they make
Standards for Safety Critical Software (Score:1)
How about commercial vendors (Score:1)