Industrial Strength Open Source Code?

dnnrly asks: "I work for a company that writes software for the pharmaceutical industry. We have to work in quite a tight regulatory environment because some of our code ends up in the process of drug testing. Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs, our clients have strict rules that we must follow for coding. We have to review all of the code that is written, making sure that everything is traceable to a design specification. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source. This is a bit of a problem when we would like to use open source stuff in our code. Projects like log4net and NUnit would be tremendously useful in our code but we're not allowed to use them because they don't tick the right boxes. Now, *I* know that these projects (and others) are incredibly stable just because of the volume of use that they have seen but that isn't enough for some people. How can we certify such software?"

ISO 9000

[deanj]

No offense intended, but an ISO 9000 source?

ISO 9000 just means the people that "certify" this have all their procedures documented.

I'd be much more interested in a company that delivered software that worked, and stood behind it, rather than a company that puts all it's time and effort into ISO 9000. Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.

Man, oh man.... The next thing people will be saying is that the company mission statements actually make a difference.

Open Source and ISo 9000

[brunes69]

ISO 9000 mainly revolves around the procedures for developing the software being documented. Now, I may be oversimplifying things, but in the Open Source world, isn't *every* procedure documented, since the whole process is totally transparant?

I mean, you can *see* when the code was changed, you can *see* when bugs were fixed, you can see any QA that was done (if any)...

Shouldn't any Open Source product be able to get ISO 9000 certification, if they do the paperwork? I know that paperwork is *massive*, but

Re:Open Source and ISO 9000

[Entrope]

No; quite the opposite. Open source development that accepts third-party contributions makes it extraordinarily difficult to meet ISO 9000 process requirements. The standard is not just about documenting the process, but making sure you follow it consistently. When you have workers whose compliance is not audited, you cannot pass the audit. This means that continuously accepting third-party contributions has to be designed into the documented process -- something alien to most ISO 9000 practitioners --

Re:

[charlesnw]

So have a procedure in place that people have to follow in order to submit code. Problem solved. You then have a secondary procedure that does automated testing. Which you should have in place anyway. Like the mozilla project for example.

Re:

[mabhatter654]

A Distro or individual project could get certified if they put the work into it.. it could even be a selling feature. I'd expect Suse and Red Hat already have something in place to satisfy these requirements as they deal in large enterprise installs already. OSS and ISO really do go hand-in-hand, but hackers tend to like to do things their way... and ISO is all about following instructions. Still, it could be a neat project to provide ISO, Sox, HIPAA, etc. testing to OSS projects in some fashion that wa

Re:ISO 9000

[Stradivarius]

I'm sure many of the companies that are certified would probably agree with you. But the fact is that many companies, particularly those selling to government entities, are required to be certified in ISO 9000 or similar programs as a precondition of doing business. So they have to go along, at least to some extent, with the fiction that certification equals quality.

IMHO, certification really means repeatability (do you always follow some established process). Which is certainly important, especially when it comes to very expensive/complex programs, in order to mitigate risk. But it's not the same as quality. One can repeatably implement a lousy process or procedure. One can also be repeatably bad at implementing a good process (you can have the best design process in the world, but a bad designer will still give you a bad product).

I think the whole certification buzz comes about in large part because it's difficult to evaluate the other big factors that impact quality, which are largely human factors - talent, workmanship, management acumen, etc. Because customers have little visibility into those factors, they instead evaluate the factor most amenable to measurement - process compliance.

People with some common sense recognize the certification as a useful but limited tool for evaluating a company. It's the other folks that you seem to have encountered - the ones who think process compliance is the Holy Grail for business success. Those are the ones I try to avoid :-)

Re:ISO 9000

[Monkelectric]

I think ISO 9k is an industry scam. I worked for a company which had no process whatsoever -- literally none. When a new piece of software needed to be written -- someone would walk into your office and say, "I need something to do X" That *WAS* the entire software process. No requirements, no architect, no analysis, no test plan, no documentation, no testing, nothing.

We were 9k certified.

Re:

[ErroneousBee]

until a new, even worse, thing supplants ISO 9000.

"Extreme" methodologies, which are rapidly morphing into "Slash development time, skip the testing" methodologies.

ISO9000 used to be 'Know what you are doing and how to do it' before it became "document everything, including your documentation procedures, especially the procedures for documenting how you document the docuemntation".

Re:

[Lonewolf666]

Scam might be a bit exaggerated, but ISO 9000 is indeed easy to get.

In a small company I worked before (medical field), the documentation and process were not quite as nonexistent as you describe, but they were thin to put it politely. We were ISO 9000 certified, too.

Re:

[Sentry21]

Every time I hear someone asking for that, they're more interested in being buzzword compliant than anything else.

Or, in my company's case, is in the medical fields, and thus is required by law to be ISO compliant (as well as CSA-compliant, etc.) in order to do business. Being and staying ISO9000-compliant isn't a cakewalk - you have to have documented procedures for everything you do, and your employees have to know (or know how to find) those procedures. I'd wager that most companies that become compliant

Re:ISO 9000

[NateTech]

ISO 9000 is a documentation process, not a quality process. People don't seem to get that, and the marketing spin around it for years has not exactly made that very clear.

One of the engineers at work keeps a photograph of the Firestone plant that produced all the deadly, defective tires for Ford.

The photo shows clearly:
- The plant was shut down and closed.
- The plant has a large "ISO 9000 Certified" sign on the entrance sign.

ISO 9000 just means that you documented your procedures and you can verify and prove that you followed them. It does not tell you whether or not they're smart, safe, profitable, or anything else about your business.

In other words: ISO 9000 forces a company to document what they're doing, but can't save the idiots from doing the wrong things.

Re:

[meiao]

I'd like to see some ISO 14000 code.

"No animal nor tree was injured in the making of this program."

Laff

[Konster]

"Seeing as the FDA are quite picky about making sure that there can be no errors in testing new drugs."

Re:

[ClamIAm]

The testing has to be perfect. The drugs? Not so much.

And I don't understand why the parent is modded troll.

Import it into your own code base, and review it.

[ankhcraft]

Simply import it into your own code base, and then review it as if it was written internally. Basically, learn it inside out, as if you wrote it yourself. If that is not legally sufficient, then the laws need to be rewritten since the lines they would be attempting to delineate would at this point be completely imaginary. It doesn't matter whose head it originates from, what matters is that it is fully reviewed and completely understood to the point where everyone on your team is prepared to stand behind the entire body of code. If that confidence comes from actual understand, it becomes irrelevant who wrote the code in the first place. How would it be any different if, instead, it was code written by somebody who no longer works at the company.

Mod Parent Up ISO9000 Compliant

[argent]

ISO9000 just means you have documented procedures and you track the steps you take when you follow those procedures.

You shouldn't even need to go to the lengths of merging the code bases.

Document the procedures for auditing and verifying external software to the standards of your internally written software, create an ISO9000 process for tracking those procedures, and follow them.

Re:Import it into your own code base, and review i

[Stellian]

Simply import it into your own code base, and then review it as if it was written internally.

I think that's exactly what he's trying to avoid: it is probably cheaper to build your own from scratch, than to integrate, adapt, understand, test and review foreign code. After all, actual coding takes negligible resources (20% ?) when writing mission critical software.
I think what he wants, is to import open source code directly, without extra testing and reviewing, under the premise that it's popularity is a goo

Re:Import it into your own code base, and review i

[nonsequitor]

ISO9000 is just part of it, and ISO9000 audits are a joke. Right now I'm testing avionics code, tracability of requirements is of course big. To use a 3rd party tool in verification that tool must be verified to the same standard that the project is being verified against. So for Level C code, all 3rd party tools must be verified for DO-178B Level C, and for Level A projects, the tools have to be verified to Level A as well. There are various loopholes as well. To speed things up writing C unit tests f

Re:Import it into your own code base, and review i

[real gumby]

dnnrly: ankhcraft has it right (see below for my credentials for why I say this). Your customers are presumably operating under GMP and/or GLP (the pharmaceutical equivalent of ISO9000 and which is all described in CFR Title 21). They need a basis for your attestation as to the function and performance of the product you provide. Your using vendors claiming to adhere to ISO 9000 is really just a (reasonable!) way for you to not do as much cross-checking on their execution. If you import the software package and review it you can then make whatever performance claims you feel comfortable making -- probably far more robust ones than you could about most of your other outside vendors!

Don't forget that you do have to do all these checks. As a pharma manufacturer I tell the FDA that I rely on CoAs from my vendors, but I rely on them only by getting samples from them, cross-checking their work, and then also cross-checking that on the raw materials that are actually used in manufacturing. And I check during the manufacturing process and after manufacture as well just in case!

But you can't reasonably do all this for, say, a whole O/S. It's just too big and too complicated. This is why you'll see medical systems (or avionics) running on LynxOS or Green Hills' OS rather than standard Linux, ITRON or eCos (though eCos is small enough that you could probably review it yourself too). Regretfully, some are starting to ship with Windows which I am sure has not been subject to the equivalent review.

So why am I so confident in saying this?

• I am currently in the pharma business and running under GMP right now myself so am painfully aware of what it requires.
• I've previously (in previous companies) had plenty of customers who themselves run under ISO 9000 (in telecom, avionics, automotive systems, medical devices, military etc) and so know what they demand of themselves and of their vendors (e.g: downtime requirements of less than 3 minutes per decade)
• I was a cofounder of the first free software (we predated the term "Open Source") business, Cygnus Support, where we had those ISO 9000 customers and satisfied them.
• you're not my company. You'll have to cross-check my suggestions yourself.

In other words I'm probably the only person on the planet who's been under GMP, under software ISO9000 and also been a free software developer.

Mod parent informative

[Eivind Eklund]

It's got the perfect goods for this particular discussion.

How can we certify such software?"

[thrillseeker]

You need to get the kind of people involved who can statistically show that particilar software approaches lead to less problems. Otherwise, you're stuck with the expense NASA goes to in writing software.

Own it

[keithmo]

Could you, for the purposes of certification, take ownership of the libraries you want to use? By "take ownership" I mean either a) acquire, or b) create the necessary design documents for the libraries, then code review them as you would your own code.

Re:

[Doctor Memory]

That's part of his point, in order to accept the OSS he (or someone on his team) would have to write the design specs, then validate the code against them. Which would basically take as much time and effort as writing it from scratch themselves.

FWIW, I have written FDA-approved software (LIMS), and I know I wouldn't (and didn't) use any third-party stuff in any of my systems if I could write it myself. It takes more time, but we were known as miracle workers at my company, because we actually delivered wo

The starving programmers

[w33t]

This brings to mind a conversation I had the other day.

I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.

In this case, the budget to get a piece of software properly certified.

There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money.

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

But the problem is that few actually get paid to do this, it is done in spare time.

In my conversation I wondered how else software writers could make money, besides the tried and tested subscription and serial-activation systems in widespread use today. How else could programmers, who are doing some of the most mind-bending, skillful crafting of any career, get compensated for their work?

Wouldn't it be great if AMD, Intel, ASUS - or indeed any large electronics manufacturer - would offer up a pool to develop software?

The trick would of course be that the software should of course be standards compliant and thus run even on competitors hardware. But maybe AMD could have a certification that says, "written for and extensively tested on AMD hardware".

Software seems that it should be freely available - it just seems the nature of all information in general - but there is that problem that the programmer needs to make money.

It just seems to me that logically the consumer should buy the hardware - the physical, tangiable thing - and that it should be up to the hardware manufacturers to make hardware as a whole more useful.

Here's the irony in my eyes - right now hardware manufacturers pay Microsoft in order to get a little sticker that says "built for Windows XP".

This doesn't seem right. It seems totally backwards to me.

It reminds me of a quote from "V for Vendetta": "People shouldn't be afraid of their governments, governments should be afraid of their people."

In other words, "Harware manufacturers shouldn't be beholden to the software companies, software companies should be beholden to the hardware."

The fact of the matter is that I just wish that after I pay $2,000 for my nice new system that it would run right out of the box. Macintosh is close - but I would have the added stipulation that the software on any hardware system be standards based so that I don't have a Dell OS, IBM OS, HP OS, Alienware OS...etc.

These are just thoughts, and I understand that there are many counterpoints.

You're right, it's budget

[Howzer]

But it's not the compliance budget, as you say.

It's the fact that the license cost is zero that works against open source.

Huh?! Yeah, I know, that's hard to understand, but stay with me.

Think about it from the POV of whatever the "head of IT" is called in your company.

What's going to work for him come new job time? What will most qualify him for a "step up" in his career?

In one sentence: the size of the projects he's run. How is that size measured? Budget.

So there is zero incentive for "heads of IT" to go o

Re:

[Eivind]

That'd perhaps be true if projects where only measured by cost, and not by benefit. Which would be amazingly stupid. What you're basically saying is the more resources you waste (i.e. higher cost) the more chance of promotion. If any companies work like that, I pity them.

More relevant is the *benefit* of a project. If your project solves a problem worth a million bucks, while costing 100K to run, it's a *MUCH* better project than the 900K project that also saves (or earns) a million.

It's called ROI, and

Re:

[Howzer]

You're right. And companies generally do try, albeit internally, although their methods differ, which is half the point. When a person leaves a company, though, the measure of projects used in interviews (which is what I was talking about) is generally just cost. Candidate 1 Interview Excerpt: "I put in a new X system that cost $20 million." "How successful was it?" "Oh, extremely." Candidate 2 Interview Excerpt: "I put in a new X system that cost $5 million." "How successful was it?" "Oh, extremely." W

Re:

[mrchaotica]

Candidate 3 Interview Excerpt: "I put in a new system X that would have cost $20 million, except I used Free Software and did it for only $1 million instead." "How successfull was it?" "Oh, extremely -- and I saved the company $19 million!"

In other words, calculate the cost of doing it the stupid way and then frame the discussion in terms of the amount of money you saved rather than the amount of money you spent.

Re:

[Howzer]

Interviewer: "So really, Candidate 3, the budget of the largest actual project you've managed was $1 million?"

Candidate 3: "Yes, but..."

Interviewer: "No, no, thanks, that's quite sufficient." *makes mark on clipboard* "Thanks for coming in. We'll call you."

Again, refusal to see the point I'm making doesn't change the point I'm making!

Of _course_ in an ideal environment all aspects of a person's experience would be taken into account. However, we're dealing with the real world, and the very real "pissing con

Re:

[Eivind]

But that's just bad marketing of self on behalf of the candidate.

When someone asks what kind of projects you where responsible for, what is stopping you from saying; I was responsible for a large project that earned over $20 million for the company. You yourself has to consciously choose to use the cost of a project as a primary parameter. And that's just dumb.

The starving programmer? A myth.

[Colin Smith]

It is truly inpspiring that so many can work together so well towards a common goal, and it is truly stunning to take in the vast amount of software available which is written pretty much completely philanthropically.

Philanthropically? Um. no. Free and Open Source software is written because the author needs it. They requre the use of it. By allowing others to use it, the author loses nothing, they are not losing their time because they needed the software in the first place and the effort to copy it is neg

Re:

[shmlco]

Too many open source projects think the project is done when the code is 99% feature complete. We were evaluating content management systems the other day and started looking at FOSS projects. So we're looking at Joomla, features, etc., and it looks good. Then we get to the documentation and it seems like practically every other entry in the User manual's TOC is "todo". Not good.

With a commercial product they can at least afford to document the silly thing.

Re:

[ratboy666]

"There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another to freely donate large sums of personal money."

Probably because "Accountant" is not a creative process. There are many "Open Source Writers", "Open Source Poets", "Open Source Sculptors", and "Open Source Painters", etc.

"...

Re:

[Sentry21]

I think the main reason that (F)OSS is still having trouble competing (despite the widespread acceptance amoungst industry experts) is because of budget.

In this case, the budget to get a piece of software properly certified.

There are many aspects of creating software that require non-technical administrative personell to handle. I don't remember ever hearing about OSA (Open Source Accountants) ;) These people cost money. And it's one thing to freely dedicate your time to devlopment, but it's quite another t

Problem is documentation

[Kadin2048]

I think the problem that OSS software has is documentation.

Namely, that there isn't any. And I'm not talking about end-user documentation here, I mean process documentation. Specification documents and all that. The kind of stuff that normally gets developed alongside code, in any commercial/industrial development methodology.

There is an unspoken assumption behind the OSS ideals, and this is that the program's source code is the only documentation that anyone should ever need. This, frankly, is not true. It

Re:

[charlesnw]

Something like this? http://readyset.tigris.org/ [tigris.org]

Re:

[Lodragandraoidh]

I would have the added stipulation that the software on any hardware system be standards based...

Okay - who's standards?

You have to bare in mind interoperability. It is all well and good you want to standardize on 'Docbook' format for your documents - but might be a problem if you attempt to share these documents with your Microsoft Office-centric partners who have no desire to change.

The FDA's One Thing; Your Customers Are Another

[occamboy]

I have a fair bit of experience on the medical device side, not pharma - but they should be somewhat similar.

The FDA has standards for validationg COTS (Commercial Off-The-Shelf Software). ISO9000 is useful, but not necessary for FDA "stuff". For example, I don't think that any OS (Windows, Linux, etc..) is ISO9000 compliant, but your software runs on it, no? How could that be if ISO9000 was mandatory?

It seems to me that, for the FDA to be happy, you'd need to run some sort of reasonable validation of the COTS software, based on how it's being used; write a set of requirements, write a set of tests that prove that those requirements are met, and go to town. It would be good to recognize in the FMEA (Failure Modes and Effecta Analysis) that the COTS software probably has somewhat of a chance of failure - but then, the FDA (at least on the device side) has already announced that you must assume that software will sometimes fail, no matter how it's been tested. Finally, be exacting about audit trail - versions used, how things were built and installed, and so forth.

If your customer insists on third-party code being ISO 9000, despite the FDA not requiring it, then you're SOL. However, that requirement just doesn't make sense, unless they are not using an OS to run their software on (as described earlier).

Good luck!

Re:

[SIWaters]

The FDA has a standard that's part of SRC 21 for medical device compliance -- 510(k). At its most basic level, it's a documentation process -- you have to document every market requirement, part spec, manufacturing prtocess, QA process, as well as the history of every part that goes into a device including who ordered it, when it was ordered, who it was ordered from, who received it on the loading dock, who accepted it, who did testing against spec, what device it went into, if there was ever a complaint fi

Fork it?

[rolfwind]

I'm not familiar with this software nor their licenses, but can't you just take a build of their software, fork it off as your own build and start treating it as internally made software? The greatest expense would be then certifying that first build.

Re:

[Jeff DeMaagd]

I'm thinking that the cost of certifying OSS code might be pretty close to that of just writing new code in the first place. And then there are concerns over whether the software is considered to be redistributed such that the code changes would need to be returned to the public, especially if it is full GPL and whether any commercial code that links to the GPL code would technically need to be released if the software is considered to be redistributed.

Frankly, if the developer has any desires to not produ

Re:Fork it? The Perfect leach.

[gd23ka]

Huh? What's wrong with a corporation to "give back" once in a blue moon
even if it's only with funding and supplying the man power for the certification of a
certain piece of OSS?

Why not guide your boss's thinking down that avenue and let the corp,
which is probably benefiting from OSS with a couple of hundred Linux servers,
give a little something back to "the community" ??!

Why can't you just certify the source code??

[JoeCommodore]

1. We have to review all of the code that is written, making sure that everything is traceable to a design specification.

2. Where we use 3rd party software/code we have to make sure that it comes from an ISO9000 source.

Since it is "open source" wouldn't just #1 apply? I think #2 would only be if you can't review the source, or do you guys get that privledge at that level?

What ISO9000 really means

[hacker]

ISO9000 means one thing:

Our process sucks, but its well-documented.

Re:

[jjohnson]

That's the standard joke, but it exposes the basic point of ISO9000 certification: Your processes suck, but do you even know why?

Every organization struggles with doing what they say/think they're doing. How often have you heard the joke "do we do it the way we're supposed to do it, or the way we actually do it?" At the manufacturer where I worked, this was a common refrain.

The first step in eliminating the gap, and the point of ISO certification, is establishing the discipline of figuring out how things

Re:

[darkmeridian]

As opposed to processes which may or may not suck but which cannot be measured because of the lack of documentation? In defense of ISO 9000, documentation allows reversions and enables organizations to learn from mistakes. I think of ISO 9000 as CVS: the code may be retarded, but CVS allows you to follow the progression of code so you can change it.

This isn't as easy as it sounds.

[jd]

I've been told by bosses when working for the DoD that although the code I wanted to use was indeed audited, FIPS-compliant and published by the DoD themselves, it wasn't on an approved list of certified FIPS-compliant software so wasn't acceptable. When I asked how something the DoD wrote and published as an official DoD application could possibly not be acceptable for use by the DoD, I was told that procedures were procedures and had to be followed. The fact that it was GOTS and written by the people who do the certifying was of no consequence.

So, don't imagine that this is going to be an easy one. Open Source projects by IBM might be easier to get past the Great And All-Seeing (but definitely not all-knowing) Pointy Haired Bosses - at least some will be familiar with the phrase "nobody ever got sacked for buying IBM" and may even still believe that. Some of the Apache projects might be workable, provided you use the line of reasoning that since Apache is listed on IBM's website as a project they are working on, it is covered by the "nobody ever got sacked for buying IBM". You don't have to tell them that IBM is only one member of a large consortium, and it might be better not to.

Some projects were connected to IBM and other major corporations but are now independent. Postfix is an example of that. I believe evlog (Enterprise-grade event logging for Linux) is also such a project. Speaking of evlog, I would DEFINITELY suggest using it in any commercial or Government setting. It's not that good and Linux has plenty of other security, but "Enterprise-grade logging" is mandatory in many cases and this provides it. It ticks the right box, even if it doesn't do a whole lot more for you. It's a pure CYA and nothing more.

ISO 9000 (or later) compliance is probably the toughest requirement, as it stipulates documenting the process and activities, where the level of documentation depends on how critical the project is, and Open Source projects have neither that type of documentation or any real concept of criticalness as components are freely reusable. Your best bet is to work through vendors that are themselves ISO 900x certified AND supply either the Open Source OR the links to those projects, then argue that by documenting the use of a project that comes from an ISO 900x certified source, you inherit the certification indirectly. Some bosses will buy that easier than others and depending on the structure of the organization, you may have flexibility on who you present the case to. If so, shop.

It's jst a tool.

[StrawberryFrog]

All the code that goes into your program must be certified and follow the coding rules.

But must all the tools? Is your compiler? Is your IDE? I'm not sure that all the supporting infrastructure needs to be held to the same standard as code that goes into the final build. IF a tool finds a flaw, is that flaw somehow not a flaw if the tool is not certified?

Do it like a programmer

[npgmr]

Do the necessary unit testing to make sure it fits your requirement.

Open source use not prohibited for FDA approved...

[Fejimush]

I work as a software engineer for a company that manufactures an FDA approved medical device. We are ISO certified as well (for design and manufacturing). There are no restrictions on using open source software for FDA medically approved devices. We use Linux on our embedded single board computers, several of the boost libraries, and many more open source libraries. If you have restrictions using open source in your workplace it is most likely because your ISO certified design process limits open source use

Re:Open source use not prohibited for FDA approved

[madvenu]

I fully agree with the parent comment.

And quoting from the document 'General Principles of Software Validation; Final Guidance for Industry and FDA Staff' at http://www.fda.gov/cdrh/comp/guidance/938.html#_To c517237928 [fda.gov]

The Scope section of the document refers to just this type of situation:-

'Where the software is developed by someone other than the device manufacturer (e.g., off-the-shelf software) the software developer may not be directly responsible for compliance with FDA regulations. In that case, the

Cheat as everybody does

[wysiwia]

Nobody ever followed completely all the rules of ISO9000, everybody cheats to some degree even to the level where documents and actual development doesn't have anything in common. Do be ISO9000 compliant all you need to have the necessary docs.

If you want any OSS project be ISO9000 certified all you have to do is employ somebody who creates these docs. He doesn't have to actually develop all he has to do is create all the docs. Yet none of the actual developers have to change anything since what counts are

Certifying open-source code

[Quiberon]

• Write a policy
• Implement the policy

  It's like 'how do you certify a new drug'. Taxol is an extract from yew trees; in principle an 'open-source' pharmaceutical, you discover it growing, rather than synthesise it from first principles. It's rumoured to be useful in treating some forms of human cancer.

  It doesn't save everyone; but it does extend the lives of some.

  If you go through the steps of how that got FDA approval, then you'll understand a credible way of arranging approval for "open source code wher

FDA regulations

[Paul Johnson]

I worked in a similar company, and had similar issues. The thing to understand is that the FDA inspectors are not (usually) software literate. Therefore they have to follow a "tick in the box" attitude to the CFR 820 (which if IIRC is the relevant regulation). This is very frustrating because it completely ignores all the real quality issues, and yet if you don't tick the right boxes then you can wind up in court or have your company closed down.

Don't blame the individual inspectors, BTW. They are just doing the job handed to them.

The trick here is to find a way to satisfy the regulations. Get your quality department on side here, because they are the ones who actually have to justify it to the inspector, and its their jobs on the line if they don't succeed.

As for open source, you need to get it considered under the same rules as other "COTS" software. I don't recall the details, but basically you have to come up with convincing evidence that some kind of process is properly followed. ISO9000 is the cheat sheet for proprietary software because its evidence that something of the right sort is done and audited. Open source doesn't have that, so you have to do it yourself.

Start by writing down what a good open source project looks like. Take as much as you can from ISO 9000 here. Do they have coding standards? Show they are adhered to. Configuration management? CVS. Code review? Contribution policy. Defect tracking? Bugzilla, and cite statistics showing that reported defects get fixed. Most large successful open source projects can ace this stuff anyway: if they couldn't they wouldn't be any good.

For requirements you have to get a bit more creative. Is it documented at the user level (e.g. API)? If so then you can call that documentation the requirements document and show that the source complies with it. The fact that this documentation was written after the software is irrelevant from your point of view because you only need to demonstrate that the software is fit for purpose, and that the documentation shows this. Write this argument down and it pretty much covers you.

You might also run a code review of a sample of the software to demonstrate that it meets your normal quality criteria.

Obviously this costs more than simply saying "supplier has ISO9000". But if its cheaper than buying ISO9000 then its still the right thing.

Incidentally, when I was doing this (a couple of years ago) some FDA staffer talked to our Quality Dept saying that open source was random, uncontrolled stuff that should never be allowed near real software. Basically they regurgitated a load of MS FUD. Be ready for this, and be ready for your QA dept to recite it back to you.

Paul.

Re:

[Fnkmaster]

Some creative person might think to set up a company that follows ISO-9000 procedures, take a bunch of Free/Open Source Software projects, manage a particular build or version of them, and offer them to companies as ISO-9000 certified versions of said software. They'd be exactly the same software, only accompanied by detailed documentation/audits of the code. For the service of having hired a few people to audit the software and document the hell out of it, you could charge companies all sorts of money to

self-certifying is very expensive

[rbrewer123]

The problem with doing your own audits of open source projects is that typically the open source project will have lots of extra bells and whistles and generality that your application doesn't need. It may also depend on various libraries that your application otherwise doesn't need. It may be very difficult to untangle the 20% of functionality that you actually need from the rest of the open source package. In the end, it may be easier to just write a "good enough" solution with only 80% of the feature

CFR 21 Part 11

[RUFar]

I work in the Pharma industry as well, and have some experience with validated systems - along with the associated regulations. CFR 21 Part 11 is the pertinent regulation for any and all pharmaceutical data systems - whether used for clinical trials or other cGMP purposes.

In a nutshell, it says nothing about an ISO 9000 requirement - that's SPECIFICALLY your company's policy. I understand why, it's the same deal as Sarbanes-Oxley compliance - they use ISO 9000 standard as the benchmark on which they make

Standards for Safety Critical Software

[systemeng]

I don't have FDA experience but for industrial machinery, chemical plant safety systems, and the like, the standard followed is IEC 61508. Unlike ISO 9000, IEC61508 has various hard requirements for failure probabilities and measuring said probabilities. It also has specs like the highest possible reliability than can be assumed in engineering calculations for a device and what has to be done to mitigate failures. It occupies 7 binders on my shelf and provides the basis for how to develop systems that ha

How about commercial vendors

[savio13]

You could probably find a commercial software company that supports the open source software. For example, you can get support for many Apache products (HTTPD, Tomcat, Geronimo, Axis) from several vendors (HP, IBM, Red Hat, SourceLabs, SpikeSource, etc). And I remember seeing something about SourceLabs 'Certified Software', so maybe try them? S