Firefox Analyzed for Bugs by Software 226
eldavojohn writes "In a brief article on CNet, a company named Coverity announced that Firefox is using software to detect flaws in Firefox's source code. Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects." An interesting tidbit from the article: "Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. The buggiest program is the Advanced Maryland Automatic Network Disk Archiver, or AMANDA, a Linux backup application first developed at the University of Maryland. Coverity found 108 bugs in its 88,950 lines of code, or about 1.214 bugs per thousand lines of code." We've covered this before, only now Firefox is actually licensing the Coverity software and using it directly.
If this is the same (Score:3, Interesting)
Interesting... (Score:3, Interesting)
Re:Errr... (Score:5, Interesting)
AMANDA is cross-platform (Score:1, Interesting)
Amanda IS, however being very actively developed right now, lots of new features -> lots of new bugs. Other issue is that it's a componenty, plugin architecture, made of a few processes communicating over pipes and sockets. A failure in one component won't necessarily be a security risk or take the whole system down, it's extremely robust in normal operation in my experience, despite this "high bug count". Unlike XMMS, various contributed plugins (e.g. tape changer robot drivers) are redistributed in the source tarball but only used by very small numbers of people with outlandish hardware.
I suspect if you included various XMMS plugins in the XMMS count, things would be different...
None of that *really* excuses a high bug count - but what really pisses me off is coverity's "we've found X bugs, but we're not going to tell you what they are or substantiate our claims (some of amanda is quite old code, has a lot of strcpys, I know that some automated security checkers will treat a strcpy as a "bug" even if it's safe), just FUD your project in various public fora...
For those who are interested in Firefox' results (Score:5, Interesting)
Open Coverity Bugs [mozilla.org]
All Coverity Bugs [mozilla.org]
Re:this slashdot news is already outdated (Score:5, Interesting)
You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.
Yeah, if they could make a program that would detect all bugs in a program, it would violate Turing's proof that the halting probelm is undecidable. [wikipedia.org]
From the articles, it sounds like they're basically looking for mistakes that could lead to security flaws, e.g., buffer overflows. If AMANDA is particularly buggy by their metric (detectable bugs per thousand lines of code), it's probably because AMANDA doesn't interface to the web, so the people coding it knew that certain classes of buffer overflow "bugs" wouldn't be a problem, because they wouldn't be exploited through an internet-facing interface. If you went back and ran this program on Unix apps written in C from the 1980's, you'd probably find zillions of bugs, but it wouldn't indicate low quality, it would just mean that the programs weren't written for an internet-facing environment in the year 2006, when the internet has become a battle zone for evil spammers, botnets, etc. If the only way such a bug can show up is for the user to supply carefully tailored input, and the result is simply that the program dumps core, then that's not a bug for a program that isn't facing the modern internet.
Re:Errr... (Score:1, Interesting)
I should hope not, as that is demonstrably false. For example, at one point the KDE project with its I-don't-know-how-many-millions of lines of code had a coverity rating of 0 open bugs, but I'm sure no one is silly enough to think that such a large and complex project has no bugs at all!
Most static analysers look for very simple, easily machine-detectable, low-level imperfections which could conceivably lead to hard-to-spot bugs - not initialising a variable before it is used is probably the classic example of the kind of "bug" that would be detected by an analyser such as Coverity. I imagine Coverity is quite a lot more sophisticated than that, though :)
Re:this slashdot news is already outdated (Score:3, Interesting)
I follow the news:linux.samba [linux.samba] Newsgroup a bit. Various Samba features have been shipped broken in various recent releases.
CIFSfs? (it is replacing smbfs and some Linux distributions have taken to disabling smbfs in the kernel to force people to switch) Cifsfs was broken in the newest major release. An intermediate release fixed that.
'Valid Users' used with 'smbpasswd': that was broken in the intermediate release. The next intermediate release will cover that.
No major piece of software is ever bug free, at least the Samba guys are very responsive to error reports.
I dislike the idea of Coverity (Score:1, Interesting)
I would find heuristic analysis annoying. I'd get quite annoyed if the program says "fix this buffer overflow" 1000 times because I use "strcpy" somewhere - even though I'm very careful and only use it when I know it can't overflow.
I should write a program that searches for odd perfect numbers [wikipedia.org] and terminates if it finds one. I wonder whether Coverity would say it is an infinite loop.
Coverity sounds like scare tactics to make money by claiming to do the impossible. They won't even disclose what their algorithm is. I would never trust them, especially on closed-source programs. Firefox doesn't have that risk, but they are wasting money.
Microsoft's PREfast is simpler but seems like a much more realistic solution: mark up your code to say how things are supposed to be used and the compiler can decidably sense problems. I'd just get tired of typing 2 underscores a million times.
Melissa
No rsync? (Score:3, Interesting)
Re:Errr... (Score:5, Interesting)
Re:Errr... (Score:3, Interesting)
Types of bugs (Score:3, Interesting)
It looks like most of the real bugs consist of not checking return values, the worst being routines that act upon an object allocated by another routine without checking for null pointer.
Dan East
But how much does it cost? (Score:1, Interesting)
The less up-front anybody is about costs, the less worthwhile their product usually is. And the more variable the cost usually is (ie: as they figure out how much they can overcharge you). And no, I will not register with them for the "honor" of finding out more information. I'm guessing that it's something stupidly outrageous since the cost of running their application on a bunch of Open Source programs cost $1.2 million - which anyone with a single copy and a free weekend probably could have done for themselves.
They also don't disclose what their product actually does. So I'll join with the other voices here in calling for the need of an open-source alternative to this project - an alternative that has full disclosure about what the product is capable of and what it's going to cost you to use.