Firefox Analyzed for Bugs by Software 226
eldavojohn writes "In a brief article on CNet, a company named Coverity announced that Firefox is using software to detect flaws in Firefox's source code. Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects." An interesting tidbit from the article: "Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. The buggiest program is the Advanced Maryland Automatic Network Disk Archiver, or AMANDA, a Linux backup application first developed at the University of Maryland. Coverity found 108 bugs in its 88,950 lines of code, or about 1.214 bugs per thousand lines of code." We've covered this before, only now Firefox is actually licensing the Coverity software and using it directly.
Math (Score:5, Informative)
Re:Math (Score:5, Informative)
Re:Math (Score:2)
Re:Math (Score:3, Insightful)
Re:Math (Score:2)
Re:Math (Score:3, Informative)
Music On Console [daper.net] does everything I need :) What's more universal to a *NIX system than the console?
Re:Math (Score:2)
Re:Math (Score:3, Informative)
If this is the same (Score:3, Interesting)
Re:If this is the same (Score:2)
Re:If this is the same (Score:3, Informative)
That's true. It's impossible to have both (all bugs and no false positives, soundness and completeness), and even one of them is usually extremely expensive (computationally).
It helps to follow "good practices" and be more precautious
That's not true of Coverity (disclaimer: I work for them), we find real bugs. You can see a couple examples here [slashdot.org]. The engineers are usually the ones excited about it, once they've see
Re:If this is the same (Score:2)
Another example?
Re:If this is the same (Score:4, Informative)
Corrected link [coverity.com]. Unfortunately there are only 2 examples since there are trade secrets involved with bug reports.
This might look like a slashvertisement, but I didn't submit the original story (which does pick up on a press release)
Re:If this is the same (Score:3, Funny)
That would tend to reccomend it to me (Score:2, Insightful)
Re:That would tend to reccomend it to me (Score:2)
The point of the article was that this particular tool - not some other generic tool like it - is being used by some major projects. You are in a hole. Stop digging.
Re:That would tend to reccomend it to me (Score:2)
What utter nonsense. Coverity goes far beyond basic QA.
Re:That would tend to reccomend it to me (Score:2)
I'm afraid that distinction is reserved for this user:
BadAnalogyGuy [slashdot.org]
this slashdot news is already outdated (Score:5, Informative)
Re:this slashdot news is already outdated (Score:5, Insightful)
You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.
How does this tool compare to tools that do analysis by introspection on bytecode from languages like C# and Java. I use FxCop [gotdotnet.com] on C# code, and while it is very cool, using it is not newsworthy at all. Does this tool do more? Is is the news that it's used in a high-profile C++ program?
Integrating tools like this into your build process may be cutting-edge best-practice at present, but give it a while.
Re:this slashdot news is already outdated (Score:2)
Re:this slashdot news is already outdated (Score:5, Interesting)
You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.
Yeah, if they could make a program that would detect all bugs in a program, it would violate Turing's proof that the halting probelm is undecidable. [wikipedia.org]
From the articles, it sounds like they're basically looking for mistakes that could lead to security flaws, e.g., buffer overflows. If AMANDA is particularly buggy by their metric (detectable bugs per thousand lines of code), it's probably because AMANDA doesn't interface to the web, so the people coding it knew that certain classes of buffer overflow "bugs" wouldn't be a problem, because they wouldn't be exploited through an internet-facing interface. If you went back and ran this program on Unix apps written in C from the 1980's, you'd probably find zillions of bugs, but it wouldn't indicate low quality, it would just mean that the programs weren't written for an internet-facing environment in the year 2006, when the internet has become a battle zone for evil spammers, botnets, etc. If the only way such a bug can show up is for the user to supply carefully tailored input, and the result is simply that the program dumps core, then that's not a bug for a program that isn't facing the modern internet.
Computer != Turing machine (Score:3, Informative)
Unless the program's domain is restricted to context-sensitive languages. In fact, it is impossible for a computer to try to decide anything more general than a context-sensitive language because anything bigger requires the resources of a Turing machine, which has infinite memory. Computers implementable in a finite amount of matter are equivalent to linear bounded automata [wikipedia.org], not Turing machines.
So? (Score:2)
You say this as if it invalidates his point. Since (as you would obviously agree) no computer is more powerful than a Turing machine, if something is impossible for a Turing machine it is necessarily impossible for a computer as well. If anything, your quibble makes his argument stronger.
--MarkusQ
Halting problem on LBAs is solved (Score:2)
It does. The halting problem proof applies only to machines with infinite memory. There exists a trivial algorithm to determine whether a program halts when run on a linear bounded automaton: given a capacity of b bits and s states, run b*s*2^b steps and see if the machine has halted. This reduces an undecidable problem to an NP-hard problem. Further optimizations are possible, such as by detecting obvious cycles in the state of the machine and/or by recognizing
Solved for smallish LBAs (Score:2)
If I understand you correctly.
Suppose I have a 1Gb (=2^20) memory and some smallish number of states (say 2^8). Then we would need to run the program for 2^20 * 2^8 * (2^(2^20)) steps. This might take a while - if I were to start it running tonight on a nice teraflops machine (2^30 operations per second) it would be done in only about 2^(2^20) seconds. I'll start it running tonight and let you know when its done.
You aren't disputing anything anymore (Score:2)
From disputing his original claim:
You have been reduced to saying:
Re:So? (Score:2)
But it DOES invalidate the point made. It is possible to write a computer program that can tell me if ANY given program on any specific computer will have a bug or not. Every single one, perfectly. Do you disagree? Because you'd be completely wrong.
if Ackermann(4,3) == Ackermann(4,6) then do_bad_thing
You could write a computer program to find out if "do_bad_thing" happens, but it wouldn't be able to tell you the result, because you'd be dead.
Re:this slashdot news is already outdated (Score:2)
Local exploits (Score:2)
Of course, amanda probably run as root on "hostile" input, so local exploits can be relevant.
Re:this slashdot news is already outdated (Score:2)
Re:this slashdot news is already outdated (Score:3, Interesting)
I follow the news:linux.samba [linux.samba] Newsgroup a bit. Various Samba features have been shipped broken in various recent releases.
CIFSfs? (it is replacing smbfs and some Linux distributions have taken to disabling smbfs in the kernel to force people to switch) Cifsfs was broken in the newest major release. An intermediate release fixed that.
'Valid Users' used with 'smbpasswd': that was broken in the intermediate release. The next intermediate release will cover that.
N
Re:this slashdot news is already outdated (Score:3, Informative)
--
Cheers, Gene
Interesting... (Score:3, Interesting)
AMANDA _is_ in active development (Score:3, Informative)
Re:Interesting... (Score:2)
The current stable snapshot is amanda-2.5.0-20060424..tar.gz. And there are, sitting on my hard drive, about 12 versions of the next generation that will lead to a 2.5.1 release in a month or so.
Please put brain in gear and go check your so-called facts before spouting off in front of nearly a million
Re:Interesting... (Score:2)
As a long time amanda user, since the late 90's, which you obviously aren't and never have been, we've had far more trouble with the gnu folks screwing around in the tar srcs than we've ever had with the code for amanda proper. Am
Bug/Lines of Code (Score:5, Funny)
Sounds like someone needs to run this debugger on their calculator.
Re:Bug/Lines of Code (Score:2)
-
Once detected can they also fix the flaws? (Score:3, Funny)
Which type of bugs? (Score:3, Informative)
"Meh. So much for the 'many eyes' theory" (Score:5, Insightful)
In this age of SarbOx and risk management there is a real competitive advantage to F/OSS over proprietary code to large companies: audit-ability. In previous roles I've had to attest under HIPPA::Security that proprietary code was "secure" -- how? All I could do was obtain a vendor statement that was as non-commital and burden-shifting as possible. Yet, with a true ability to audit the code my pharmaceutical company depended on it would tilt the balance between similar-featured Closed vs Open source solutions. Especially today.
Ok, maybe nobody really cares about the 'many eyes' theory anymore. Regardless, the "open the hood" theory still applies, perhaps more than ever.
Automatic Exploitation (Score:2)
Actually, I would argue that it isn't just a freedom, it's a necessity. Having the source open means that wrongdoers can use bug-seeking programs to find exploits (presumably they have already been doing so for
Meanwhile... (Score:3, Funny)
Re:Meanwhile... (Score:3, Funny)
Re:Meanwhile... (Score:2)
Re:Meanwhile... (Score:2)
-
For those who are interested in Firefox' results (Score:5, Interesting)
Open Coverity Bugs [mozilla.org]
All Coverity Bugs [mozilla.org]
Re:For those who are interested in Firefox' result (Score:2)
The only thing that this teach me is that a language/platform that allow better typing, memory management and static analysis is far far more robust and pro
Re:For those who are interested in Firefox' result (Score:2)
Re:For those who are interested in Firefox' result (Score:2)
Mozilla developers' comments (Score:3, Informative)
Why AMANDA is buggy (Score:3, Insightful)
AMANDA (and others) have fixed the defects (Score:2)
Those that follow amanda-hackers will know that there was less than a week [yahoo.com] between when coverity released the report on March 6th and it was announced that all bugs were fixed in AMANDA on March 12th.
No rsync? (Score:3, Interesting)
In defense of amanda (Score:3, Informative)
Thats not to say that as new features are added, new bugs haven't been too, but to actually call amanda a truely buggy application does stretch this users belief a wee bit. I'm currently running a 20060424 dated snapshot of the 2.5.0 tree, with no hiccups at all.
--
Cheers, Gene
Firefox is again the most unstable program... (Score:2)
The 1.5.0.4 version of Firefox was quite stable, if the Flashblock extension was installed. The 1.5.0.6 version is unstable again. The CPU-hogging bug is back!
This comment posted from a copy of Firefox that is constantly using 2.8% of the CPU, even when all pages have been loaded, and there is no active content. That's 2.8% on the way to 70% or more, making it necessary to close Firefox and reboot Windows XP.
There are some bugs found b [mozilla.org]
The CPU hogging bug only occurs... (Score:2)
This causes lots of severe problems for heavy browser users, like equipment buyers, for example. Buyers often visit several pages, then have to wait for information, and while they are waiting, they work on buying other items.
Types of bugs (Score:3, Interesting)
It looks like most of the real bugs consist of not checking return values, the worst being routines that act upon an object allocated by another routine without checking for null pointer.
Dan East
Re:Types of bugs (Score:2)
Checking for things like this is useful, though: If you've got code in software that doesn't get executed, or a variable that never gets used, you've almost certainly got something in there you didn't intend. Since bugs are when programs don't behave the way they're intended to, code that isn't quite what you expected is a good place to look...
-F
Homeland Security Tested XMMS?! (Score:3, Insightful)
40 programs were tested. 40 open source programs. Not even all the programs installed by, or regularly used on, a default install of a particular distro or two; just 40 programs. I thought maybe these 40 were just the first 40 tested, but the original announcement of the award of the grant states that 40 programs would be tested.
And yet they didn't test BIND? ssh? Also, PostgreSQL is on the results list, but MySQL isn't? Did Homeland Security put this list together?! Using a dartboard and a list of open source applications, or what?!
This seems like a great software package, and I'm glad that Homeland Security acknowledges that "much of the critical infrastructure runs on open source", but I could think of a few other ways they could've spent $1.2 million, or at least a few other applications they should've tested before they got to XMMS.
Is this supposed to be news? (Score:2)
Hiring dozens of QA people to discover discover a broken error path that a developer could have fixed in 5 minutes is inefficant an
Coverity on Windows? (Score:4, Insightful)
Re:not just any software? (Score:3, Informative)
How does this bug detection software work anyway?
Re:not just any software? (Score:3, Informative)
Bugzilla is a issue tracking software; it's useful only after you've already found a bug. The only other bug-related tool they use is the FullCircle crash reporter thingy, again, after-the-fact thing. This is different - this tool finds flaws from the source code automatically.
Firefox is a browser (Score:2, Insightful)
And I'm assuming that they mean "Mozilla is using Coverity..." or "Firefox developers are using Coverity...". After all you don't hear about what Internet Explorer is doing, but rather what MS are doing with it.
Wouldn't it be great if the summary was clearer and neither of us had to make mental amendments?
Re:not just any software? (Score:2)
I just thought the summary could have been worded a bit better, that's all.
Re:Errr... (Score:2, Insightful)
I am sure that they know their tools limitations, but I am pretty sure that others will interpret
no outstanding bugs as if the application is secure or bugfree. Ethereal (now known as wireshark) has
a very low bug count, but I will not use it due to numerous past remote exploits coupled with
little interest in fixing bugs contra adding new features.
> Hmm, they should run their tool
Re:Errr... (Score:5, Interesting)
Re:Errr... (Score:3, Insightful)
This should be common knowledge to a good object oriented programmer, but I wonder how often it's employed in the 'C' discipline.
Re:Errr... (Score:5, Insightful)
Re:Errr... (Score:2)
Re:Errr... (Score:3, Interesting)
Re:Errr... (Score:2)
Re:Errr... (Score:2)
As an object oriented programmer, I always follow the general rule of having a function always give the same output for the same inputs.
So your objects are pure function bundles with no state? Or do you count the internal state of the object as part of the input and part of the output?
Re:Errr... (Score:2)
Re:Errr... (Score:2)
>the code with all possible inputs.
Even that won't find all the bugs. To do that you also need to know what the code is supposed to do.
I.e. You need to know in advance correct outputs for every combination of inputs. Depending on your input domain, that may or may not be impossible.
Re:Errr... (Score:2)
Re:Errr... (Score:2)
Re:Errr... (Score:5, Interesting)
not to sure.... (Score:2)
Maybe, but i'm skeptical.
Classification is just a way to hide bullshit and prevent exposure of incompetance/mismanagement/squandering/etc/etc.
Here's a bug: The program does not terminate in a finite amount of time!
Can this bug be found in polynomial time by another program?
Well, the name Turing should ring a bell.
Generally speaking, this is an unsolvable problem (in *finite* time) on almost all programming languages. [It is tr
Re:Errr... (Score:2)
|ironic mode on|
It must be bullshit when the worst bugs/codelines ratio comes from Amanda... the Advanced Maryland Automatic Network Disk Archiver, from University of Maryland
|ironic mode off|
I know, I know... just joking!
Re:Errr... (Score:2)
And if they figure out how to get the tool to modify and improve its own code we'll have Strong AI.
Re:Errr... (Score:2)
Re:Errr... (Score:2)
Re:Errr... (Score:5, Informative)
We aren't (I'm a Coverity employee). We find real bugs, and we find false positives (but not too many of those).
Hmm, they should run their tool on its own source code, that would be fun.
We do that regularly.
Re:Errr... (Score:2)
What are they teaching kids nowadays?
Re:GNAA (Score:4, Funny)
Re:GNAA (Score:2)
Re:Check the checker (Score:2)
Re:I dislike the idea of Coverity (Score:5, Insightful)
It is not possible for a program to analyze another program and find all the bugs; see halting problem .
Wrong. It is quite possible to analyze a program and find all the bugs that violate the language constraints (null pointers, buffer overflows, etc.). That's what program verification is for. For some programs, you can't tell whether a bug condition will occur, so you treat that as a bug.
Automated program verification is a good idea that went away because C and C++ have such ambiguous semantics. It's hopeless for those languages. The "pointer equals array" concept alone makes it very tough, because the language has no idea how big an array is. Worst idea in the language, and the root cause of buffer overflows.
Good verifiers were written for Pascal (I headed one of those projects [animats.com]), a good one was written for Java [dec.com] (at DEC, just before DEC went under), and Microsoft is working on one for C#. [microsoft.com]
The halting problem is not an issue (Score:5, Informative)
The halting problem is not an issue for program verification. This claim is raised repeatedly by the clueless, and it just isn't an issue.
Yes, you can construct a program that's formally undecideable. It's a hard way to write a bad program. It takes some work, and the resulting program is unlikely to be useful.
Most crash-type and security-hole problems in programs are entirely decidable. This is because almost all subscript calculations are composed from addition, multiplication by constants, and logic operations. Those are totally decideable, and there are good decision algorithms for that problem. Only when multiplication of two variables (both non-constant) is introduced can formal undecidability appear. See Presburger arithmetic [wikipedia.org].
In fact, halting is decidable for all deterministic machines with finite memory. Either you repeat a previous state, or halt within a finite number of cycles. The decision process may be made arbitrarily hard, but that's not undecidability. True undecidability in the Turing sense requires infinite memory.
Most of the practical problems with program verification come from dealing with interactions between various parts of the program. Containing those interactions well enough that you can localize problems is constraining on the programmer. "Design by contract" languages like Eiffel try to do that, but they're not popular. Retrofitting design by contract into C and C++ has been discussed, but the proposed schemes all have holes you could drive a truck through. A big truck.
Although software work seldom uses proof of correctness techniques, there's a whole industry doing it for hardware. There was a machine-generated formal proof of correctness for the FPU in AMD's K7 processor. [onr.com] AMD thus avoided the "Pentium division bug".
Re:The halting problem is not an issue (Score:2)
Yes, but no. While a computer as we know it is a finite-state machine, the number of possible states exceeds the number of atoms in the universe. In reality, your assertion that it is decideable is worthless except for contrived examples and does not solve the problem i
Re:I dislike the idea of Coverity (Score:2)
Re:I dislike the idea of Coverity (Score:5, Informative)
The fact that it is impossible to solve the whole problem of program correctness and that false positives will come up doesn't mean that the problem Coverity is adressing isn't usefull.
Regards,
That's silly (Score:2, Insightful)
What a silly reason! How about gzip etc then?
"gzip sounds like a scam. It is not possible for a program to analyze any data and always compress it successfully"[1].
I could go on: "life sounds like a scam..."
But I suggest you wake up to the harsh imperfect real world some time and leave that sort of thinking to the run-of-the-mill "academics".
How you deciding whether Coverity is good or not should
Re:I dislike the idea of Coverity (Score:2)
On your own private programs, maybe. On public OSS programs where God-knows-who is patching and forking your code, how do you know someone isn't changing your input strings, or even copy/pasting your code to somewhere where you don't have those guarantees?
If Coverity can't tell t
Re:I dislike the idea of Coverity (Score:2)
It is to identify some bugs that can be identified by scanning with software. many of those bugs obviously were not found by the human eye in the first place.
Re:So then ... (Score:2)
Re:Managed Code? (Score:2)
Java doesn't make small apps (scripts), or big apps (photoshop), or system apps (OSes, shells) very well at all. I realize that most people's bread and butter is writing silly little web applications, but I can assure you that there's a lot happening beyond the horizon. It's going to be quite a few years before Photoshop will be written in java (Eclipse is about as fruity as java gets, and even that depends heavily on native code - written in C). So please don't make