Programmers Learn to Check Code Earlier for Holes 212
Carl Bialik from WSJ writes "Many companies are teaching programmers to write safer code and test their security as software is built, not afterward, the Wall Street Journal reports. This stands in contrast to an earlier ethos to rush to beat rivals with new software, and, of course, brings tradeoffs: 'Revamping the software-development process creates a Catch 22: being more careful can mean missing deadlines.' The WSJ focuses on RIM and Herb Little, its security director, who 'uses Coverity every night to scan the code turned in by engineers. The tool sends Mr. Little an email listing potential red flags. He figures out which problems are real and tracks down each offending programmer, who has to fix the flaw before moving on. Mr. Little has also ramped up security training and requires programmers to double-check each others' code more regularly.'"
This just in: (Score:5, Funny)
Re:This just in: (Score:2, Informative)
A program has any number of execution combinations, and without a decent test-harness some paths may not be checked. If ever piece of software written was tested in every concievable scenario we wouldnt have any bugs, when that day comes I'll be a happy coder. The more 'features' one adds to a program, the problems of detecting bugs increases. Simply creating
Re:This just in: (Score:2, Funny)
Re:This just in: (Score:5, Insightful)
The trick is to get 2-3 other people to review it.
1. The earlier you spot a defect, the cheaper it is to fix.
2. Test results are only as good as the test code written.
3. Edge cases don't normally show up in test code. Test cases are typically designed to show that the code works, rather than finding the boundary where it fails.
4. You can suggest better ways of writing the code/learn new tricks during code reviews.
Re:This just in: (Score:2, Informative)
Sometimes.
But sometimes, you do spot problems. I'll often spot errors in prose I've written here prior to submitting it. It's one of the reasons I use the "Preview" button (as shocking and unconventional as that may be). I don't necessarily catch everything, but I do catch a lot.
It's the same for coding. If I go over what I've written fi
Re:This just in: (Score:2)
She finds a fair few bugs that way. Like i said, she's not a programmer, far from it.
Wow. A 'Developer' article (Score:5, Insightful)
But to stay with the topic, analysis tools are just that: tools. They are not a cure to chronic software problems. Developers are not excused from the responsibility of at least attempting to write quality code.
Some current project development methods really contribute to buggy and insecure code. Example: XP. I really think that some aspects of XP programming are a bad idea. Namely, the "code as fast as you can" aspect of it is fraught with errors. A more thoughtful, disciplined approach might seem like it is terribly slow. Yet being inherently less buggy, it can reach the target faster than the sloppier, more haphazard approach. This is much like the Tortoise and the Hare. Or maybe a better analogy would be like a rally driver who is more careful with his fuel and tires.
Don't get me wrong. Some parts of XP are fine. The Buddy System is an excellent way to get things done quickly by short-circuiting the collaboration cycle.
XP != code as fast as you can (Score:3, Insightful)
Re:Wow. A 'Developer' article (Score:3, Funny)
Yes, I believe they've pretty much got Solitaire down.
Mr little will be the most loved person there? (Score:3, Insightful)
If you want to learn someby something (I hope mr belittle does) it works much better if you have a quick feedback loop, react immediately when something is going wrong, not one weekend later when the programmer has all but forg
Re:This just in: (Score:2)
He keeps using that word. I do not think it means what he thinks it means.
static_analysis++ (Score:5, Interesting)
Static analysis is great stuff. I've worked on an open source Java static analysis tool, PMD [sf.net], for the past few years and I've gotten lots of feedback from folks who have used it to find all sorts of things in their code. Just a quick scan for unused variables can yield some excellent results, and the copy/paste detector works quite nicely too. And there's a book, too! [pmdapplied.com]
Coverity's doing a nice job with their tech marketing, too - l think a couple of open source projects are using the stuff they found to clean things up. At least, there's been a fair amount of traffic on the Ruby [ruby-lang.org] core list about some things Coverity's scan found. Good times...
Re:static_analysis++ (Score:3, Insightful)
The vast majority of the code that uses StringBuffer could save a bunch of time by using the new-ish(JDK 1.5) StringBuilder class [sun.com], which has the same API but is not
Re:static_analysis++ (Score:5, Insightful)
Unit tests don't prove your code works any more than drawing a few right angled triangles and measuring the sides proves Pythagoras' theorem. If you want to prove your ode works you use a theorem prover. To do tht you usually need to provide more detailed specification (beyond just type signatures) about how your code is intended to function. That tends to be more work, though if you really need to know your code is going to work it can often save time in the long run (over ridculously long and exhaustive testing). There are things out there that provide toold support for theorem proving aout your code: SPARK Ada [wikipedia.org] along with the SPARK tools provides a powerful theorem prover, and HasCASL [uni-bremen.de] with CASL tools (including the HOL theorem prover) provides string theorem proving for Haskell. Even ESC/Java2 [secure.ucd.ie] utilises a theorem prover (called Simplify) to provide extended static checking of Java code. I'm sure there are more examples.
My point is not that Unit testing is bad (it's very good), but that you shouldn't overstate its effectiveness. Unit tests are a great way to provide a reasonable degree of assurance that your code will hopefully ork as intended. It isn't a substitute for actual assurance however. It really depends on exactly how sure you need to be - how much an error will cost, and whether that can be tolerated.
Jedidiah,
Re:static_analysis++ (Score:2)
By "prove", I mean to present evidence to someone that will cause them to believe whatever it is that you're proving. This of course depends both on what you're proving and whom you're trying to convince. The unit test may not convince you that my code is "correct", but it will convince my boss not to fire me.
It's not all about satisfying your theorems, buddy...
Re:static_analysis++ (Score:4, Insightful)
Jedidiah.
Re:static_analysis++ (Score:2)
I'm not so sure about that. A proof is usually a series of statements that convinces somebody that something is true. Because of this there are many proofs that are wrong. If someone were to prove something it would be more likely to be correct, that is you actually proved it, if a series of formally specified steps were taken like in a declarative language such as Prolog or something of that nature. There is at least one project that is attem
Re:static_analysis++ (Score:2)
All code is mathematics applied and thus can be proven or disproven to work in the way it was intended.
Re:static_analysis++ (Score:2)
Our math friend was right to say that arguments that your code wo
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:3, Interesting)
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:2)
Sounds like a good rule for the migrating [sourceforge.net] ruleset!
Re:static_analysis++ (Score:2)
Amen to that. Apparently no one at Sun got the memo though when they designed these craptacular libraries in the first place. Saddling us with locks, locks, locks, locks everywhere we look because it assumes you're using shared state in the first place -- even if there's not a thread API method to be seen in the app.
The JVM is not slow. Java the language
Re:static_analysis++ (Score:5, Informative)
FYI, it costs about 50.000 $ for a medium sized project (500.000 lines), and is no more than a lint on steroids. Here [infoworld.com] is a somewhat cheaper competitor.
None of this tools is a mach for a manual audit performed by a professional.
Re:static_analysis++ (Score:2)
None of this tools is a mach for a manual audit performed by a professional.
At $1/line, not everyone can do this.
Re:static_analysis++ (Score:5, Insightful)
Yes it's incredibly expensive. Yet, plenty of well known companies pay for it, so I suspect it's worth it to them.
Er, no. No, no, wrong, no.
I've got access to the Coverity results for WineHQ. It's already found many problems that evaded both manual code review and unit testing. Its rate of false positives is remarkably low once properly configured. A lot of these problems would only occur in obscure circumstances or on error paths - but these are precisely the kind of errors that unit testing tends not to reveal. It can detect problems like race conditions or memory leaks that lint cannot. The recent X security bugs were revealed by the tool first.
I've seen tools like this before, but not one as good as this. I've never used competing commercial products, so cannot speak as to their effectiveness, but for a large C++ codebase I would certainly be happy to have such a tool helping me out.
Microsoft have used similar programs developed by MS Research on the Windows codebase for some time now and they're apparently very effective. Quite a lot of security problems revealed by them were silently fixed along with other problems in updates.
Totally wrong. Every patch that gets checked into Wine passes code review by at least Alexandre who is without question the best programmer I've ever met. He is easily as good as Linus but his much quieter and more conservative personality means he doesn't get Linus' press attention (a good thing, imo). And all the patches are posted to a public mailing list where several other people can and do review patches too.
Static analysis can reveal problems that simply don't get spotted by the human eye because they're too complicated to follow, because they occur in very weird situations, or because the code evolves over time under the direction of many different people and inconsistencies creep in.
Fortify also very good (Score:2)
Have not looked at Klocwork yet but it seems similar.
Re:static_analysis++ (Score:2)
Jedidiah
Re:static_analysis++ (Score:2)
A dirty little secret for Perl has been Test::Devel. As you write your tests in Perl collects stats on what has been called in your tests, and has what not. An excellent book for learning about Perl testing can be found here Perl Perl Testing: A Developer's Notebook [amazon.com]
Re:static_analysis++ (Score:2)
I hold any bet (Score:5, Insightful)
After all, that's how the software market works. People buy anything. "LOOK! THE NEW (insert program/OS name here)! I MUST HAVE IT!"
Stable?
Secure?
Mem-leak free?
In one word: FINISHED?
Who cares? It's new, it's shiny, it's been all over all the mags and preview pages, the hype is on, WANNAHAVE!
And as long as we keep buying the unfinished crap, it won't change.
Yes, I'm sure everyone in the tech departments would see this as the right way to go. Test your software, preferably during development, not afterwards. Go through memleak tests, go through stability tests, have some experienced whitehats poke at it, and if it survives, let it go into beta.
If anyone gets that idea past marketing, I will bow down to him.
Re:I hold any bet (Score:3, Interesting)
Re:I hold any bet (Score:2)
The problem is that we let companies sell software with licenses that give them all the rights, while at the same time they waive all responsibility... I don't think I've ever seen a software piece that didn't come with a disclaimer of the type "This is provided as is and we are not responsible for whatever happens
Re:I hold any bet (Score:2, Insightful)
Is it a fact that early testing will delay a project?
I must admit that I don't know much about large software development projects. But I do know a lot about large development projects in my own profession. It seems that any problem which was unresolved/ignored/insignificant during early development will turn into huge problems a few days before a deadline.
Are software projects different?
Re:I hold any bet (Score:2)
While you will never fix every single bug and not all bugs make sense to fix, this practice leaves a lot of bugs that should be fixed.
Slow and steady... (Score:2)
It's when you rush and abandon good practises that the project is in danger of becoming seriously late.
It's something everyone knows, and everyone occasionally forgets.
Re:I hold any bet (Score:2)
Re:I hold any bet (Score:2)
Seriously, I haven't bought a piece of software in years... Not that I have any commercail software, or anything. I'm all OSS at home, and at work, there is no non OSS user (read: me) installed software either.
Meanwhile, I just bought a sweet used small-form 933MHz Dell to replace my old 500MHz Dell. Not much of a step up, but lets face it: I don't use this thing for gaming.
OSS != noncommercial (Score:2)
Re:I hold any bet (Score:2, Troll)
Playing NES games on my GBA requires no recently purchased software. I have an old NES with games I bought in the late 80's/early 90's. I download the corresponding ROMs to those games, place them in PogoShell (an OSS project) and upload them to my GBA's X-ROM cart (a commercial product, but hardware) using a non-open freeware driver for it.
Now, what were you saying, Troll?
Re:I hold any bet (Score:2)
Well, I'll excuse the troll. You're not purchasing games, you're infringing on copyright. At this point you're going to quote me several quite valid justifications, but it is not the law. Whoever you downloaded it from doesn't have distribution rights, you don't have reproduction rights and in any case a copy of an illegal copy is an illegal copy.
Re:I hold any bet (Score:2)
If I'm guilty of infringement, I can't give a shit. Legal or not, I don't believe that copyright should be binding past ten years - check my past posts, I've got a record of saying five-ten should be the copyright limit, and I do live by that.
Ever read 'The Moon is a Harsh Mistress' by Robert Heinlein? In that book, the character Bernardo De La Paz states his political 'affiliation', which pre
Re:I hold any bet (Score:3, Insightful)
Unfortunately for you, what you believe doesn't matter.
"I'm sorry, officer. I don't believe that the speed limit should only be 45 on this road. I'm far enough away from the urban area, aren't I?"
Re:I hold any bet (Score:2)
fortunately for us, you don't make the laws
"I'm sorry, officer. I don't believe that the speed limit should only be 45 on this road. I'm far enough away from the urban area, aren't I?"truly insightful. i'm sure him downloading ROMs is the same as going over the speed limit, crashing into a gas station and blowing away a good chunk of road.
these analogies... please stop.
Catch 22? (Score:3, Insightful)
Alright, so writing better code means you might miss a deadline. But not writing better code means.. things are exactly as they've always been, or the software development cycle will be revamped appropriately?
Not much of a catch 22.
Or, ... (Score:2, Funny)
Re:Catch 22? (Score:2)
Re:Catch 22? (Score:2)
*stops paying attention*
Re:Catch 22? (Score:2, Insightful)
Re:Catch 22? (Score:2)
No, it isn't. [wikipedia.org]
Re:Catch 22? (Score:2)
QA is..... (Score:3, Funny)
I do this personally. (Score:3, Insightful)
I'm not sure if it really saves me any time in the long run, but I'm much more comfortable coding this way, which is probably more important.
Also, so far, I've been the only coder for my projects at work and my games at home, so it *might* not be quite as effective for large teams, although what I've read on XP seems to suggest that it can still be very effective.
Re:I do this personally. (Score:2)
Exactly. I really don't think taking more time to "code" is the answer. The majority of security problems in software come about because of design issues. The focus should be on security as a functional requirement. Sure buffer overflows and such can result from poor coding techniques, but I have found it is just a general lack of functional definition which dooms a project to a lot of revision later in the process.
Catch-22 (Score:5, Informative)
Slippery slope (Score:3, Funny)
What will the XP weenies do then?
Ain't gonna last (Score:5, Funny)
It sounds good and all but there's a direct correlation between the deadline and how bullet proof the code is.
insert sig here
Well I learned that at Uni (Score:5, Interesting)
Where did these people learn to code?
Re:Well I learned that at Uni (Score:5, Interesting)
That's why... (Score:5, Funny)
Agreed, periodic checking for holes has it's own value, but nothing beats using the best quality, industrial-strength (tm) bits to start with, moreso while developing reliable software in the post-911 world.
Re:That's why... (Score:2)
For those not in the industry sector but in information technology sector, are those the same as the Best Information Technology Strength (BITS) bits? I've used BITS bits before and they were solid and performed quite well.
The Big Unsigned Superior Technology bits are the best choice for the adult entertainment sector. They're so successful that I hear its those bits that dominate the internet and slashdotters' hard
Re:That's why... (Score:2)
I bought a box of 'Great Quality' bits from Fry's.
It only contained zeros, but they price was great!
This Just In From Microsoft (Score:5, Funny)
gets() and people (Score:4, Insightful)
How do people learn to code like this? Is it just early habits that do not go away?
Re:gets() and people (Score:2)
Rewrite the standard gets() function, and have the compiler force programs which utlize the old one to handle the new one (either by making the change transparent to existing code or by forcing the programmer to address the problem by making some minor changes).
Problematic standard library functions are traps just waiting to happen.
UNIX is too "steeped in tradition" for its own g
Re:gets() and people (Score:2)
Isn't that preferable to leaving a potential timebomb out there?
There is never "no way" in software. There might be "not acceptable" for various reasons, but that isn't the same.
Hopefully most compilers treat getc() as a deprecated function, anyway.
Re:gets() and people (Score:3, Insightful)
In my building there is a whole floor of guys doing simulations in Fortran 77. When I tell them about new functions in F90 or about ways they cou
OT: not a Catch 22 (Score:4, Insightful)
Re:OT: not a Catch 22 (Score:2)
"There was only one catch and that was Catch-22, which specified that a concern for one's safety in the face of dangers that were real and immediate was the process of a rational mind. Orr was crazy and could be grounded. All he had to do was ask; and as soon as he did, he would no longer be crazy and would have to fly more missions. Orr would be crazy to fly more missions and sane if he didn't, but if he was sane he had to fly them. If he flew them he was crazy and didn't
Re:OT: not a Catch 22 (Score:2, Insightful)
Your didn't define Catch 22, either! (Score:2)
It comes from the book of the same name wherein the only way to get the army to take you out of the war, you had to be proven crazy, but desiring to get out proved you weren't crazy, so there was no (legal) way out (without getting injured or killed in the line of duty).
--
``...like an alpaca sack full of hairy strawberry ice cream, bleeding, pink toes awry...''
Thinly veiled ad? (Score:5, Insightful)
Re:Thinly veiled ad? (Score:4, Interesting)
I'm all for tools like this. YOu can find a billion text editors on sourceforge.net but very few good programmers tools. Just this smells like an add for me.
Good at publicising themselves (Score:5, Informative)
Re:Good at publicising themselves (Score:2)
Why do companies think that is a good idea?
Deadlines are set wrong (Score:5, Insightful)
Re:Deadlines are set wrong (Score:2)
Still developing secure code is not easy (Score:2)
The problem I see is that hackers today use buffer and stack overflows. The compiler creates the insecure code more than the program.
I wonder how secure managed code in
Unit testing is your friend (Score:2)
Assuming you have a good idea of what input to your program is supposed to be, and you have an adequate method of checking to make sure the data is not some sort of goo (love those regexs!), then you should be able to test the software as you go. I'm of the school that tends to build each part, test it, and move on. It cuts down on the holes if I know where a piece of data comes from, where it's going, and what manipulations may happen to it along the way.
Obligatory Fight Club (Score:4, Funny)
Narrator: A new program written by my company is shipped on time, but with bugs. The network stack locks up. The OS crashes and burns and scrambles the hard drive. Now, should we initiate a code review? Take the number of licenses in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. If X is less than the cost of a code review, we don't do one.
Business woman on plane: Are there a lot of these kinds of bugs?
Narrator: You wouldn't believe.
Business woman on plane: Which software company do you work for?
Narrator: A major one.
Re:Obligatory Fight Club (Score:2)
No, you didn't have to do that. Without the callous disregard for loss of life, that exchange is just stupid.
Re:Obligatory Fight Club (Score:2)
Without the callous disregard for loss of life, that exchange is just stupid.
Ok then, does this help? [airliners.net]
Re:Obligatory Fight Club (Score:2)
He didn't mention planes. I know that, where I work, a crashed server isn't even a big deal.
Re:Obligatory Fight Club (Score:2)
Jedidiah.
Nothing to see here, move along (Score:2)
Now advertisements for COTS products are news articles?
While I appreciate the articles on NASA releasing code analysis tools - or pointers to freshmeat - embelishing about something I can't immediately use is boring. Procurement happens at a snails pace for purchased software - gimme something I can throw on the stage and start training dev's to use.
Or at least something in depth that shows statistics on *how much* the schedule slips by when taking this security first approach - that would be news:
"WSJ repo
Security is a Voyage Not a Destination (Score:3, Insightful)
Laws? (Score:3, Insightful)
Bullshit (Score:2)
And since the deadline is always unrealistic, all checks get down in priority as you try to keep your job hammeric code that barely works with the speed of light.
Missed deadlines = poor project management (Score:2)
As with everything in a project, adherence to security guidelines must be figured into the time estimates for a project. Time estimates must in turn be based on department-reviewed technical specs. Tech specs are based on design and development reviewed functional specs. Functi
I don't buy the idea of missed deadlines (Score:3, Insightful)
Its all in the build (Score:2, Insightful)
By telling people not to make mistakes is letting them know that they cannot try out new and inventive, sometimes even shorter ways of doing things.
Unit testing is fine and should be encouraged, but really the thing you want to do here is make your build process do all the donkey work as much as possible, and let your programmers worry about the programming issues and doing things smarter and achieve the most
Customers, refer to basic project rules: (Score:2, Informative)
valgrind tool (Score:2)
to make a valgrind tool which scans for secuity issus ?
http://valgrind.org/info/tools.html [valgrind.org]
being more careful DOES NOT mean missing deadlines (Score:2)
Failing to plan extra time to be more careful (ie. check for bugs) means missing deadlines. Only an incompetent manger, or a dishonest conman setting the team up to fail is going to insist on extra care in coding and/or code review and not put the extra time required on the plan. There's definitely a time vs quality tradeoff but it has nothing to do with missing deadlines since they should be set correctly.
pebkac programming (Score:2)
Re:A lot of presumption on their part... (Score:2)
We always had business analysts test our stuff. They knew how it was supposed to work, but more importantly: they usually didn't understand the technical underpinnings, so they didn't know what they *weren't* supposed to test. That sometimes produced interesting results.
Re:The five laws of software process. (Score:2)
Apparently they also play a heavy role in Donald Rumsfeld's job.