Catch up on stories from the past week (and beyond) at the Slashdot story archive


Forgot your password?

Improving Software Configuration Management? 33

Elvis77 writes "I am managing a project looking at some software to help us with software change management. There are numerous good applications around for doing this and our purchasing is complete, but I have been amazed during my investigations at how many organizations rely on good manners, good intentions and good luck to manage the configuration management of their organization's software assets, even in light of the Sarbanes-Oxley Act of 2002 (SOX) that affects US companies (I am in Australia). Organizations outside of the USA, without SOX implications, are usually still concerned about the quality of their software. What do my fellow Slashdot readers consider to be the best practices for configuration management?"
This discussion has been archived. No new comments can be posted.

Improving Software Configuration Management?

Comments Filter:
  • What do my fellow Slashdot readers consider to be the best practices for configuration management?

    There are way too many to list -- I've been in a number of organizations -- some had a whole 'workflow' process, where all of the managers were informed of any changes to be made a weekly meeting (and because they didn't understand the implications of all of the changes, and didn't let the tech folks under them know, it was basically useless), to ones where it's completely informal.

    I'd say the practices depen

    • And management signoff on changes is basically useless -- you get bitched at when something goes wrong, no matter how many many notes you make in change management that it's a bad idea, and it's going to break things.

      Useless insofar as not getting chewed out is concerned, but it can help you keep your job, or even keep you out of jail. Never doubt the power of good record-keeping to help point the fickle-finger-of-blame in the proper direction. On the other hand, if you're in a job where you are being un
  • First, try to buy software that logs this for you. That makes it easier to just point the auditors to the logs when they come sniffing around, and keeps your operators and system custodians from wasting time double-entering notes about what they did on some other third-party system.
  • Tripwire+CFEngine (Score:3, Informative)

    by Asgard ( 60200 ) * <> on Friday March 24, 2006 @12:51PM (#14988663) Homepage
    Combine a Tripwire []-like tool with a computer-immune-system like CFEngine [], add in a Change Control workflow that isn't too painful and you'll be in good shape. Reconcile the Tripwire reports with the change control paperwork to check that changes are being properly recorded in the workflow.
  • Start small (Score:4, Interesting)

    by Mark_Uplanguage ( 444809 ) on Friday March 24, 2006 @12:53PM (#14988684)
    If you get to the point where you realize how vital souce control is to your project(s) then I'll assume that you're already bogged down with work and deadlines. In this case start small. Just get your data into CVS or Subversion (for teams put the repository on the LAN/web/Sourceforge/etc)

    This first step will lead you to the next - increased communication in the team and good documentation. For documentation I've been please with the ReadySet [] project at Start with the basics and work up from there. Bringing a whole team along for the ride is time consuming, challenging and in the end - absolutely necessary (make it part of their objectives if you can).

    Continue from there
  • Two paths (Score:3, Insightful)

    by gclef ( 96311 ) on Friday March 24, 2006 @12:57PM (#14988714)
    There are really two paths you're talking about here, and folks tend to confuse the two:

    1) Software *development* change mangement. Meaning: tracking things like changes to software code.

    2) Software *configuration* change management. Meaning: tracking changes to the configuration of the software. I presume you're talking about this one, but it's not completely clear.

    Development change management is well served by tools of varying complexity from ClearCase to Subversion/CVS. Subversion/CVS seem to be very common, as they're free, but I've worked in offices using ClearCase before (not that anyone was terribly happy about it, though).

    Configuration change management is much harder, as you're talking about managing the configuration of applications across potentially hundreds of machines. The tools for this vary widely, depending on how hard-core you want to be. They vary from CFengine to full-on provisioning systems (openQMS, for example)...none of them tend to be easy to set up or manage, which makes them less common.
    • Good post, and I agree that CM covers so much ground as to make this question very open ended. I also agree about the less than gleeful opinion of ClearCase, which we've been forced to use. I started out with simple SCCS in Unix, which gives you all the basics of version control. But CM can entail much more than version control. It's version control, software configuration control, and instance/evironment control if you're working in an advanced or large-scale project.

      I would say that setting up some sort

  • by suso ( 153703 ) * on Friday March 24, 2006 @12:57PM (#14988718) Homepage Journal
    I would recommend using RCS for tracking changes of config files. CVS is based on it, but RCS allows you to work better with just single files instead of entire directories and sets of source codes. Just create an RCS directory in the directory that the config file is in, and do this:

    ci -u configfile

    when you want to edit:

    co -l configfile

    check it back in:

    ci -u configfile

    Easy enough.
    • Now, go make that same change on 100 servers, without disturbing the other contents of the file.

      RCS is great for single-server applications or one-off config files, but it doesn't scale really well to large applications.
    • If you are going to use version management, do not standardize on obsolete toolsets. RCS is very limited. CVS works around some of its restrictions. Both tools are fairly limited. I'm sure it was pretty cool to put your three files in RCS in the eighties but modern times require more modern tools like that. There might actually be changes in two files that are related that neither cvs nor rcs is capable of keeping track off.

      So use subversion. It's dead easy to set up, there's plenty of tool support. You can
      • So use subversion.

        I note, with satisfaction, that you fail to mention any specific benefits of using Subversion, over CVS.

        [reordered] There might actually be changes in two files that are related that neither cvs nor rcs is capable of keeping track off.

        And SVN would do that better CVS, in some magical way? I cannot see how. If you have to rack out of a change, you either have a tag or a known good date. In either case, you'd 'cvs diff' against it, review the changes, then revert all of them or cherry-p

        • Not in some magical way but by design and without scripting, I refer you the subversion book for further details. Really there's a lot in that book you clearly don't know. Basically subversion can do anything that cvs does (that was a design goal) and much more (because it couldn't be designed back into CVS, really it is quite bad). It turns out that the much more part is actually quite valuable if you care about version management. Many CVS users don't quite know what they are missing. You seem to be a goo
  • Check out NetDirector from Emu Software [].

    NetDirector does *NIX service configuration management across multiple servers on multiple Linux distros, Solaris, and BSD. And for SOX, it does change tracking and audit trailing, offers role-based management, change scheduling and rollback - all from a nifty Web-based GUI!

    I've seen it in action and it's pretty cool. They have an online demo on their site. Personally, I've always just hand-edited config files (vi is your friend), but for a multiple-server enviro

  • by $ASANY ( 705279 ) on Friday March 24, 2006 @01:35PM (#14989070) Homepage
    There's a whole other layer to CM, which is the issue tracking, requirements tracking, resource management and documentation management piece that probably is more critical than whether you can checkin/checkout changes and be able to back out a change.

    We supported a government client where issues were managed with word document templates and emails. It was a disaster, with things getting lost and weekly meetings being the only times decisions were made. They were spending a boatload of money developing something that looked suspiciously like a hobbled version of bugzilla, so we recommended and got approval to stand up bugzilla for issue tracking. It's been a big success and has been expanded to handle requirements management as well.

    The key was to set up "products" that matched operational areas for a product rather than thinking that a "product" was defined as a single deliverable. We set up the standard "product" as deliverable, with subcomponents which somewhat matched the functional areas where developer responsibilities were broken down. Then we established a "product" which was essentially an area where management issues could be handled, and had subcomponents for tester access control, requirements definition, external coordination issues and the like. So when we went into testing and found an issue that we decided to defer to a subsequent release, it was moved to the admin area and the requirements subcomponent. This kept policy and requirements control out of the hair of the developers and allowed parallel workflows for requirements, design, development and testing.

    It's not a perfect tool for all of that, but it's pretty close to "good enough", and the price is right.

    What would I add to this mix if I were God? I would wiki the work-in-process user's manual so developers could flag issues that should be addressed in documentation rather than in code. I would probably wiki and/or subversion the test plans, as word documents utterly suck for test plan management. And I'd spend some time with wiki "special pages" and bugzilla customized components to integrate it all -- linking and content sharing between a wiki and bugzilla would bring the solution set into the 90%+ range of requirements matching.

    Coders rarely are the problem in software CM. It's the management, architects, functionals and tester coordination that really has the potential to kill a project. But if you can coordinate all of them well, the flow of functional requirement - use case - design - develop - test - debug - requirement generation & traceability comes together cleanly and raises a development process into a portfolio management/enterprise architecture execution process.
  • by Grab ( 126025 ) on Friday March 24, 2006 @01:35PM (#14989072) Homepage
    The general process required is that a change can be tracked from a customer request, through the design, to a code file, and back up the V (or round the spiral) to the testing/verification/validation used to make sure that the design and code do what the customer asked for. And then that you can track all the changes that went into a particular version build, which means recording what files you used to build it.

    How do you do that? I don't care, and nor does Sarbanes-Oxley. We used to use Access databases at our place (and still do on some projects). Other projects use a tool called Visual Intercept, which I'd recommend people avoid like a rabid maneating tiger. Some projects have even used Excel (heavens preserve us!). Or simply use multiple text files (checked into your version control system of choice), one per customer request. Or Bugzilla or any number of other tools will serve your purpose. Tracking versions of files could involve some hugely complex build system, or it could just involve slapping a label across the files and doing "get all".

    But if you don't grok the basic premise that every line in your code must be attributable to a customer request, then no tool you can buy will help you. This is a process you need to follow, not a magic-bullet tool you can install and everything will be fine. Repeat after me - There Is No Quick Fix For Quality. :-)

    The most important thing you can do make sure your system works is therefore to review each change before it gets signed off to make sure changes are complete. That doesn't just mean reviewing that files are changed, but that they're all checked in under version control, AND that the change details (however you do it) are fully filled in. Even if there's automated fields to list all files and their versions in, review them anyway - there's bound to be cases where someone says "well I've got one test file listed, so that'll keep the tool happy, never mind about the other 20".

    • I will wholeheartedly agree with the parent post on many points.

      Coming from someone who was a ConfigMgmt person for my company, I faced a lot of these issues. First and formost, get a plan, even a simple one, and get it written down. Then modify it as needed. Also, Label every time you build. Any decent source control tool will allow you to do this. Be consistent with your labels, and be clear with them. This way, when a need to build version x.y.z arises, you can get back to it. Trackability is key.
  • by The Pim ( 140414 ) on Friday March 24, 2006 @01:53PM (#14989245)
    Any effort such as yours should begin with clear thinking, which is aided by purging yourself of such vague, overloaded, and literally nonsensical terms as "configuration management". In case you think this is a troll, note that the posters so far have given your message a range of completely unrelated senses. Describe precisely what you're trying to accomplish if you want useful answers.
  • Having seen a number of Bad Things happen as a result of incomplete CM processes, I can say first hand that process is the most important part of any CM strategy (BTW, this rule also holds for other practices such as systems engineering). If I had a nickel for every time I asked someone what their CM strategy was and they respond with something like "We're using tool XYZ...", I'd be living in the bahamas by now.

    It's downright foolish to believe that a tool will solve your problems. But you sound like a

    • ClearCase: IMHO, CC is not necessary unless you've got a project with a very large team and an extremely large code base (I'm thinking 1 million SLOC and up...)

      Last year a team of some 35 of us actively worked on a project based on Mozilla. The Mozilla codebase is spread through more than 3500 directories, and $DEITY only knows how many (large) files. If each directory contained an average of 1000 SLOC, that's 3.5 million SLOC. Subversion handled this with no problems whatsoever.

      Subversion gets 2 thum

    • ClearCase: IMHO, CC is not necessary unless you've got a project with a very large team and an extremely large code base (I'm thinking 1 million SLOC and up...)

      I'm working on a project of around 2.5 million lines of code with about 100 developers, and unfortunately Clearcase (which is what we're stuck with) is nowhere near up to the job - the merge tools die on merging large files, the reporting tools fall apart when the revision counts for a file get into the thousands, and the whole thing is so slow to do
  • by rossifer ( 581396 ) on Friday March 24, 2006 @02:48PM (#14989670) Journal
    It doesn't sound as though you trust the professionalism of your staff. Not that you shouldn't provide the tools they need to be effective, but I notice that you're equating SOX and top-level control with software quality, which leads me to the conclusion that you don't trust your staff. In my experience, top-level control has almost nothing to do with software quality, while trust has been highly correlated with high quality products.

    Step 1: Get the best staff you can. "10x" developers do exist, but you should be aiming for a staff of "3x" to "5x" developers of varying experience who work well together. The best developers won't really want to manage other people, but will want to be trusted with significant responsibility (i.e. they don't want to be fed detailed "specs"). Being good team members is at least as important as having top-notch skills.

    Step 2: Get out of their way. At this point, you need to make it easier for them to get their jobs done. Most software development processes are about making it harder to do the wrong thing, which inevitably makes it harder to do the right thing. For many managers, software processes are like violence: "If some isn't good enough, more must work better." Don't fall into this trap.

    Various things that will help (and not hinder) your developers:
    • Provide direct access to stakeholders with requirements, including customers if possible, domain experts, etc.
    • Fire poor performers quickly. Sooner if possible. Let the team decide who these people are (you'll hear the complaints quickly if you are receptive).
    • Get the team to agree on formatting and design conventions. This will save you more time and frustration than you know. People who insist on their own conventions are unprofessional and should have failed the criteria for being hired (they're not good team members).
    • Identify the team tie-breaker. You may use the title "team lead", "project manager", "architect", whatever. This is the person who makes the call when consensus doesn't happen quickly.
    • Get an effective issue tracking system. Bugzilla is a free and minimal option. Scarab may be ready for prime time (but wasn't a year ago when I last checked). Trac is simple but clean (and once they add the issue process enhancements... woo hoo). Rally is commercial and hosted, but very effective for agile teams.
    • Install subversion (use the file-based installation, the db installation option is still flaky). Understand and utilize subversion's features (branches and variations on branches are #1 on this list).
    • Protect the tip with a minimum of a nightly build. There are build checkers (calavista is good, there are some open source alternatives that can frequently check the status of the tip... I wish I could remember some names).

    When you get to this point, you'll have read a lot more about software development (your good developers can recommend some fantastic books) and you'll have much more precise questions.

    If you don't read at least "The Mythical Man Month" [] by Fred Brooks before managing a software team, you will fail and you won't understand why.

    • Thankyou for your response.

      We do have very good and well trusted developers and fairly good processes and systems for bug tracking and so on...

      We are a PeopleSoft site and our problem is that the complexities of releasing changes has become an issue for quality, we are concerned about developers overwriting other developer's changes because of the poor release tools we're using rather than the professionalism of the staff. We have a dedicated release team and each week we release our latest updates to t
  • [] Everything you would want to ask.
  • Hi.

    I worked in a team trying to adapt XP to their work. I cared especially about nightlies automation. About the process, what I kind of remember ( a bit fuzy though) is: yes do have a theoretical process pretty clear in mind but trying to build the real solution without an "experimenting game" with the developpers is pretty hard. We had regulars meeting in order to change the process we where following. My point: I agree with previous post, start simple, and expend. The tools : ClearCase + a bug/service_

  • I feel that making a clean well preconfigured install is the first step in configuration management. It is also crucial to your backup plan, as it relieves you from making complete system backups. This is not to say that you shouldn't be tracking your installed files via IDS, but the actual files should be already in your repository. I use debian with apt-repositories, but the general idea should be universal. This method lets me make more selective (and smaller) backups.

    I started with FAI - http://www. []
  • This is the first thing that needs to get established. I have too often seen people trying to equate these things and it's bound for failure. I've been there and I'm in Config let me tell you it's not pretty.

    Software development is one process in of itself.
    The goal of software development is to produce a stable release that is suitable to be installed on a "live" or realworld or production system. In this sense a Software Config Mgmt system keeps track of configuration items which are commonly known
  • Radmind [] leverages tripwire-like information to provide very large scale configuration management. It supports certificate-based authentication of servers and managed machines. The latest release supports compress in the network layer for cases where CPU is more plentiful than network bandwidth. :w

Who goeth a-borrowing goeth a-sorrowing. -- Thomas Tusser