Forgot your password?
typodupeerror

Big Challenges for Vista Bug Hunters 213

Posted by Zonk
from the game-over-man-game-over dept.
The New York Times is reporting on the final rush to bug fix Windows Vista. Even with massive numbers of testers and five years of work behind them, the folks in Redmond are pushing it to the wire in order to make sure it releases soon. From the article: "Vista has also been tested extensively. More than half a million computer users have installed Vista test software, and 450,000 of the systems have sent crash data back to Microsoft. Such data supplements the company's own testing in a center for Office referred to as the Big Button Room, for the array of switches, lights and other apparatus that fill the space. (A similar Vista room has a less interesting name -- Windows Test Technologies.) This is where special software automatically exercises programs rapidly while looking for errors."
This discussion has been archived. No new comments can be posted.

Big Challenges for Vista Bug Hunters

Comments Filter:
  • by jonnyj (1011131) on Monday October 09, 2006 @08:03AM (#16362535)

    The real world is probably worse than the statistics suggest.

    I tried to install Vista on three PCs, all of which passed Microsoft's Ready for Vista testing tool, but all three failed before they were able to sent any crash data back to Microsoft. Two installs hung due because Vista didn't like my SATA / motherboard combination. The other got its knickers twisted over my partitioning scheme. And that was before I got a chance to find out if any of my other hardware (printer, scanner, TV card, firewire, network, graphics, CD/DVD, monitor, soundcard, suspend/resume, camera, etc) had any kind of working support.

    Recent Linux kernels work perfectly with all this hardware. Could Vista be the first Microsoft OS that lags behind Linux for hardware support?

  • by klubar (591384) on Monday October 09, 2006 @08:04AM (#16362545) Homepage
    Hardware is almost required to debug some low-level system code. Real-time stuff, like device drivers and scheduler really requires hardware tracer to determine what happened and when.

    With XP, almost all of the crashes are due to bad (usually non-MS) device drivers. If you run a system with pure MS drivers and quality hardware you'll never see a BSOD. If you run the usual business suite of software (Office, Outlook, IE) you probably never see an application crash.

    It's the crappy hardware and badly written drivers that cause the crashes. That's the difference with Apple.... since they control the hardware there's less crashes due to bad hardware and there are fewer third-party drivers for Mac hardware. The software is probably the same quality.
  • by rsidd (6328) on Monday October 09, 2006 @08:23AM (#16362689)

    Microsoft wanted a more reliable machine, improved memory management, a better filesystem, etc... Instead of throwing resources at doing these things from scratch, why didn't they just

    • Take Linux, or one of the BSDs (like Apple did)
    • Spend small amounts of money improving it (all that's really needed is improved device drivers)
    • Spend some money on improving Wine (it would be really easy for them, compared to anyone else who wants to do it), et voila -- near-perfect backward compatibility (certainly far better than Apple's MacOS 9 -> MacOS X or PowerPC->Intel moves)

    From every point of view it seems to make more sense. They spend less money, get a more reliable product that can run very nicely on existing hardware, get some good press for a change, and benefit from the work of unpaid open-source programmers all over the world. But it isn't their way.

  • by UnknowingFool (672806) on Monday October 09, 2006 @08:30AM (#16362759)

    Something that I haven't heard much recently is about Vista compatibility. MS has said before that it will be compatible and for most software and hardware, it was true in previous versions. But there were enough exceptions. ME was supposed to be backwards compatible. But many specialized drivers had to be written for it. XP definitely required some driver updates. Since Vista is a architectural change, so one would except some compatibility issues especially when DRM and enhanced security is thrown into the mix.


    Technically would MS classify incompatibility as not a a bug, especially if is does not cause a crash?

  • Big bang testing (Score:3, Interesting)

    by plopez (54068) on Monday October 09, 2006 @09:29AM (#16363251) Journal
    Sounds like typical end of project 'big bang' testing. All those issues they ignored in development? Let's fix them now and fast! I would hate to be the MS QA person.

    (Yes I am aware I used singular, it was a joke, OK?)
  • Re:special software (Score:1, Interesting)

    by Anonymous Coward on Monday October 09, 2006 @01:46PM (#16366749)
    In Office, tests are written primarily in C#, using an aspect oriented / attribute based test framework. This framework provides logging, process monitring, UI automation, methods for coordinating multi-machine tests, etc.

    The automation system is responsible for running tests autonomously, which includes imaging machines, installing the product, distributing tests to machines, ensuring that a test-gone-bad does not prevent further tess from running, analyzing and storing results, etc. The database backing the automation system has roughly 50gb of data (consisting primarily of scenario and results data).

    "Big Button" is actually a part of the test automation system which allows developers to create a change on their machine and run a batch of tests against it in a test lab. It handles collecting the changed bits, distributing them to various machines, selecting tests to run against the change, etc. One of its more notable features is that it allows developers or testers to specify failures to hold machines on in order to more readily reproduce, investigate, and fix intermittent failures.

    During any given day, this system will run roughly 100,000 test scenarios against various builds of the product.

    You laugh and joke about how it must be some archane system held together with shoestrings and bubblegum, but it is by far and wide the most advanced system of its type I've ever seen or heard of.

The world is moving so fast these days that the man who says it can't be done is generally interrupted by someone doing it. -- E. Hubbard

Working...