Journal Alioth's Journal: Microsoft: Only at SEI CMM level 1? 1
Right now, I'm almost convinced that Microsoft (in particular, the Windows division) only rates level 1 on the Software Engineering Institute's Capability Maturity Model. (A quick precis: the SEI CMM defines 5 levels - level 1 for basically the typical 'code and fix' shop, bugger design and process, and Level 5 for the sort of places that develop software for avionics - a very tight process, and software engineering that really is an engineering discipline). The Wikipedia page on the SEI CMM is here: http://en.wikipedia.org/wiki/Capability_Maturity_Model
Some background: I have worked on a SEI CMM level 3 project which was audited by external auditors (I was the development rep for the audit, too). The project was for the US Government. I joined the project as a callow youth of 23 years old, just out of college. Not long after I started, it was announced we were going to have an SEI CMM level 3 process. I'd read lots about SEI CMM at university. When confronted with the reality, I thought it'd be awful - lots of bureaucracy getting in the way of good honest coding. It turned out to be quite different (well, once I had written some tools such as my Perl automatic code inspection package generator) - and we went from a death march, 80 hour a week (40 hours unpaid) project to a project where things were always sized right, came in on budget - and importantly, it was no longer a death march. We were almost always working standard 40 hour weeks. But now I'm digressing (but this project is important in my suspicions about the Windows group in Microsoft).
Here is what the SEI CMM says about a level 1 organization (if organization is even the right word). From the Wikipedia page:
Level 1 - Initial
At maturity level 1, processes are usually ad hoc and the organization usually does not provide a stable environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes. In spite of this ad hoc, chaotic environment, maturity level 1 organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.
Maturity level 1 organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes again.
Frequently exceed schedule (and therefore budget)? This sounds so awfully familiar. Windows NT was nicknamed 'Windows Not There' because of the delays. Windows 95 nearly ended up missing 1995 altogether and was in real danger of ending up as Windows 96. Longhorn, now Vista, is several YEARS late and has had virtually all the compelling features such as WinFS dropped (see the bit about 'overcommitting' in the extract about SEI CMM level 1). Now read the utterly damning assesment of Vista from Paul Thurrott, a cheerleader for Windows (mentioned on the front page of Slashdot, but in case you missed it: http://www.winsupersite.com/reviews/winvista_5308_05.asp). Thurrot opens up with both barrels about how Vista is so disappointing with serious usability problems.
Now for Exhibit B.
In the project I mentioned at the start of this JE, a couple of years into the project I had a reputation for getting my hands dirty with the lower level stuff. Things like dealing with device drivers. So one day I found myself as the lead developer for our replacement GINA. The GINA is a DLL that is loaded by winlogon.exe, and provides things such as the user interface to allow you to log onto Windows, catches things like the Secure Attention Sequence (i.e. Ctrl-Alt-Delete for most of you), authenticates you by calling LogonUser and all that jazz. It also sets up your login environment. For reasons I won't get into, we had a requirement to write a total replacement GINA. It was to look like our actual application - the systems we were supplying were not so much PCs but appliances (which happened to be PCs). It had to authenticate and pick up the user role for the appliance, as well as log the user onto Windows in the more traditional way. Most GINA replacements are written using the example 'stub' GINA - and call functions in the Microsoft one to do the heavy lifting and to provide the GUI. Ours had to do an awful lot more. We were running into problems getting the user environment set up correctly - and the documentation supplied for writing replacement GINAs was appalingly bad - basically, just a Windows help file that got you going to write a 'stub' GINA. But no fear -- we had a US $40,000 support contract with Microsoft just in case the very eventuality happened that we got stuck on something. And we were stuck on something.
I'll give Microsoft some credit here: they didn't muck about and pass us from call centre to call centre - within about an hour of raising the call we were talking to a real live Windows developer.
Trouble is, he couldn't answer our questions. We continued to work on it independently while asking questions, and answers just were not forthcoming. It turns out they had exactly the same inadequate documentation as us. I suspect the original developers had long since left, and the developer we were talking about was having to figure it out. Since we had a head start on him from developing and testing our own GINA, we were already up to speed - and we found the answers by reverse engineering.
Now Exhibit C. Recently, a Slashdot user was discussing SMB, his experience with Microsoft developers on the protocol, Network Neighbourhood and the like. Now it'd be terribly easy to dismiss as just another Slashdotter giving Microsoft some easy stick. Well - it would have been - except his experience with Microsoft developers simply not having any documentation for a major part of Windows exactly mirrored mine.
You know all this antitrust brouhaha in the EU? About Microsoft not providing adequate documentation to third parties on protocols such as its SMB network file/print protocol? "Never ascribe to malice that which can be adequately explained by incompetence". I honestly don't think Microsoft is deliberately dragging their heels - I think Microsoft simply doesn't even have the documentation and is having to now actually read the source code and write the documentation.
Of course it can't be proven - unless you get work in the Windows group at Microsoft and see for yourself. But there are just so many indicators time and time again that Windows is only as it is through the outstanding talent of key developers - and is now seriously jeapodized by its unweildy size and total lack of any kind of software engineering process. Undoubtedly, Windows Vista will be a huge success (it's about as easy as falling off a log for Windows to be successful, given it will be installed by default on every single OEM PC) but when even someone like Paul Thurrott roundly pans Vista as a complete train wreck - years late, and missing all the important promised features - something is rotten in Redmond.
Given the suspicion that the Windows group in Microsoft is likely SEI CMM level 1, is it surprising that Windows is such a security nightmare? Is it surprising that the use of transparency in the Vista user interface is overdone like some chav covered in bling to the extent it actually makes the product less usable? Will it be any surprise when Vista is roundly exploited by spammers and DDOSers, just like Windows XP? Microsoft has massive talent and it doesn't need to be like this - but I suspect it is: and just like our initial reaction when we were told we would be SEI CMM level 3, I bet the Windows developers have held off having a proper software engineering process because it would 'spoil their fun' - and the developers, given the freedom they are at Microsoft, have probably squashed any attempt to have a proper software engineering process - and it's been to their own cost (even if they may not realise it) - and most importantly, it's been at their customer's cost.
CMMI (Score:1)
I don't take things like CMMI or ISO 9000 certs very seriously. When I worked for the prime on my current contract, they did all of this Six-Sigma training getting people in key areas trained up to help reduce costs and streamline Processes (God I hate that word). This basically ended up with the travel budget and some program management related stuff having reduced costs which made our government customer happy, resulting in high aw