Some analysis on results (Score 3)

It seems that those benchmarks have been done properly and that in those two benchmarks Windows NT performs significantly better than Linux. It would be useful to think about reasons for this.

First of all, as far as I know almost every major company has a habit of cheating in benchmark tests. For example video card drivers detect that a test is being run and enable code that skips most of the drawing primitives. This is easy to do in code that is not open source since it would take a major effort to reverse engineer the device drivers. It might be possible that NT has a feature that detects different kinds of tests and optimizes its performance accordingly (if you are for example testing throughput you would trade throughput for latency times). While this is not cheating in usual sense I think that this would be quite useless in normal mixed load situations.

The second thing is that Microsoft is quite a large company. If it wants to outperform Linux then all it needs to do is install Linux, tune it to its limits and then analyze its performance and find out weak points. Then it makes the same thing with NT. After that it just puts hundred well paint workers to make NT faster than Linux. This is made easier by the fact that if Linux works faster than NT they can just look at sources and figure out what Linux is doing better than NT. Also, it is possible that Microsoft would look at the weak points in Linux and would publish only those benchmarks where Linux performs significantly worser than NT. Anybody who does those same benchmarks would get similar results and the original benchmarks would be considered objective.

Third thing is that those benchmarks might only test peak performance - performance under high load. It is also possible that the structure of the load is untypical. This is true with most benchmarks; they rarely test systems under realistic conditions. Since I have not looked at those benchmark programs I do not know if this is the case. Anyway, peak performance is important if you want to identify bottlenecks and see what are the limits of programs. Peak performance does not tell how programs work under normal every day use.

Last thing is that I think those benchmarks are already outdated. What I would be more interested would be performance of cutting edge Linux system against similar NT system.

As a conclusion I again state that I think those benchmarks look valid. It seems that Linux kernel (and possibly also Apache) still has bottlenecks in its performance. I'm not sure if those have been fixed since this benchmark. However, I think that this benchmark should be thought of as a challenge to improve the performance of Linux. I actually think that Linux did quite well; performance differences are not THAT large when you take into account my comments above.