Okay. It's clear you haven't looked into the Phoronix Test Suite. (PTS)

The software product itself is a Test Execution Environment. The suite's value add is that it simplifies the download and execution of the actual tests and benchmarks that are executed. Those test or benchmarks are in whatever language the author of that test profile wanted to write it in.

PHP is not involved in any actual _measuring_ of performance, but is involved in the orchestration, interpretation and aggregation of the results.

Check out the version of PTS in your local convenient distribution (it's in most of the recent ones).

Feel free to contact me (I'm not posting an email address here), but it shouldn't be difficult for you to find me (visit Phoronix-Test-Suite.com or openBenchmarking.org). I'm more than happy to explore your experience and look how it can be made.

PHP has allowed Phoronix Test suite to support everything from embedded ARM systems to cloud compute infrastructure. The fact it uses PHP makes it extremely portable. Virtually anywhere that there is a compiler and PHP, Phoronix Test Suite can run. Note that the benchmarking itself is not in PHP, it's done in whatever native language the benchmark needs.

How do you know that running on an AMD doesn't affect one kernel version more than another vs Intel. The same argument stands. It's a machine layer for running code.

Sure, it's not what you want, but don't consider it completely invalid. There are many people who have interest in virtualized performance.

At the last companies I have worked at, the modern hardware doesn't run the old version of Red Hat. The result is that they run RH or other legacy versions of Linux are easier to deal with a consistent and simple hardware abstraction provided by a VM. I haven't seen a bare metal deployment of anything older RHEL 5 for a while (either that or the system it's running on is on life support).

I agree about embedded devices having older kernels - I'm regularly involved in "shiny and new" vs "old and known" discussions. Of course we're talking about PC class hardware that EOLs faster than the software. Again, my comments are focused on the enterprise use of older kernels which is a sensible interpretation of the article.

This made me laugh - in a good way, not at you :).

When Phoronix does a distro-comparison the crowd calls out that the tests are only really testing gcc differences, and should have less variables changing. When Phoronix does a fixed comparison varying only one part of the system, the crowd calls out that it isn't a good basis since people don't run it that way.

Phoronix runs tests in different ways to explore the performance landscape. For some it precisely gives the information that they need, for other it's completely irrelevant. In this particular case, I'm glad that the data gave you enough to have some open questions about 2.6.32 vs 2.6.37. If people walk away with those sorts of first order interpretation, the article served it's purpose.

Of course the next step would be how do we take a tighter look at the delta between 2.6.32 and 2.6.37 - any thoughts?

Regarding meaningful vs meaningless tests. The tests Phoronix runs are a collection of tests to explore. The tests were run, and for some of them, the results yielded nothing interesting but were still reported. You don't know until you run the tests, and if the tests are run, you report on them. Some tests may be stable now, but may have sensitivity to other parts of the systems. Even CPU bound tests will yield different results in different cases (scheduler, etc).

The "get to statistical variance" has been in Phoronix Test Suite for the better part of a year.

As part of the new work happening with Phoronix Test Suite, and the online aggregation site OpenBenchmarking.org, we'll be looking to expose the raw data and allow people to view a particular set of results in a possible more meaningful way. What is being examined now is raw data (scatter diagram), box plot (percentiles), violin plots (kernel function based), full standard error reporting (error bars, numerical reference to SD and SE.

Of course the general articles just show a simple form.

Obviously, infinite time and infinite runs with a broad variance of hardware would be better. As per usual, contact us at Phoronix with a fully baked suggestion for improvements in Phoronix Test Suite or a benchmark suggestion or article suggestion and we are more than willing to consider it.

As mentioned in a different comment. The only place where you will find older kernels in production these days will be in a VM. These are completely relevant for the people who will be running older kernels. Old hardware dies and the services migrate. Old software doesn't die, it just keeps on living in a VM.

Considering the efforts going into VM these days and the massive deployments in Fortune 500 companies, the performance of VM based systems is predictable. All the testing with Phoronix Test Suite is repeated until there is less than 3% variance between the results - or the result set is discarded.

Realistically, looking at older kernels on modern hardware is actually a very critical dimension for corporate server environments. There are applications in that space that are deployed and supported only on some old distribution. Being able to achieve and understanding how Red Hat 7.1 will act vs Red Hat 5 is critical for some environments.

If the requirements and restrictions on the Internet in China are enshrined in Law in China, you may be putting your visa at risk.

It's like a Australian 18 year old coming to the US and drinking alcohol and getting caught. In Australia, there no restriction above 18, in the US, it's 21. You get caught, you may not be able to enter the country again.

A local law is a local law, no matter what your views are. What you can do freely in your country may be illegal and carry harsh punishments in others.

Oh yes it does - look again - also look at http://slashdot.org/comments.pl?sid=1659510&cid=32286766

Phoronix Test Suite ( http://www.phoronix-test-suite.com/ ) supports Win7 now. It also allows comparison against OSX and Linux ( http://www.phoronix.com/scan.php?page=article&item=linux_windows_part3&num=1 ).

It's Free, it's Open Source and has a bucketload of tests already. You can combine result sets and you can even get the results uploaded for comparison at http://global.phoronix-test-suite.com/

Creating your own tests is nice and easy too.

(Full disclosure - I am one of the project members).

The ATI drivers support Xinerama and RANDR.

In multi-card/X Screen (:0.0) mode, you only have Xinerama - and minimal configurability.

In multi-card/multi-screen mode, you have RANDR per screen + Xinerama extensions for glass layout.

If the above sounds like some obscure language, it is. The X code is only really understanding 2 heads with lots of X developers trying to kill multiple-GPU configurations (thanks Intel). It's not pretty, but the skeleton is mostly there. Look for comment the comment below http://games.slashdot.org/comments.pl?sid=1655284&cid=32248342 for an example of a 4 multi-card/4 screen setup with 3x2 monitor layouts for each screen.

Setting that puppy up was a complete PIA!