The only way this "scientific" paper could have been given a prize is because it rubs people's preconceived notions the right way: the grand-parent post is living proof of this. But scientifically, it's absolutely worthless.
Here is the paper in a nutshell: if you operate under the crazy assumption that the competence of someone has absolutely NO IMPACT WHATSOEVER on how well they will do their job when they get promoted to a higher level, then it makes no sense to promote skilled people since they won't do any better than any bottom-of-the-ladder grunt. And it's actually counter-productive: because those skilled people you promoted will get assigned a new (unrelated) random skill level, the average skill level will drop more if you promote skilled people than if you promote unskilled people...
Now these are the WTFs that come to mind:
- why on Earth did they need a crazy numerical simulation to figure that out?
- why on Earth did they not put a sensible explanation like this one anywhere? before diving into the paper the abstract and conclusions were so devoid of any insight that I was expecting something much more subtle and hard to explain than the trivial reason I outlined above
The really disappointing part is not that they have a completely unrealistic model, it's that they're trying to hide it behind fancy-looking graphs so that the science appears superficially sound. But before you call me a nay-sayer, I'll throw in some constructive criticism. Here is a simple way to analyze the problem that could have saved some computer cluster energy: Let Xi be a random variable describing the current value of employee number i. Let Yi be his value at his new job if he were to be promoted. When a new higher-level position needs to be filled, we seek to find i maximizing E(Yi | Xi=xi) + Sum(j!=i) xj. This is equivalent to maximizing E(Yi | Xi=xi) - xi. Hence, if the E(...) term depends neither on i nor on xi (as in their hypothesis), the best way is to minimize xi, hence promote the lowest-skilled person (which in the real world makes zero sense at all as they are likely to be complete newbies, difficult to work with, not giving a shit, or otherwise moronic).
So that's how you can prove their simulation result. However, you can go further: if you look at the last formula, you see that there are two terms: one term E(Yi | Xi=xi) that increases as xi increases (skilled people are good at their current job, so they are more likely to be good at their next job), and one term -xi that decreases as xi increases (it's better not to risk losing a valuable person+job combination: that's why it's well-known that being indispensable to your specific position is bad for your career as management will be reluctant to promote you). So the real job of management is to understand those two contrary goals, and balance the forces due to "skill" and to "inertia" together. You shouldn't under-estimate the first term and promote random or unskilled people, just like you shouldn't under-estimate the second term and promote the single most valuable employee blindly.
And that's where you see why using a numeric model while wearing a blindfold is a bad idea: not only is it overkill for simple phenomena like this one, but it also deprives you of a deeper understanding of the subject. Don't get me wrong, what I said in the previous paragraph wasn't all that deep: I'm pretty sure most competent managers have internalized the equilibrium, without the fancy statistic notation; but at least it goes way deeper than the paper's computer simulation. At the end of the day, a manager reading the Ig Nobel paper is going to be misled into thinking that there is proof he should try to disregard skill (or, on the contrary, that he should disregard scientific papers), while being offered no reason apart from scary computer models. True science is about enlightening people by giving them tools to understand reality: this article is about getting mainstream media coverage by giving pseudo-proof of a popular theory, with no concern for scientific honesty or a wider search for truth. If you think I'm being unfairly harsh, the authors have a webpage dedicated to media coverage of their paper, so they are clearly comfortable with their paper reaching a wide audience and didn't feel necessary to make any addition or clarification of the scope of their paper.