Comment Re:Statistical studies (Score 2) 139
1.What was the selection process for the studies. The phrase "All 14 of the most highly cited papers in the study" implies that there were papers not in the study. Possible selection bias?
Of course there were papers not in the study, they didn't look at every single paper ever submitted to a peer reviewed journal in all of human history. The paywall means I can't see if they explained how the 1008 got selected - well not can't, won't since my interest isn't so high as to fork over cash for it.
2. They do not go into why the 14 papers were cited so much and if any further research or refinement of the papers were done before they were accepted by other journals. Surface analysis of numbers can be manipulated to say anything.
Did you read the paper? As you said it's behind a paywall. I would hope they'd dig into the top 10 at least.
3. They also say that it might be better to not have a review and just publish everything. This just means that everyone who reads the papers has to do the review. That is not practical. There are many papers that should not be published due to shoddy practices or malfeasance. Instead of trowing out the whole system how about looking at why the 14 papers were rejected and modifying the system accordingly.
I see no such claim is that in the paywalled paper or did you mix up a random commentator and the authors?
The top papers being rejected seems like a perfectly fine system to me. The prestigious elite journals will have some risk aversion to publishing things way out of the mainstream - that's fine because there are other journals that take more risks. And according to this study the most cited articles were in fact not published in those elite journals but were published elsewhere (or else they couldn't be cited) and thus "the system" appears to work just fine.