Experts Rate Wikipedia Higher Than Non-Experts

Grooves writes "A new Wikipedia study suggests that when experts and non-experts look to assess Wikipedia for accuracy, the non-experts are harder on the free encyclopedia than the experts. The researcher had 55 graduate students and research assistants examine one Wikipedia article apiece for accuracy, some in fields they were familiar with and some not. Those in the expert group ranked their articles as generally credible, higher than those evaluated by the non-experts. One researcher said 'It may be the case that non-experts are more cynical about information outside of their field and the difference comes from a natural reaction to rate unfamiliar articles as being less credible.'" That's the problem people face when 'everyone who disagrees with you is a moron'.

Why I Doubt

[greysky]

I tend to take most things I read on Wikipedia that I'm not an expert on with a grain of salt, simply because I keep finding errors in articles that I am.

Read more carefully...

[no reason to be here]

Um, I don't think being a Biology Ph.D. candidate makes one an expert at astronomy. If you try to pull that stunt in court as an expert witness, the judge won't like it.

They're not saying that, and that's not the "stunt" they "[tried] to pull". They're saying that the biology Ph.D candidate is an expert in biology, and he, as an expert in biology, rated biology articles rather high as far as accuracy goes. He then rated astronomy articles (a field in which he isn't an expert) lower. Now, move on to the guy who is a Ph.D candidate in astronomy, and you end up with opposite results (biology articles rated lower than astronomy articles). They weren't testing grad students against non-grad students, they were testing grad-students of different disciplines against each other.

Re:it's a question of open-mindness

[heroofhyr]

I believe that the difference between the groups that this study used was not really the fact that in one group there were "experts" and in the other "non-experts", but that in one group there were "grad-students" and in the other "non-grad-students".

And I believe someone should RTFA before weighing in on it. It wasn't divided into "people who are grad students" and "people who aren't grad students," it was divided into "people who are grad students or researchers in a certain field and are given an article from Wikipedia about that field" and "people who are grad students or researchers in a certain field and are given a random article from Wikipedia's 'Random Article' link in the Navigation Menu on the front page." Or maybe we shall let the study itself explain:

A total of 258 academics (research fellows, research assistants and PhD students) were asked to participate in the study. 69 (27 percent) agreed to take part with 55 (21 percent) actually completing the survey. Each respondent was randomly assigned to one of two experimental conditions. Under Condition 1 they were asked to read an article in Wikipedia that was related to their area of expertise. For example, a member of the Fungal Biology and Genetics Research Group (in the Institute of Genetics at Nottingham University; see http://www.nottingham.ac.uk/biology/Genetics/index .phtml [nottingham.ac.uk]) was asked to look at the article on metabolites. Areas of expertise were found from the academics' own Web sites with the choice of article being made by the author. If there was any doubt the expert was contacted for advice. Under Condition 2 respondents were asked to read a random article. Wikipedia's own random article selection feature was used to assign a different article to each Condition 2 respondent. (http://firstmonday.org/issues/issue11_11/chesney/ )

It's very short, so it's not too big of an inconvenience to actually read it.

Re:The problem with this is...

[lahvak]

I agree with your first assertion. I am an expert in some areas of higher mathematics, and in my area of expertise, articles on Wikipedia are generally very accurate. there is very little noise, very few mistakes (almost all of them typos, that get quickly corrected), and occasional controversy is nearly exclusively limited to questions of notation and terminology. People who contribute to these articles generally know very well what they are talking about, and any mistakes and inaccuracies are easy to spot and easy to fix.

I think there is more to the results of this study, though. It raises good point about the nature of Wikipedia, IMHO. If I see an article in my own area of expertise, I can personally verify its correctness and accuracy. That's why I am perfectly willing to quote such articles, refer to them in discussions, and point people to them if they want to learn something about the topic.

If I, OTOH, see an article say on organic chemistry, I have no way to judge how good it is. It may very well be an excellent, completely accurate, article, however, I will never know, without actually asking an expert. All I know is that this is an article on Wikipedia, and may have been written as a prank by a high school student who has no clue about organic chemistry whatsoever. Therefore I will be very hesitant to refer to such article, and I will be very hesitant to give it high rating on correctness and accuracy.

Re:A Possible Reason

[Knuckles]

Is there a grounding for this in Mein Kampf or in speeches Hitler has made?

Hitler is by far not the only source of Nazi ideology. Other main contributors were Alfred Rosenberg, Gottfried Feder, Carl Schmitt, Karl Haushofer, Josef Goebbels, Heinrich Himmler, and many others. In general, "National Socialism" was far more complicated and ingrained into (not only) German thinking of the times than seems to be taught in US schools today (which does not make the ideology and its deeds less horrific of course.)

Re:Peer reviewed

[OctaviusIII]

Problems still arise with bias, but generally they arise in some of the less travelled articles where individuals can cut what they don't like. For example, the article on the Laurentian Leadership Centre [wikipedia.org], where I happen to be right now, was expanded upon by one of the students. Another editor simply didn't like the host school and cut it back considerably (although it looks like the proper edits are back), censoring what he didn't like and creating a bias. It's like the plagiarism thing a while back - quality decreases when traffic decreases, but that's the nature of a Wiki project, I suppose.