Slashdot Log In
Experts Rate Wikipedia Higher Than Non-Experts
Posted by
Zonk
on Tue Nov 28, 2006 12:57 PM
from the they-know-what-they're-talking-about dept.
from the they-know-what-they're-talking-about dept.
Grooves writes "A new Wikipedia study suggests that when experts and non-experts look to assess Wikipedia for accuracy, the non-experts are harder on the free encyclopedia than the experts. The researcher had 55 graduate students and research assistants examine one Wikipedia article apiece for accuracy, some in fields they were familiar with and some not. Those in the expert group ranked their articles as generally credible, higher than those evaluated by the non-experts. One researcher said 'It may be the case that non-experts are more cynical about information outside of their field and the difference comes from a natural reaction to rate unfamiliar articles as being less credible.'" That's the problem people face when 'everyone who disagrees with you is a moron'.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
A Possible Reason (Score:5, Insightful)
Historian A: "The Nazis were horrible awful people who killed and murdered millions of people during World War II. They created nothing but pain and suffering while seeking out total fascist control of the entire world."
Historian B: "Nazism is not a precise, theoretically grounded ideology. It consists of a loose collection of ideas and positions: extreme nationalism, racism, eugenics, totalitarianism, homophobia, anti-Semitism, anti-communism, and limits to freedom of religion."
Now the reason I put those two up there is because your average person (I'm American so I may be biased on 'average') would probably favor historian A's perspective as opposed to historian B. Historian B is actually an excerpt from the Wikipedia entry [wikipedia.org]. It's more encyclopedic as it's not opinion oriented. I'm not saying Wikipedia is free of opinions but what I'm proposing is that non-experts have an opinion and often when they read something that doesn't align with that opinion, they consider it to be incorrect.
The (on average high) neutrality of Wikipedia is most likely what causes non-experts to rate it as more erroneous than experts. Since the sample set was so low (as the report notes) then it is perhaps more likely that this happened.
I think that this is what the "Everyone who disagrees with you is a moron" article is getting at. I'm guessing experts are training not to suffer from that disease.
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
I think another issue with this is that neither the ArsTechnica NOR the actual write up actually say what pages were viewed. I think that these are VERY important questions that should be asked about this "stu
Re:A Possible Reason (Score:5, Funny)
Parent
off-by-one error invokes thread exception (Score:3, Funny)
Being the first comment, an off-by-one exception occurs, resulting in an aborted termination of the thread.
Re:off-by-one error invokes thread exception (Score:4, Funny)
Parent
Re:A Possible Reason (Score:5, Funny)
Parent
Peer reviewed (Score:5, Interesting)
However, wikipedia is different from such journals because it is a commons which is shared by people with differing viewpoints. It doesn't get the same bias that some journals may get where submitters and readers gravitate towards one of several different publications with slightly different biases (e.g. some journals favor publishing articles related to global warming as a concequence of human activities while others favor articles about it being a more natural phenomonon).
Debate is healthy, as long as it is reasoned. Wikipedia's nature enforces reason on debates about its contents. If a wikipedia entry gets edited by a person with a bias, a person with an opposing bias deals with it directly by editing the _same_ article, instead of proposing an alternate view somewhere else where it may not be seen by readers of the article. This beats the status quo , where oposing sides tend to just keep shouting their message without having any true debate.
Parent
Re:Peer reviewed (Score:4, Informative)
Parent
Re: (Score:3, Funny)
That's the stupidest thing I've ever heard, you moron
Experts qualify (Score:3, Insightful)
The less competent see fewer nuances and therefore make more straight forward assertions, they qualify their position less, therefore it looks clearer to an uninformed observer.
Re:A Possible Reason (Score:4, Interesting)
I'm pretty agnostic on the whole Global Warming debate, but it bothers me that the people who are so opposed to it argue on what they believe to be true, rather than what they think to be true. That is what you have done here. You've offered no substantial evidence to support your conclusions, rather you simply imply that all those opposed to your belief are morons.
So why are you so surprised when you are called delusional? You certainly don't offer anything to counter that impression.
Parent
Re: (Score:3, Insightful)
Like you, I consider myself agnostic on the global warming. But I'm a little confused at your response to the parent post. How does one offer "substantial evidence" when one feels that the evidence is inconclusive? What type of evidence would he offer?
The whole subject of global warming being caused by people would seem to me to fall under the heading of an "inferred best explanation," which suggests a strong probability, but falls short of being proof according to the Scientific Method [wikipedia.org].
The main p
Re: (Score:3, Informative)
Hitler is by far not the only source of Nazi ideology. Other main contributors were Alfred Rosenberg, Gottfried Feder, Carl Schmitt, Karl Haushofer, Josef Goebbels, Heinrich Himmler, and many others. In general, "National Socialism" was far more complicated and ingrained into (not only) German thinking of the times than seems to be taught in US schools today (which does not make the ideology and its deeds less horrific of course.)
Re:A Possible Reason (Score:4, Interesting)
Short answer: Yes
Long answer: HELL, Yes.
When you read Mein Kampf, you realise a) exactly how out to lunch, sick & twisted Hitler really was, and b) how out to lunch Chamberlain & the other European politicos were to even TRY to negociate with him.
Parent
Maybe Experts are just as biased (Score:3, Insightful)
Simple (Score:3, Insightful)
Re:Propoganda? (Score:5, Insightful)
Probably because wikipedia is a charitable non profit registered 501(c)(3) educational foundation [wikipedia.org] which means that it is legally obligated by both the US government and State of Florida to serve a public purpose, in this case education. While those companies that you speak of are for profit multi billion dollar corporations trying to people their products and sevices and are often lobbying the government to pass laws to make it easier to sell their stuff.
Sure anything that adds to wikipedia's reputation for accuracy will make donors feel more comfortable about donating to wikipedia. But the simple fact is that every page view on wikipedia is an expense for the Foundation, they make no money directly from their content. The best way to judge a non profit is to look at the number of people getting paid by them. And so far, the Wikimedia Foundation still seams pretty lean compared to other foundations and they are keeping their other overhead expenses reasonably low as far as I can tell.
So, yes it is good to question all studies which promote one product over another, but this simply confirms something that we might have thought anyway. That if you know more about something than others, then you are in a better position to judge the accuracy of what was written about that something.
Parent
Commonly used in IP field for prior art (Score:5, Interesting)
Good to Know (Score:4, Funny)
Re: (Score:3, Funny)
Ignorance breeds fear? (Score:4, Insightful)
Why hate wikipedia? (Score:5, Insightful)
It is free, a lot of people have put a lot of effort into it, and it is incomparable to any other repository of knowledge known to man.
Why the fuck would anyone want to piss on it? Don't like it? Shut up and go to a library.
Re: (Score:3, Interesting)
It has to do with why the "people who disagree are morons" article is wrong: if everyone could suddenly identify who the geniuses were, the not-so-geniuses would immediately kill them all out of fear, or jealously, or whatever.
Wikipedia is just a repository for information and who is informed on various subjects (whether the information is right or wrong, agr
Re: (Score:3, Insightful)
Or get yourself an Encyclopedia Britannica [amazon.com]. Only $1,100.00 new from a reseller.
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
Wikipedia is alright for cursory information, but it's not and never will be a legitimate, citable source unless it's methodology changes and it and it's supporters need to stop trying to make it seem like it can be.
To me, this is the strangest thing about the Wikipedia "debate". Critics of WP frequently claim that supporters consider it to be a "legitimate, citable source", yet I've never seen any supporters of WP say that it is, or is ever meant to be.
No encyclopedia is a legitimate, citable source. Not in any publication that matters, anyway. My kids have cited Wikipedia in elementary and junior high school papers, and WP is just fine for that, as are Britannica, World Book, etc. They're probably okay for som
I hope they didn't act on it. (Score:4, Funny)
I just hope that those non-experts didn't feel the urge to "fix" anything.
The problem with this is... (Score:4, Interesting)
BUT, when it comes to policitically charged articles (or other non-academic articles), b/c of people's "MY true is reality no matter what the facts say" mentality nowadays, the acuracy plumits.
Basically, this study is nothing but a false positive in favor of wikipedia.
Re: (Score:3, Informative)
Rawr (Score:4, Insightful)
Anyways I guess in summary people are way too afraid of the wiki model.
Re: (Score:3, Insightful)
Why I Doubt (Score:3, Informative)
Re:Why I Doubt (Score:5, Interesting)
But that's why this experiment's results are so interesting. What you're saying reminds me of how people look at mainstream media's coverage of things. It appears somewhat reasonable when they're talking about things you don't really understand, but then once they get onto a topic you know anything about, suddenly you see how full of shit they are. Your ignorance allows you to trust them, and your expertise makes you distrust them.
This study perversely suggests that Wikipedia is having an opposite effect on people, than mainstream media does.
I wonder if it has to do with what happens when people find errors in things they're familiar with. When you find errors in Wikipedia articles, do you do anything about it? With mainstream media, you can't do anything about it, but with Wikipedia, you can. Maybe you don't correct errors, but eventually someone may, and perhaps the motivation to do that, is somehow proportional to expertise.
Parent
One idea on why (Score:5, Interesting)
it's a question of open-mindness (Score:3, Insightful)
One of the things that one learns while doing his/her PhD is that he/she is NOT an expert in ANY field. It is only a matter of time for some big-headed know-it-all grad student to get crushed in a conference by a more experienced, better informed researcher. Being a grad-student and having research as your job makes you more open to new ideas and other people's opinions.
When you daily come accross many different approaches that try to solve the same problem, you are bound to learn that you must examine them all first before you decide. Otherwise you might miss a good idea that may eventually cost you your PhD. Sure you will have a favourite in the end, but that will be only after giving way to every possible option.
So a grad-student reading a Wikipedia article with an "alternative" (i.e. mistaken) point, would say "Hmm.. why not?", while a non-grad-student could say "WTF is this?" Of course, this would be the case only when the point is more close to being debatable and not obviously wrong.
Re: (Score:3, Informative)
I believe that the difference between the groups that this study used was not really the fact that in one group there were "experts" and in the other "non-experts", but that in one group there were "grad-students" and in the other "non-grad-students".
And I believe someone should RTFA before weighing in on it. It wasn't divided into "people who are grad students" and "people who aren't grad students," it was divided into "people who are grad students or researchers in a certain field and are given an article from Wikipedia about that field" and "people who are grad students or researchers in a certain field and are given a random article from Wikipedia's 'Random Article' link in the Navigation Menu on the front page." Or maybe we shall let the study it
Good for initial exposure to ideas. (Score:3, Insightful)
Personally, I find it to be a very useful resource for information on technical topics outside of my field of specialization. I do lots of modeling and conceptualization for games, so it's reeeeally nice to have an easy resource to explain the basics of say 19th century steel production or aircraft engines from the 30s. It's also really cool just to be able to read about a historical event and click a related topic to trace a thread through time. It's not a complete resource, but what is?
As It Should Be (Score:3, Interesting)
Also, note that these experts aren't necessarily saying that Wikipedia is 100% accurate or reliable. The real issue might be that where a non-expert might mistakenly disregard a large amount accurate information from Wikipedia, an expert might understand that while the majority of the information was accurate, a few important inaccuracies were also present.
"Caution ... needs to be used..." (Score:5, Interesting)
Yes, well, caution--and further research--needs to be used before citing anything learned from the Encyclopaedia Britannica... or the Handbook of Chemistry and Physics... or the World Almanac as a fact.
All of these are secondary sources. All of them are highly useful and are used as actionable sources of information every day, but none of them would be an acceptable citation in a research paper.
Furthermore, Wikipedia has always had policies that all information in Wikipedia must be derived from a published "reliable source" and that the source should be cited. Although these policies have mostly been honored in the breach, in the past year or so there has been an increasing tendency to cite sources explicitly. This is virtually a requirement for an article to become a home-page "featured article," for example. In some cases it is easier to trace the source of a fact in a Wikipedia article than in a traditional encyclopedia.
One possible reason (Score:5, Insightful)
Think it's not the problem with even science? Why do so many people attack Bjorn Lomborg with a fanatical ferocity for daring to raise scientific questions about how, why and if global warming is happening? Why can't people who claim to operate on civilized values like reason sit down and have a friendly chat. "Interesting, Bjorn, let's look at your facts; Hmmm, interesting, but I don't think you considered the following (X, Y, Z); Touche, but I would like to present this, this and that to prove that global warming is not human-caused." Instead it's more like, "YOU MOTHERFUCKING ASSHOLE WHO ARE YOU TO QUESTION ANY ASPECT OF GLOBAL WARMING?!"
The truth is that there are so many people who are significantly maleducated today that it's no wonder why people are screwed up. I mean, it was a real eye opener for me, when I started reading up on my own time, about some of the cultural practices of the ancient world. Most of the people who look horrified at religion today have never even heard of such practices as Pater Familias nor know that their celtic ancestors (if that applies to them) often practiced human sacrifice. I honestly think that based on some of the conversations I have had since I started doing these things on my own, that the maleducation of the American public today is worse than the lack of education that existed 200 years ago. There is nothing worse than having a horrendously bad education--it'd be better to simply be a void that can be filled by actual knowledge.
Now, the reason that I brought up the global warming issue was not to beat a popular pinata, but to illustrate the fact that to many "laymen," the "experts" often come off as narrow-minded fanatics. That doesn't inspire confidence in the average person. What does inspire confidence is a calm ability to articulate on his or her level with facts that back it up. Problem is, too many people have an agenda and too many people are too caught up in it to be convincing to the majority who won't immediately accept what they say at face value as though it were penned by the hand of God.
Re: (Score:3, Interesting)
I don't think the problem is not trusting it so much as not being able to critically evaluate it. I don't usually trust the general media for what I think are some very good reasons, but many people I see, mostly outside my circle of friends, seem to be quite happy to simp
Re: (Score:3, Insightful)
It doesn't matter what your credentials are, either your facts and conclusions are right or they are not. You don't agree with Lomborg? show us when and where he rushed to conclusions (in fact, some others have).
Your post is a classic example of why credentials DO matter to the 99.99% of us that aren't climate scientists. I really don't have the time to listen to people that have no training in the field they're talking about. 99% of the time people that have no training in the subject they talking about
Isn't it time Slashdot had a Wiki icon? (Score:5, Interesting)
The MediaWiki sunflower [wikimedia.org] would only be suitable as an icon for Wikis powered by that piece of software. I don't have an idea for an icon to represent all wikis.
Re: (Score:3, Interesting)
These articles could have been written by anybody. It only seems appropriate that I would be skeptical about a topic written by a less than credible source about a t0pic I know little about.
The same is true for reference books, articles, television programs, etc. That's what the references are for. I agree you should be skeptical of wikipedia articles, I'm just not sure you should be more skeptical than you are of other sources of info.
Re: (Score:3, Interesting)
Reference books and articles (in my industry at least) are peer-reviewed, if you are getting them from the major outlets. You know they are credible, or at least validated by several other PhD's in the field.
Or maybe they are corporate funded propaganda. You don't know until you check the references and see who has peer reviewed them. The exact same thing goes for Wikipedia articles. Maybe articles in some given publication are always reviewed by certain parties and you can build up a level of trust, but
Re: (Score:3, Insightful)
Oh. Sorry. Left my tinfoil hat at home.
Who needs a tinfoil hat? Have you been living in a cave? It has been standard procedure for many large companies to fund the publication of "scientific" studies for many, many years. If you aren't aware of it, the chances are you have read some unknowingly. Heck, it is not even uncommon for "news" programs to run advertisements made to look like news announcements in the middle of the news with no disclaimers.
t least I know when I submit my papers to the AIAA they
Read more carefully... (Score:3, Informative)
They're not saying that, and that's not the "stunt" they "[tried] to pull". They're saying that the biology Ph.D candidate is an expert in biology, and he, as an expert in biology, rated biology articles rather high as far as accuracy goes. He then rated astronomy articles (a field in which he isn't an expert) lower. Now, move on to the g
Re: (Score:3, Funny)