Fake Scientific Paper Detector 277
moon_monkey writes "Ever wondered whether a scientific paper was actually written by a robot? A new program developed by researchers at Indiana University promises to tell you one way or the other. It was actually developed in response to a prank by MIT researchers who generated a paper from random bits of text and got it accepted for a conference."
Testing... (Score:2, Interesting)
RESULTS: FAKE
Yep, it works!
Sadly, It appears that I am a robot. (Score:4, Interesting)
The nice thing is that we've finally settled the argument if machines can be made to drink beer and like it !
The program is a failure. (Score:3, Interesting)
Re:Self defeating? (Score:5, Interesting)
I recently had to check out an essay-grading robot for my Introduction to Natural Language Processing class.
I'd fed it the introduction of a randomly generated essay. It got a 4/5 on all counts.
I figure, if teachers are going to use robots to grade essays, we should use robots to create them in the first place.
Re:Yes! (Score:1, Interesting)
Novel idea.
Can fool it by duplicating first page (Score:2, Interesting)
It shouldn't be hard to compare the distribution of n-gram recurrence rates (or distances between recurrences) to the observed distribution for actual papers. Something like a KL divergence would capture deviations in either direction.
Re:It Caught Mine (Score:3, Interesting)
Re:Self defeating? (Score:2, Interesting)
This could lead to a whole series of literary robots: The Too Many Coincidences in Fiction Detector, The Humanities Thesis Verbiage Reducer, The This Movie Is Going to Suck No Matter Who Acts In/Directs It Detector, and so forth.
False positives (Score:3, Interesting)
Hmmm, it's an interesting idea, but it seems to give a lot of false positives. (So naturally, it will detect fake papers, if it thinks every paper is fake.)
First thing I tried was some pages on computational oncology website [uci.edu], in particular, my cancer primer [uci.edu], which I wrote in not a short time. Everything I fed was determined to be inauthentic. Perhaps I just write like a robot. :-) I figured that perhaps the detector was more primed for real papers, so I figured it wasn't too big of a deal.
So, next I tried my most recent research paper [sciencedirect.com], and it, too, was determined to be inauthentic, and in fact with less authenticity than my website. So much for the theory of being primed for scientific papers only. This thing is starting to look pretty bogus to me ... but an interesting idea, nonetheless. -- Paul
Re:Yes! (Score:3, Interesting)
Trying Wikipedia articles (Score:5, Interesting)
Some variant on this thing might be useful as a new article filter in Wikipedia. We need more automation over there to stem the flow of incoming dreck.