Essay Grading Software For Teachers 535
asjk writes "Software to help teachers with grading has been around for sometime. This is true even with respect to grading essays. A new tool, called Criteria, will look at grammar, usage, and even style and organization. It works by being trained by at least 450 essays scored by two professionals. The difference this time? Here is a snip from the article: '"There's a lot of skepticism," Dr. Spatola said. "The people opposed see it dehumanizing the student's papers, putting them through some sort of mechanical, computerized system like the multiple choice tests. That's really not the case, because we're not talking about eliminating the human element. We're making the process more efficient."'"
Interesting.. (Score:5, Insightful)
Uh.... (Score:3, Insightful)
Maybe I'm just a bit jaded by this because of all the stupid grammar and spelling nitpicking that goes on here on Slashdot. Evidentally, it's much easier to criticize my spelling than it is to provided a rebuttal to my point.
Oh goody. (Score:4, Insightful)
2 - your resume can suck, but with the proper buzz words, it'll come out looking like gold to those automated resume checkers.
1+2 = students who turn in good papers that aren't structured perfectly (and you have to admit, there is some fluidity to language) will get marked down, and those who know what bullet points to put in their papers will get good marks, even though the content is crap.
How long until you get kids selling manuals in the bathroom on what the machina are looking for?
More efficient, my ass. (Score:3, Insightful)
Then, let's feed this thing Ulysses and let's see how high it grades Joyce.
Anybody who can't see that this thing is useless for promoting any sort of creativity among students is off their rocker.
If it has flaws (Score:2, Insightful)
Hope it works well, though, and gets used as a proper checking tool.
Fine for help, but... (Score:5, Insightful)
The English language is so full of subtleties, nuances, combinations, and fantastic structural intracacies that make phenomenal writing in it possible (Faulkner, Bradbury, etc.). There's a reason English is a field of study for graduate degrees: it's absolutely worthy of them. There is no subsitute for the educated, refined judgment of someone who is exceedingly well-versed in the language.
What humanity? (Score:5, Insightful)
There is no "humanity" in a modern constructed essay. There are certainly going to be "judgement calls" when standards are not as fully fleshed out for the computer as they should be, but as long as those are appealable, I have no problem having a computer assign me the other 95% of my essay points. The only instructors who will fear this are those who like to assign grades arbitrarily. And I don't feel too sympathetic toward those people.
obDead Poets Society quote (Score:5, Insightful)
If the poem's score for perfection is plotted along the horizontal of a graph, and its importance is plotted on the vertical, then calculating the total area of the poem yields the measure of its greatness.
A sonnet by Byron may score high on the vertical, but only average on the horizontal. A Shakespearean sonnet, on the other hand, would score high both horizontally and vertically, yielding a massive total area, thereby revealing the poem to be truly great. As you proceed through the poetry in this book, practice this rating method. As your ability to evaluate poems in this matter grows, so will - so will your enjoyment and understanding of poetry.
(From the full script [impawards.com].
Removing the human factor. (Score:3, Insightful)
Actually it's about time! I don't see the essays themselves being dehumanized, but what I do look forward to is the day a middle school student doesn't receive a bad grade just because his book report was on the "Theory of Relativity" and the teacher couldn't comprehend the subject. (This is from experience) What it will do is take the human factor out of the grading process and grade all reports equally regardless of subject matter.
Comment removed (Score:3, Insightful)
perfect! (Score:2, Insightful)
Course:
College of Liberal Arts / Sci: Rhetoric 105
- or -
College of Engineering: Pattern Analysis 202
Objective:
To teach the principles of essay-writing skills. Liberal Arts students will be encouraged to follow boiler-plate styles and formats, while Engineering students will be graded on their ability to analyze and defeat pattern recognition software.
- rabs
*Shudder* (Score:4, Insightful)
The only use I can see for this thing is as a "first pass" grading tool that quickly finds obvious mistakes (spelling, grammer, redundancy, etc) and flags them for the instructor. On the other hand, it's probably just as time consuming for the instructor to read over the flagged items as it is to just catch them on the first time reading through the paper.
Re:Uh.... (Score:4, Insightful)
Essays have two aspects, spelling/grammar, and content.
Right now the computer can grade the technical side of a paper, and the teacher can grade the creative side. Now if the essay is for English class, the focus should be on the technical side of papers, so the computer can judge the whole paper from A to F on spelling and grammar.
Really it depends on the class. English classes especially in highschool are all about improving grammar and technical ability, you dont actually do any creative writing until college usually.
Style? (Score:3, Insightful)
My style isn't completely mine. I'm sure over-use would be bad. Granted this. Granted that. Where do those softer features of writing come in? Or are we all to be sterile and write with no tone or style.
Let us not forget our great achievements (Score:5, Insightful)
As far as the achievements of ancient cultures go, it is all relative. We have harnessed fusion, mapped the genome, created antibiotics, peered deep into the hearts of galaxies a 100,000,000 light years away, forged fiber optics, designed the integrated circuit, et cetera. People three hundred years from now will look back upon us and wonder how a civilization that could barely put a man on the moon (a feat that will surely be trivial to them) was able to usher in the Information Age in only a decade worth of work.
Semantics (Score:3, Insightful)
;)
This could be a good thing... (Score:2, Insightful)
Objective material is factual, a simplification is "Most dogs have 2 eyes."
Subjective material is opinionated - "Australia should legalise heroin injecting rooms." Obviously this is controversial, and there are serveral positions on the matter.
Most teachers/lecturers/graders/tutors have their own (pre-existing) subjective opinions on certain topics. If you submit an essay that opposes their views, the chances are very high that you will get a lower grade, even if your essay is well formed/written/structured.
In high school, I always took this into account and wrote essays that agreed with the teacher's point of view, even if I didn't. Such software could lessen the need for writing what they 'want to read'.
Go to a better school. (Score:3, Insightful)
You werent taught English. I'm not trying to insult you but thats one of the problems with our public schools, they dont do a good job teaching
When I went to high school 15 years ago, we didn't do any grammar in high school English class, it was all read-and-interpret (i.e. read-and-make-up-some-bullshit).
Yes and thats why when you got to college you couldnt write a good research paper.
We were supposed to learn the technical stuff in middle school (and we did to some degree).
You are supposed to learn English through highschool as well, if you want to get a 1500+ on your SATs. This is exactly why students get such low SAT scores in urban public schools, they dont get a focused education, when its time to take tests the test does not care how creative you are or even how intelligent you are, the only thing that matters to the SAT test is your technical knowledge.
Teach technical English and later on let a person learn creativity.
Teachers are getting lazy (Score:2, Insightful)
The good old fashioned teachers, on the other hand, would never do such a thing. No, they have been xeroxing the same handouts for the last two decades. You can tell by the fact that they have become half unreadable.
Homework is either trade it and grade it or credit/no credit. Major tests are all done on scantrons.
That's not to say I don't have good teachers.
But I've also had teachers who put in the bare minimum. At the end of the day, they're gone before the rest of us. They teach 5 periods (one's prep) for 180 days a year and gripe about having an average salary with mediocre benefits. On the other hand, the conservative holdouts who arrive at school before the janitors, who stay up all night meticuously grading essays, and who teach sheerly high schoolers simply because they want to (many have Ph.D's and could easily be working at the local University if they so desired) never gripe.
There has been a lot of proposals about reforming teacher pay. I don't know what can be done to attract better teachers. But sheerly for the sake of fairness I would like to see the good teachers taking a disproportionate share of the money. The trouble is, they would never ask for it.
WHY this is BULLSHIT (Score:2, Insightful)
Are essays spam for teachers? (Score:2, Insightful)
Automated is good. (Score:3, Insightful)
Automated is good because theres less chance of error, and its almost always fair.
The only way to get fair grades in university is to be smart enough to pick the right teachers, and drop the ones who you dont get along with.
I heard a statistic once that if you chose answers randomly on a MC test that you could get a C by not knowing anything beyond how to circle a letter! ----- Discovering this, I made sure that I took all the obsure english classes that had no more than 30 people in them. An unexpected positive side effect to this system of choosing courses was that 90% of the other students in them were girls. Yea, life was good.
[ Reply to This ]
Who wants a C? Thats as good as an F in college, if you get a C you can just drop the class and take it again!
I dont really like small classes myself, there is no real benefit, what I notice from smaller classes is, teachers are more critical of you, you get greater punishment for poor attendence or for being late to class, you also get more focus from the teacher and this can be good or bad depending on if the teacher likes you or not.
If the teacher likes you, getting this extra focus is a very good thing because a personal connection with a teacher who likes you is to your benefit, however if the teacher dislikes you and decides to personally focus on you, this is bad.
The rich will stay rich (Score:3, Insightful)
Human element is required. (Score:4, Insightful)
If you remove the human element, then you aren't writing for any audience, unless, of course, everyone starts writing for computers' entertainment and education.
Re:Go to a better school. (Score:3, Insightful)
You have to master (or be working hard towards) the technical aspects of something before you can have "fun" with it. Be it music, coding, or any language. If you are unable to write effectively then you will be unable to properly assert your point, which means that no matter how nifty an idea you have in your head, your reader will still understand it very well because you haven't done a good job explaining yourself.
And no, I have not been taught very good grammar myself. I suppose its pretty damned good for coming out of Kentucky though...
Re:Interesting.. (Score:5, Insightful)
I love this quote in particular because it has to be the most disingenious claim one could make. The entire act of making something a process, and then making that process more efficient IS "removing the human element". It's the type of subtle point that would be completely missed by, say, a computer grading system.
Re:Uh.... (Score:3, Insightful)
RTFA! Criteria does not merely grade spelling and grammer. Rather, it has a database of 500 papers graded by humans, and the program uses statisical analysis to compare a given paper to those in its database. If a paper uses the right technical terms, contains phrases similar to those in "A" papers, and uses phrases like "thus", "because" and "in conclusion" which suggest a logical flow, then the paper gets an A.
However, you are right that Criteria grades based on form rather than on content. As anyone who reads usenet can tell you, it is quite possible for a paper to have the form of coherent argument, to use the right buzzwords, but in fact not contain a logical or persuasive argument.
Criteria is indeed flawed, but not in the way that you suggest. Rather than check spelling and grammer, it checks for the appearance of an argument. As well all know, merely looking like a good argument isn't good enough.
Re:Go to a better school. (Score:5, Insightful)
NO!
That's the problem right there.
Highschool should be to prepare you for the real world (ie: A job, life, maybe marriage).
University is there to prepare you for a lifetime of learning on a subject.
Instead, we have employers that require university educations for secretaries. It's insane, wrong, and needs to stop if we expect everyone in society to be useful (and they ARE, it's just that stupid employers use university education as a filter).
Re:Go to a better school. (Score:2, Insightful)
Creativity comes naturally, it doesn't have to be learned.
I couldn't agree more. However, meaningful expression must be learned, and without it, creativity is less useful. Technical ability in language improves one's ability to express oneself verbally. Similarly, technical prowess in sculpting, painting, and performance aids expression in these (or any) media.
Re:Go to a better school. (Score:3, Insightful)
Things are worse than they sound... (Score:2, Insightful)
Comment removed (Score:5, Insightful)
High school essays? No creativity to lose there (Score:3, Insightful)
But when I was in high school, we were told that proper essay writing was an essential skill for the departmentals, and when they said "proper," they meant "Must conform to between five and seven paragraphs, with the first and last being this opening and conclusion with three to five paragraphs of body--each containing one topic of discussion."
Furthermore, it was made VERY clear that creative or unconventional ideas (let alone language!) would be strongly frowned upon. There was One True Way to write an essay, and One True Opinion on any given subject. Any deviations from that would cost you.
I hated it then, I hate it now, but I don't see any problem with having computers mark essays like this. After all, they were trying to turn us into computers to create them.
The Point of Grammar (Score:1, Insightful)
What this will do is reward a class of students good at memorizing rules. Most of the great writers of English would fail were their work submitted to the machine. Let's see what the system does grading Shakespeare, or Donne, or Lincoln, or George Eliot, or Joyce, or Pynchon.
Re:Automated is good. (Score:5, Insightful)
Beyond that there is no way the computer will be able to distinguish between something truly interesting and something that just lists the facts in simple Dick and Jane language with an occasional compund sentence to keep the grammar checker happy. All it can do is check for fact1, fact2, fact3, and any interesting conclusion you draw in the paper will be completely lost. Anything more would be turing test worthy, and I heartily doubt they've achieved anything close to that.
Elegant prose is often not strictly grammatical, so a boring paper would likely score the same or better than a far better written essay with the same facts. I routinely turn off grammar checking in every program I've ever used it in. Aside from the occasional misplaced modifier or dangling participle, its worthless.
In conclusion, this idea is a pipe dream which would discourage high quality writing (i.e. the kind actual PEOPLE like to read), teach people the substandard grammatical constructs used by most grammar checking software, and create a market for software that writes term papers, thereby removing the last actual bit of work your average liberal arts major has to do. I think it's a hopelessly terrible idea. TA's already do this work; why waste time coming up with a program which will do the same thing, poorly?
Just my opinion.
assumes everyone has a computer (Score:2, Insightful)
Insecure, are we? (Score:1, Insightful)
Haven't you learned anything from Deep Blue? How many problems must the AI field conquer before it's taken seriously?
Don't flatter yourself with delusions of superiority. If the computer agrees with human graders 99% of the time, then its response is definitely valid.
Besides, its not as if professors grade your work anyway. That's a job for TAs.
Overlooked point? (Score:2, Insightful)
Re:Interesting.. (Score:5, Insightful)
"Hemingway bifurcated his sensibilities between post-modernism and jazz. This I posit without having read the majority of Hemingway's work: it seemed irrelevant to the focus of my current project. What is this focus, and is it monocular? My focus can be summed up as ascertaining the usefulness of the program analyzing this document.
Without really being cognizant of the background of Freud's bisexuality, or Hemingway's sado-masochism, I cannot continue this paragraph. I will repeat this sentence without attaching any meaning to the words typed, or to my gonads. An essay in experimental dissection might be more appropriate for the issues presented here. Entirely too many bifocal wearers insist that I am currently composing gibberish. However, both Freud and Hemingway felt that bifocal wearers gloried in their bisexual sado-masochistic attachments. I concur, and I do so without reservation.
Reiteration is the root of all nonplussed renegades of origami. Nothing can be elucidated from nonsensical verbiage, but some will make the valiant effort singing praises to the whisperer. When origami is embraced by the valiant trio, the nonsensical proctologist dies. Whenever a proctologist expires in a semantic heap, Hollywood has fodder for another musical, or at least the plotline for the final unaired episode of Barney meets Fred Flintstone. Barney is a seminal reductionist. When the elucidated evidence is thrust into trusting Barney's smiling orifice, San Franciscan nuns applaud loudly.
Today I type my penultimate paragraph. I use penultimate artificially, but not without candor. Within this myriad exegesis, I pause. A Hollywood proctologist questions Freud's reasoning, and validates Barney's temporary hypothesis. In conclusion, the validity of essence cannot be lessened by the earnings of providence.
If I have not typed 500 words, this paragraph is not my penultimate, nor was my last. To assert otherwise is prudent, but lacking in elegance. What a sad commentary on misery did Darwin conspire to unfold. He rejected utterly the Hemmingway of his, and our, forebears. His eloquence was Freud and lust personified."
This earned me an overall 78% score, with no effort whatsoever. I composed this nonsense in minutes.
Doesn't this system have a baloney detector?
Fact, Fiction, and Brutal Truth (Score:3, Insightful)
Fact: Teachers would read only the introduction and conclusion paragraphs, and rely on the grading software to account for the quality of writing of the paper, and grade that way.
Brutal Truth: Teachers already read only the introduction and conclusion paragraphs, so use of this software actually would be an improvement.
two thoughts (Score:2, Insightful)
2. But here's a constructive use that would save us university faculty significant time. If the software really does grade spelling, grammar and syntax well, one could require of the student that before an essay gets handed in, it gets some high minimum score like 90% or even 98% (unless the student is dyslexic or something like that). Then we would not have to look at essays that had poor spelling, grammar and syntax, would have to do less red-inking, and would have more time to grade for content. (Which is all I grade for anyway in philosophy; though grammar, spelling, etc. get marked up but don't count unless they get in the way of my understanding.)
Re:Automated is good. (Score:3, Insightful)
Well too damn bad, this isnt about the teacher, the teachers job is to grade papers, its my job to submit paper work. What I do in between is none of the teachers business, as long as I do my job the teacher should do their job.
Uhh, NO. The teachers job IS to teach. And you're right, what you do outside of class is none of the teachers business. But what you do INSIDE class most definitely is.
Its not the teachers job to judge me as a person, its the teachers jobs to judge my work. This is exactly why we need machines, because certain people such as yourself want to judge the person and not the work.
"This person doesnt dress nice, this person has long hair and looks like a hippy, this person is always late, I dont like this person"
A teacher disliking someone on account of how they dress is an ENTIRELY different thing from disliking a student for being late and DISRESPECTFUL to both the class and the teacher. As a student your JOB is to go to class and to learn. If you can't do that, you shouldn't be at school, you should just go to frat parties, or do whatever else dropouts do.
That depends on the professor, and on the textbook. Most of the time I learn better from the textbook, mainly because I am not designed to learn in a structured environment, and because I dont learn at the pace of the class, also because rarely did I have great teachers so I'm more comfortable just looking information up myself instead of asking questions.
Really professors are coaches, I dont learn directly from them, they do sometimes make absorbing the textbook material easier when they give good lectures based on the material but most of the time, I could just read a transcript of their lecture and get the same knowledge.
Don't get me wrong, a good professor and good books compliment each other. Last semester I had an African History profesor who had lived in Zimbabwe for 5 years, and in the Southern Sudan for several years. Her first hand observations, insights, and experiences that we are able to question, analyze, and ask NEW questions of are far more valuable than being able to read "The Sudan is hot." It sounds to me like your bitch is that you haven't had any good professors? At community college? I hate to say it, but that's not really surprising.
Yes and I will drop your class if I were ever one of your students. I dont want to be judged by stupid shit that has nothing to do with my intelligence or my ability to learn. I want to be judged on my knowledge. Some people just live too far away from school, sometimes they dont have cars, other times they just overslept, either way this has absolutely nothing to do with learning and I dont see why you should have a right to punish me for my attendance if I get an A on ever paper.
You're a student, your JOB is to go to class and learn. And if you don't see the correlation between going to class and learning, or you apparently already know _everything_ why are you even going to school? Or, your _particular_ school?
Just do your fucking job and I'll do mine
You're a student, your JOB is to go to class and LEARN. Let's say you became a teacher--do you think you would get good reviews it you stumbled into class 15 minutes late every day? Or you skipped business meetings because you just want to be fucking left alone to do your work? I don't think so. Real life is about more than just saying "let me do my fucking work"