Both Students and Teachers Use Technology to Cheat 149
Mr. Slippery writes "Baltimore City Paper's Cyberpunk column (which, incidently, is where I first learned of /.) has an interesting bit on the impact of technology on college essays - students downloading pre-written papers off the net, and professors using automated systems to grade them. Ah, the circle is complete." This story has no "news" in it, but writer Joab Jackson's take on the subject is interesting. (Disclaimer: Joab's a personal friend - and I used to write for City Paper too. - RM.)
Teachers in High-School are exempt from rules (Score:3)
The administration at my school had a huge number of draconian rules that made even breathing difficult. We were expected to follow every rule to the letter, and anything less brought down hell on us. More than once, I was reprimanded simply for expressing an opinion that differed from that of the current speaker. There was only one "truth", and the teachers had a corner on the market. I would often find loopholes in the rules to allow me to do things that I wanted to. These efforts were not appreciated. I would be punished for "violating the spirit of the law", and the rules would quickly be changed.
The teachers, likewise had a set of rules that they were supposed to live by. They were by no means as strict, and quite fewer in number. However, they were never required to follow those rules. For example, the teachers were required to write up a Disciplinary Action Form if they sent a student to the office for disciplinary reasons. However, they rarely did. More than once I claimed that I was never sent to the office and asked them to produce the forms as proof that I had been sent. They never produced these forms, but (big surprise) I was disciplined again for my challenges. Simply standing up to ridiculousness and having an opinion are important skills that are stamped out in our schools. I'm a free-thinking individual, with no use for these fascist institutions. I'm only 20 years old, but I'll swear to anyone reading right now that I will never cripple my children by subjecting them to such a limiting, brainwashing environment.
It's so ridiculous. My school's curriculum preached the values of democracy and a free society out every orifice they had, but when it really came down to it, they practiced fascism, favoritism, and contributed to an animalistic social hierarchy favoring those who did not think for themselves. Pardon my language, but give me a fucking break! Now, the schools in my area are implementing cameras in all areas of the school, including bathrooms. It's an outright violation of privacy. A student shouldn't have to put up with anything at school that his parents don't have to put up with at work. That includes abuse by teachers and students, destruction of free will, etc. If I had to put up with anything even resembling my experience in high school at a job, I'd walk out the door in an instant. I swear, if I could fix this, I'd do it right now. It's one of the biggest tragedies in this country.
If you need to point-and-click to administer a machine,
Symptom of too high teacher-to-student ratio? (Score:2)
This is kind of funny in a twisted sort of way - I wonder if things were allowed to continue, would there come a mass academic realization that nobody human has been generating or grading papers for years...
I would hazard a guess that we'll end up coming full circle to oral tests (or at least some kind of test involving interaction between the instructor & each student). Of course, this would require more teachers per student - but at least the teacher wouldn't be "required" to take work home with them.
:rant mode on:
I'm really angry about the current state of the US educational system. (Bias warning: my mother teaches learning-disabled kids from grades 2-5).
Both the top-level decision makers & US society seem to be more interested in paying lip-service to education than supporting it as an opportunity equalizer. I've seem _individuals_ worried about the US education system, but there seems to be a system-wide lack of respect for the importance of education for a healthy society.
The amount of work & training that "good" teachers put themselves through is easily the equal of anything I've ever done in my professional career (and I'm a 10-hour/day workaholic), but after 30+ years of service, they get paid 1/2 of what I"m getting now. As far as I'm concerned, these "good" teachers ought to be getting at least 6-figure salaries & the equivalent respect of any other professional. And they don't just have to work hard - they have to be _SKILLED_ (anybody who doesn't think this, I invite to try and "debug" a class full of 7 year-olds like they debug a set of programs or a piece of hardware)...
There ARE "bad" teachers, but I see this as more of a symptom of the lack of respect that educators get in the US society - with the proper compensation & respect, you'd get top-quality professionals, just like any other field.
Unfortunately, all I see now-a-days is how to make teachers work harder with less money, complaints about how teachers aren't "doing their job" (usually associated with people who want to make their kids aren't "contaminated" by the hoipolloi in the public education system), complaints about too-high taxes (which are lower than just about any other 1st-World country), and skyrocketing corporate subsidies (with skyrocketing corporate profits).
:rant mode off:
And to think they will inherit the earth... (Score:1)
Re:stupidity of mechanical grading (Score:1)
As to a human grader not giving it above a zero- bullshit
Thanks,
LetterRip
Re:Freud (Score:1)
It's amazing how many people who claim to be scientists don't follow the scientific method at all. Like, simple pedantic stuff like evidence.
Re:A Perplexing Problem (Score:1)
Anyway, I've never understood literary criticism, but if it did have a point it would surely be nowhere near maths, and thus pointless to measure with a test.
But I don't believe in IQ tests. Though IQ tests are something like maths, they don't actually measure anything tangible other than the ability to pass IQ tests. Intelligence shouldn't be confused with this or any other specific ability. Intelligence is non-algorithmic, a very profound truth which Roger Penrose shows beyond reasonable doubt in his very rigorously argued "Shadows of the Mind".
Re:A Perplexing Problem (Score:1)
I once tried to get a computing teacher sacked who literally read every lesson from a book and literally never knew the answer to any significant computing question. Unfortunately the dept head said "Nothing I can do". It would have been hilarious if it weren't so sad. Mind you, it also demonstrated that most teachers make less difference than they'd like to believe - while the results weren't good, they weren't disastrous either.
Re:Is this a joke? (Score:1)
It really says that as well!
For goodness sakes don't anyone email them. Just let it stay there for ages. What a sad reflection on their site! :-)
John Holt (Score:1)
College IS response? (Score:1)
If you're responsible for the network at a college, do you:
If you do either of these, what justifications will you give when the student body eventually calls for your head? Discuss.
Re:I invented the Intelligent Essay Assessor... (Score:1)
Re:A Perplexing Problem (Score:1)
I had a teacher who marked like a computer. (Score:1)
I think this sort of attitude makes computer grading an attractive option.
Education today vs. yesterday (Score:1)
In generations past, to be successful, Johnny had to finish high school. When I was in high shool, to be successful meant having to get a bachelor's degree. Now, a bachelor's degree doesn't mean as much and a master's or doctorate degree is required.
This story proves that.
The only question left is when will writing a thesis be this easy ??
Pretty common (Score:2)
If someone wants to write an article, they can either get their for-pay login to Lexix-Nexus and suck down a couple of pre-written articles to regurgitate as an uninspired rehash of data found elsewhere.
Or they can hop on the web for free (minus ISP connect charges etc.) and check out Slashdot, ZDNet, C|Net, PCweek, EETimes, Salon, or any of ther other "bigger" online 'zines to get pre-written articles to regurgitate as an uninspired rehash of data found elsewhere.
The people read them anyway, probably because they've been educated in the kind of environment where they've learned that the best kind of learning is to suck some data from another source or two and regurgitate it for the teacher in an uninspired essay of rehashed material.
The few percent of people who don't go for that kind of stuff are going to enjoy doing the research and the writing, or at least learn something from it on their own, despite any bad grades they might receive from a "grading machine" and either learn to regurgitate uninspired essays until they get out of school and do something they enjoy, or just drop out and use their stunning intellects to RULE THE WORLD!! (Err, sorry. I'm getting carried away. But you get the idea.)
-=-=-=-=-
The next project is then a CS assignment - (Score:1)
Incidentally, does anybody know if the grading software is subject-specific, i.e. can evaluate that essay FOO is a good essay on Macbeth rather than a good essay on British colonialism?
Yonatan
More on them there right answers (Score:1)
-Laktar, a.k.a. Nick Rosen, laktar.dyndns.org
If I Ever Became An Evil Overlord:
83. If I'm eating dinner with the hero, put poison in his goblet, then have to
leave the table for any reason, I will order new drinks for both of us instead
of trying to decide whether or not to switch with him.
-- Peter's Evil Overlord List, http://www.eviloverlord.com/lists/overlord.html
Re: (Score:2)
No substitute for human thinking... (Score:1)
He includes in-class writing assignments as well as outside assignments and reads every word from every student. If you care about your students, it would be pretty easy to spot a major difference in quality and style between inside and outside assignments.
In the end though, some students are determined to NOT get an education, no matter how much mommy and daddy are paying. And they will find a way, which may be the only thing they really learn.
Forrest Cavalier
The Reuse Rocket [mibsoftware.com] more than 6000 open source apps, functions and libraries, PLUS the FAQs, info, and references you need to use them.
THIS IS SO GREAT! (Score:1)
A comment above said "Why assign it at all?!". That is totally the correct attitude! Education in this country SUCKS. We take all these STOOPIT General Ed courses just so we can have the PLEASURE of taking specific courses for our career. Gee thanks... could I waste MORE Of my time?!
I'm glad to see the lameness of our education system being beaten by techno smart kids. Reading books like The Scarlet Letter is an utter waste of time. Rent the damn movie! If it's the story that's important you can get that in a 2 hour movie. EVERY "GREAT" piece of literature has been made into a movie. And with the exception of DUNE they probably do a pretty good job of getting their point across. Why the hell do students need to waste time on critical thinking about some puritans with their panties too tight!? Uhmm... HELLO imagine how incredible a programmer you'd be if you spent your entire K-12 education and then your college years focused ONLY on programming, or physics, or whatever your interested in.
Stop government waste, get rid of english teachers.
Re:Big Brother (Score:1)
No, this was years ago. I no longer have the source...
---
This wasn't even new a long time ago. (Score:1)
This is really scary... (Score:2)
I would never copy an essay straight off the net (I can usually produce better work on my own) but the thought that a prof can use a computer to mark it, and never actually read my work frightens me...
What if there is some brilliant idea that is lost because the computer doesn't recognize creativity... only form and structure.
Re:This wasn't even new a long time ago. (Score:1)
And there are automatic source code checkers (besides compilers, of course) too, they use them for some CS courses at my university. =)
Break out the pens and pencils (Score:1)
Granted, it might well force schools and teachers to re-think the idea behind many of the papers that are assigned. Give it more thought and change the core assignments of the class from semmester to semmester and this might give us a better learning experience then a lot of schools are giving now.
Custom essays? (Score:2)
On the other hand, who *writes* these custom papers? Odds are, there aren't that many people who'd be interested and clued enough to do such things as use specifics from the course (such as rare primary sources cited, or previous in-class discussion), that'd actually do a halfway decent job -- particularly if you're trying something obscure. Papers on, for instance, the ability, or possible lack thereof, of BGP4 routers to automatically recover from the theoretical instance of a maliciously configured router might be a tad less usual than a trite discussion on the evolution of western fortification technology during and following the Crusades.
What next? (Score:1)
get drunk and fall down at parties for me
while I catch up on much needed sleep.
Re:Johnny, what did you learn today? (RANT) (Score:3)
As I've posted in the past, overspecialization and the believe that one doesn't need to have any knowledge outside one's own profession does a lot to contribute to this. Few people are left who see the value of a good, well-rounded education. Whether it's the kid who majors in business or CS to "make a lot of money," or the classics major who thinks that communication majors "sit around watching I Love Lucy re-runs all day," or the hard science major who has never voluntarily cracked a book of poetry in his life, or the humanities/social science majors who take watered-down math and science classes so they can keep their high GPAs with a minimum of effort, or
Now I realize why the deja vu... (Score:2)
So true.... (Score:1)
Nick
Exactamundo (Score:1)
Like a prior post said - break out the pens and pencils
Re:How does this really evaluate papers? (Score:3)
It's like having humans read resumes instead of computers searching for buzzwords. Seems like a good idea, but unsavory changes might result.
Re:Big Brother (Score:1)
Good grading algorithms :) (Score:2)
My favorite professor had a worksheet he stapled to the back of each paper. Each element of the paper (relevance of topic, coherence of argument, adherence to the assignment -- things like page length and use of APA style, and spelling/grammar) were given a certain amount of points. He would put down exactly where you "lost" the points, with comments. (I usually lost a point on bad APA style because I prefer MLA. But oh well.) Then again, this is the same professor who would always tell us "I have no problem giving everyone an A. I also have no problem giving everyone an F. You control your destiny in this class."
HAHAHA -- just like Fredrich Pohl's "Midas World" (Score:1)
Gpod satire. Good science fiction.
["Another Dave Langford review." [demon.co.uk]] [Amazo n [amazon.com]]
Teachers these days are often students (Score:1)
As a former graduate student and "Teaching Assistant" this article came as no surprise to me.
When I began my first year at the University of deleted, I was informed approximately two weeks prior to the start of classes that I would be teaching a section of Freshman English. Never mind the fact that my background was in Linguistics, not English, or that I had not one day of teaching experience. Never mind also that I had previuosly been told that I would have no more than 10 hours per week of work to do in order to receive a funding to continue my studies. Upon complaining to my advisor that this was probably not such a good thing for me or my potential students, I was told that although he agreed with me, there was not other way around the situation because funding was so limited. Since the university was paying my tuition in exchange for this job (and believe me, I was thankful for that), I decided to make the best of it and try to learn from the experience.
Many of my fellow English teachers that year were first timers like me. It would probably frighten many of you to know how truly underprepared many of them were. Some of them were so scared that they cancelled several classes throughout the semester, while others dodged the responsibility by showing lots of films. Some of the teachers were really scared of their students and would give them "academic candy" by letting them read junk articles and novels.
In spite of being pretty nervious for the first few weeks I actually enjoyed the teaching experience. I learned that high school seniors are often getting a much better education than they are typically believed to. Many of my students were bright, witty, and accomplished writers. Not all of them were super-motivated academic acheivers, but you can't expect people to be too excited about Freshman English anyway. I had a few students hand in papers with "sloppy quoting," i.e., plaigerism, but it was probably more out of fear ignorance than a desire to cheat. A few of my students were poorly prepared academically, but that's always the case. The vast majority of the students though, made a good effort throughout the year and to the best of my knowledge did not download papers off the internet. You can tell if someone consistently makes the same good kind of comments or the same mistakes that they are probably writing the papers for themself. As long as students feel that their instructor cares about what he or she does, and treats the students with some respect, they probably will feel confident enough to try out their own writing and not cheat.
Anyway, what really does not surprise me about this article is that teachers would use an electronic grading machine. The reason I am not surprised is because I dropped out of grad school at the end of the year. I enjoyed teaching, but I could not handle the pressure to keep up with my own classes (12 graduate credits), grade 21 6 or 7 page papers every other week, read articles and novels, and prepare interesting lectures twice a week. I felt that I had to cut back on one thing every week: one week I would do a good job with the lectures and leave off the rest; the next week I would focus on my own classes, etc. Something always got left aside and my own "academic performance" suffered.
This is what it comes down to. Universities are not willing to pay for real instructors for young students any more. They have a fundamental disrespect for the students coming out of high school and don't bother to ensure them good quality instruction at the Freshman level because they think it's a waste of time. What they do instead is find poorer graduate students and make them bust their butts teaching in order to pay their way through school, all the while telling them that they are doing them a big favor. In the end, everybody loses out except for the adminsitrators who put the savings in their pockets. (the president at that university earns $100,000+ per year and recently had the university buy him a house). As someone who has seen it from the other side, I know that they are wrong, but as long as it is an issue of money, the situation will continue to deteriorate and eventually Americans will have to travel to another country for education - or better yet - pick up the books and educate themselves.
Re: (Score:1)
Re:This is really scary... (Score:1)
This isn't noble, nor desirable, but it is the real world. Automatic grading just changes the medium, not the methods.
School can't be interesting, challenging and important to all students all the time.
The real problem is finding the middle ground between hypocrisy and honesty now that I send my own kids to school. And I suspect it must be much worse for teachers.
Re:Teachers in High-School are exempt from rules (Score:1)
Re:What you get with a large "prestigious" school: (Score:1)
Re:So true.... (Score:1)
Remember that challenge advertised right here on
Dr. James Haskins (noteable for his book The Cotton Club, and many, many others) once said in class that we would tell our grandchildren about when we had books in school, since it will become a thing of the past.
Now it looks as though we are not only going to be without books, but the people in school won't have to read from a computer screen either. Just get someone who knows the web to d/l something that looks original and pass it off as their own.
It's their future, but it's our present - let's fix this now.
Re:How does this really evaluate papers? (Score:2)
Re:More on them there right answers (Score:1)
THe "Oedipus Complex" is a core trait of the human psyche (according to Fraud, anyway), and Freudians claim it is the sole route of all complex' and disorders.
Basically, the story is this:
Every young male is in love with his mother, and jealous of his father. He is scared of his father so distances himself from his mother to get on his fathers good side to avoid castration (Freud was abused as a child). With girls, it's slightly different. They're jealous of her fathers penis (this is where the term Penis envy comes from, it has nothing to do with size), so distance themselves from their fathers.
See, some of the BS you learn in English is useful in other areas...
Re:Teachers in High-School are exempt from rules (Score:1)
If you need to point-and-click to administer a machine,
Re:Evolution in Action (Score:1)
The holy grail of american education the SAT is done the same way. I made quite a bit of money in high school teaching idiots how to trick the SAT to get 100-200 more points just be knowing how to guess. It didn't make the many smarter or more knowledgable, just better test takers.
Kintanon
Re:A Perplexing Problem (Score:1)
'Tis BS, true, but look at the scale of the problem that the educational system tries to solve. Potential employers of whatever stripe have to separate the wheat from the chaff quickly and accurately. It's a lot easier to do if they can say "ah, this person has X degree from Y school" rather than having to evaluate more subjective statements ("Well, I read a lot of Descartes a couple of years ago.") We're suffering through the same thing with technical certification right now. "5 years of experience as a Novell sysadmin" doesn't really tell you much. "Certified Network Engineer" allows you to at least assume exposure to a certain range of topics. Having the CNE doesn't make you a good sysadmin, but it does give a prospective employer a way to evaluate you.
The problem is that your new boss won't know how good you really are until it's too late. The guy who's gone to school and has the degree or the certification has at least shown that he's got the discipline and the ability to see something through. That gives the boss a reason to have some faith in you.
I hated some of the required courses I had to take in HS and college, but that didn't cause me to drop out. Sturgeon's law states that "90% of everything is crap," and I believe that applies to school and industry. You never escape the BS...but school inures you to it.
Neutron
Was that a mispelling.... (Score:1)
Re:Big Brother (Score:1)
Our programs were submitted via e-mail, using specific subject lines made up of your student number, assignment number, etc...
The system automagically grabbed the code, compiled it, ran it, checked output, did pattern checking for cheaters (and since we were allowed work in groups and submit the same code for each member of the group, it took that into account too), and even had a scripting method to allow the professor to check for specific things to add or subtract points for (like if we had to use a certain function, or a certain type of data structure).
Since we were usually writing classes and functions rather than whole programs, this is actually a bit easier than it sounds. He'd build a framework that each assignment must fit in, type of thing..
The professor wrote it himself, over the years. I finally got to play with it when I got the password to his unix account (easy to guess.. only took 8 tries). I added a few backdoors, which was fine by me since I wasn't in his classes anymore.
Sometimes I wonder if he ever found them, since I did tell a lot of people how to beat the system once I had corrupted it.
This story probably is very boring to you, but it gave me something to type.
---
Re:Evolution in Action (Score:1)
Design flaws in the K-16 system (Score:2)
Design flaw #2: Bell curve based testing used as a marker of how "good" a school is. All the children are NOT going to be above average, and a reward/penalty system based on this is ludicrous.
Design flaw #3: The assumption that children and young adults are not full human beings in the same way that missionaries assumed that "savages" were not yet full human beings. Instead of accepting Jesus Christ, we have to accept the American Way of Life as our Savior. If kids were treated as rational human beings, more of them would have more incentive to ACT like rational human beings. The resemblance between schools and prisons is growing every day, and it frightens me. I've seen studies that have shown that first-time offenders sent to jail have a higher recidivism rate than those put on probation. It would be nice if those making decisions that affect today's youth would LEARN from this!
Design flaw #4: Teachers are treated like glorified baby-sitters by EVERYone -- students, parents, and administration alike. Then, they are attacked for "not teaching."
The replacement system I'd like to see: Students are free to learn at their own pace, in a manner of their choosing, and their education will be fully subsidized until they reach the age of 21. They may at any time take tests or an alternative method to prove themselves "certified" at any level in any subject. They do this when they feel ready. One of these tests would be something akin to the current GED, and would be accepted as a "general high school diploma." Anyone at this level gains emancipated minor status and the right to vote if not yet 18.
Re:Obsolete skills need not be taught anymore. (Score:1)
treke
Some content checker... (FLAW) (Score:1)
(Incidentally, Knowledge Technologies [knowledge-...logies.com] is hosted on computers at Colorado University)
Anyhow, take sample essay 1. Unmodified, it scores a 9 out of ten, with blah blah blah analysis on content, sentence length, and so forth. Go back, submit the same essay after negating every sentence (Billy did NOT show signs of abandonment, etc...) and submit it again, and it still gets a 9 and is still a 'great' paper, though being totally and utterly incorrect.
Circle is not Complete! (Score:1)
Re:Obsolete skills need not be taught anymore. (Score:1)
Using tools is great once you know the underlying stuff, but if you skip that step, you've had all the fundamentals of math abstracted out from under you. You'll never make mathematical progress, because you don't know the basics. Your hand calculator really is better than you, because it was programmed by engineers and mathematicians who knew what they were doing.
Isaac Asimov wrote a truly excellent short story about the consequences of the path you're talking about. Anyone remember the name?
Re:Was that a mispelling.... (Score:1)
Re:Obsolete skills need not be taught anymore. (Score:1)
Essays (Score:2)
Granted, having to grade ~100 essays is not the most pleasant way to spend a weekend. My wife hates doing it, much the same way I hate having to spend the same time working on my code. But she does it because *that's her job*! A teacher that uses some software to grade the essays because they are too busy to do the work themselves is (in my opinion) not a good teacher. Period.
Kids need to write a lot. The only way to become comfortable with expressing oneself with the written word is by practice. Many of the essays my wife gives these students are not onerous to grade because she isn't looking for 5 "magic words" that indicate they've read the textbook (or, more to the point, copied the textbook). Instead, she is looking for well constructed essays that come from having to organize one's thoughts and write them in a concise format. This can be analyzed with a cursory reading. However, when the point of the assignment is to have done careful research, that's what the bulk of the grade is based on, and what she spends the most time evaluating.
And most importantly, she makes sure the amount of time she spends grading the work is proportional to the importance of the assignment. If the assignment is a position paper worth 10 points, she isn't going to spend 15 minutes reading each paper. She'll check to make sure the paper is cogent and meets the guidelines she's laid out. But if the assignment is a 10 page analytical paper worth 200 points, she's going to spend a lot of time on each one, because it's going to be an important part of the kids' final grade.
The kids aren't stupid. If they realize that the teacher isn't taking their work seriously, then the kids won't put serious effort into their work.
Think of it like this - if your boss didn't take the time to give you a fair review of your work, would you be willing to put forth extra effort for your boss? It's not a matter of "modern pedagogical theory," it's about human interaction, pure and simple.
And "grading software" is just a symptom of the breakdown in interpersonal interactions endemic to our society
Re:Essays (Score:1)
Plus, the summer allows her to pursue her hobby, writing historical fiction. The summer is when she typically does her research.
The point is that students will not be motivated to perform well if their teachers are not willing to treat the student's work as something worth spending time on.
Re:A Perplexing Problem (Score:1)
One thing that'd have to get fixed before the problem gets any better is the objective of schools to, specifically "get all the kids though X years of school with the appropriate number of passing grades", instead of the more appropriate, "make sure all the kids have X knowledge and understanding".
For example, if someone doesn't understand the usage of variables in equations, they should be taught it, not just "well, you got a >65% grade so you go on to the next class". On the other hand, students should *never* be required to "be taught" that which they already know.
Schools don't exist to make kids do work, schools exist to teach students new things. Schools should therefore be optimized for efficient operation, and not for anything else (simplicity, ease of teaching, minimal cash expenditure, sports programs, whatever).
I invented the Intelligent Essay Assessor... (Score:5)
First, let me give you a little history (the lingua franca article referenced in the story is a good read for a bigger history).
I am a cognitive psychologist, as are my partners Tom Landauer and Peter Foltz. I am also trained in educational psychology with an emphasis in measurement. The intelligent essay assessor started as an experiment testing our computational model of human knowledge representation. The model, called Latent Semantic Analysis (LSA), is similar to other artificial neural networks, but it learns its representations from extremely large bodies of text (it is limited to text right now). If you are interested in the underlying technology, please go to our academic website lsa.colorado.edu and download some of our journal articles.
We have found in our research that LSA makes judgments about text that closely mimics human judgments on a number of standard psychological experiments (e.g. categorization and sorting tasks). We wondered if the model could judge the quality of content in student essays similar to trained readers. I have spent the last several years testing this and we have found that the IEA consistently agrees with human judges *as well as they agree with each other* when trained properly. This tends to be an inter-rater reliability of around a
When we presented our research we got so many people wanting to use it that we decided to apply for a patent on the method and to form a company to market it.
There have been a lot of misconceptions/disinformation about what it can do and how it should be used. The current form of the IEA is appropriate to use for short answer essays (aka constructed response items) for directed prompts (aka focussed questions). It is meant as a replacement for multiple choice questions on content driven material, not as a replacement for English lit and creative writing teachers. It should be used in support of the 'Writing across the curriculum' movement so that students get more of an opportunity to write (rather than just fill in bubble sheets). It is not appropriate for 'term paper' type of essays where each student response should be unique. By using short essays to assess content knowledge rather than multiple choice questions, you encourage the student to learn the material at a deeper level. It is much more difficult to *recall* the correct answer and present it than it is to *recognize* the correct answer and circle it.
We currently want the IEA to be used as an interactive tutoring system for writing -- if you go to our website you will see some demonstrations of its use. We are interested primarily in formative assessment allowing revision rather than summative assessment to rank the students. Our goal is to help students learn. Our latest demonstration ties the technology to specific textbooks. You can have a list of essay questions at the end of each chapter of a textbook. After reading the text you choose a question then write an answer. The feedback will tell you whether or not you learned the information that the author of the textbook thought was important and where in the textbook you can find that information.
We honestly think that this system will help students learn and communicate. The press, to their discredit, has focussed on 'cheating teachers' implying that this system is a way for them to get out of their jobs. This is absurd. Look at any professor in college with several hundred students in a class or any teacher in K-12 with over-burdened resources and you will see that they rarely can afford the time to assign essay questions, so students never get the opportunity to write. This system gives students that opportunity. In some ways it is better than teachers (speed of feedback, objectivity, consistency) and in many ways it is worse than teachers (limited capabilities of understanding novel approaches, needs to be specifically trained for each domain/question), but we never wanted to see it as a replacement for teachers, rather as another tool for them to use in the daunting task of education.
I do appreciate the intelligent (for the most part) conversation you have brought to this subject. I look forward to continuing this discussion.
Cheers, Darrell
dlaham@knowledge-technologies.com
Re:Interesting, but I'm not sure things are critic (Score:1)
There was a point in time when you actually expected to learn something in high school? I gave up on that idea back in 6th grade. (I'm a HS Sophmore now, and I still think that highschool is a complete waste of my time.)
Re:More on them there right answers (Score:1)
Second, it's not the choice of literature (though I thought that Oedipus was absolute garbage and had very little of value), but rather the interpretation. Oedipus killing his father yet saving the city isn't irony in the least. Irony is something like, hmm, I don't know, maybe my reference to your little Freudian slip and its connection w/ the invalidity of Freudian psychology.
-Laktar, a.k.a. Nick Rosen, laktar.dyndns.org
If I Ever Became An Evil Overlord:
80. If my weakest troops fail to eliminate a hero, I will send out my best
troops instead of wasting time with progressively stronger ones as he gets
closer and closer to my fortress.
-- Peter's Evil Overlord List, http://www.eviloverlord.com/lists/overlord.html
stupidity of mechanical grading (Score:4)
Re:Break out the pens and pencils (Score:1)
Computers these days can turn out a decent simulation of handwriting -- what are they going to do to counter that, send the papers to a forensics lab?
Ultimately, it comes down to teachers who can tell the difference between an "A" paper and a kiwi fruit.
/.
... (Score:2)
Hello, vis a vis, per say, imagine that! If you compare the logarithamic pseudo double sine could represent paradigm shift technology internet and if I dare say so myself ology scientific method methodology sine quo non ...
Result: You get an A!
--
Re:I used to use watch calculator! (Score:1)
Re:I used to use watch calculator! (Score:1)
I actually have the reverse problem, I have a difficult time with any math which I can not work out on paper. At my school Trig and Calc were taught as 'Calculator' classes, we were more or less taught how to do it on a calculator. I can't learn that way, I never grasped the concepts because I couldn't write it out and see them working. I managed to pass Trig because the teacher was great and went outside the lesson plan for us, and I almost passed calc due to outside study. I was taking them both the same year, so that didn't help either... But I wish someone had shown me how to work stuff like sin/cos out on paper...
Kintanon
Early specialisation (Score:1)
Is this useful? Sure, after only three years I knew a lot more physics then a US four-year college physics major, but I had not taken a single course in languages, writing, philosophy or any of the other subjects that are useful in a well-rounded education. Didn't really bother me as I come from an artsy family and have always had my own interest in the arts and philosophy. But looking at other people graduating (and even more so those graduating from Eng courses) and a lot of them were complete philistines, many with a very unattractive disdain for the humanities that verged on almost fascist anti-intellectualism.
In contrast, many of my 'merkin friends, most of whom are from physical science/eng/CS backgrounds seem to have a much more well-balanced background - and are just as likely to have a conversation about Impressionism as about Romberg quadrature.
Now think what would happen if you went to single-subject teaching at an even earlier age. Sure, we'd probably get great programmers - but can you imagine holding a conversation with them at a party?
And as for the comment that people should rent the movie rather than reading the book - I can only assume this is a troll.
Nick
Re:I used to use watch calculator! (Score:1)
It's the learning objectives, stupid (Score:2)
The bottom line is simple educational theory.
You start with a list of learning objectives. You come up with an assessment process which measures a student's achievement of those learning objectives.
Lazy educators focus on "end products" and "right answers" rather than learning processes. These are easily spoofed.
The corporate culture of "I want something delivered by tomorrow, no matter what" supports this kind of "educational production". Mainstream media makes an attempt at being outraged by cheats, but essentially the bottom line is that most organisations would rather hire a doofus MSCP than a less qualified person who really knew what they were doing. Not many people really want to spend the time measuring whether a job actually gets done properly (whether it's education or a LAN rollout) - they just want a set of bits of paper that tells them it's someone else's fault if things go wrong.
Competition is getting us to a point where this isn't good enough anymore. I think you'll start to see qualifications from places which focus on a student's ability to create/buy/steal end products lose their value, compared to those which measure learning more effectively.
Danny
Re:Big Brother (Score:1)
Re:A Perplexing Problem (Score:1)
-Laktar, a.k.a. Nick Rosen, laktar.dyndns.org
If I Ever Became An Evil Overlord:
70. When my guards split up to search for intruders, they will always travel
in groups of at least two. They will be trained so that if one of them
disappears mysteriously while on patrol, the other will immediately initiate
an alert and call for backup instead of quizzically peering around a corner.
-- Peter's Evil Overlord List, http://www.eviloverlord.com/lists/overlord.html
Re:THIS IS SO GREAT! (Score:3)
Knowledge from other areas can help you improve your work in your primary field. I'm convinced that we'd have much betmore ter software if we had our programmers write poetry - and better poetry if we trained our poets in logic.
It is a tiny mind that can find joy and success in only one field of human endeavour.
Re:Interesting, but I'm not sure things are critic (Score:2)
...a firm background in mathematics and logic would obviously complement that. I'm not sure that many high schools really expose students to proving systems, symbolic logic and so forth, however, so that perhaps needs be supplied outside the system. Pity, that. ({sigh} Mentat training, anyone? Heh.)
It's also a decent way to boost writing skills. All the ideas and skills in the world don't mean jack if you're incapable of communicating, and arguably C or differential calculus are not suitable general-purpose languages for conversing with one's fellows.
This is particularly important because out there, you're going to be expected to be able to work with others...
And so forth.
Re:Essays (Score:1)
Your wife is very very generous then. Let's take a look at the starting salary of an average teacher, 27000 in maryland, now, they work 6 hours per day, 5 days a week, 10 months a year. 1200 hours total. $22.5 an hour, looks good, right? Now add in the time before and after school that they spend working on lesson plans, giving extra help to students, now we're up to 8 hours a day and down to 16.875 an hour, now go ahead and add in another 5 hours per week for grading papers and other assignments, that's now 9 hours a day, at 15$ an hour, now factor in the 2 months they don't get paid for and you have a teacher doing a WHOLE LOT of very stressful work that takes a lot of time and concentration and getting paid very very little for it. How many teachers are going to be able to deal with that for very long??
Kintanon
Re:Exactamundo (Score:2)
Erm. Four years as an undergrad, in my second as a grad, and I could probably count the number of my courses that even had a single multiple-choice question in them on one hand -- and I'm not a freakish mutant...
Heck, even in our CS classes, we sometimes give questions that include words like 'critique' and 'assess'. Why? 'coz we want to gauge understanding...
I would think that, if *anybody* would reject multiple-choice, it'd be an English department, since writing is so obviously fundamental to it. That's so utterly... FUBAR.
Everyone needs English... (Score:1)
Re:Everyone needs English... (Score:1)
You're assuming that I can't wright now. I'll admit that I'm not the world's *best* wrighter, but I can wright well enough to be posting on Slashdot.
What's being taught in my current "english" classes is basically spelling, grammar, and vocab. My spelling isn't all that bad, my grammar is understandable, and my vocabulary is probably greater than most highschool graduates in the USA. After my current year of "we don't if it's Lit or Comp", all the avalible English classes are things like "Advanced Lit: Russian Poetry translated by Italians of Aztech decent". School isn't going to help me learn to express myself better, it's just going to waste 3500+ hours of my time over the next three years, time that could be better spent on other, usefull endevours.
Wrighting? *cringe* (Score:1)
How does this really evaluate papers? (Score:1)
Kintanon
A Perplexing Problem (Score:5)
Both sides of the issue in this column point to the same problems:
1. A society that considers someone who put in practically no effort through 16 years of schooling more "educated" than someone who has learned through the "school of hard knocks." This is, of course, complete and utter bullshit, yet it persists.
2. Pressure on students by bad/uncreative teachers to come up with "the right answer" rather than an answer that actually shows some thought. Required rather than suggested topics for essays, teachers who grade based on their own personal biases rather than for content, and "students" who'd rather be doing anything but going to school all contribute to the problem.
Fix problem #1, and problem #2 might be less of an issue. Wouldn't it be wonderful to have a classroom full of people who honestly wanted to be there and who were THINKING about what they were doing?
Many college instructors try to be 'bots anyway (Score:1)
Also, several teachers that I have known outside of the classroom would not assign anything that "takes too long to grade".
Even with these bad experiences, I believe that grading 'bots for creative work are a very bad idea.
Interesting, but I'm not sure things are critical (Score:4)
All in all I don't think there is much of a threat to education, although it may widen the gap between students who are in school for a reason and students who are just there. I don't want to sound hypocritical, I hate most classes as much as the next student, but I am usually able to drag myself to class on the promise of learning something useful.
As for the grading software, I'm not sure I would be opposed to it. I have been both the victim and benefactor of subjective teachers. Nothing is more depressing than having a teacher who gives you the same low grade no matter how hard you work, or more effort sapping than a teacher who gives any drivel you write an A. It would seem the smart thing to do, if you are going to be using software, is establish a database of papers available n-line, and compare students' papers to them, putting warning flags up for similarities.
Over all I don't see a huge problem. No self-respecting student would hand in a paper written by someone else. And no self-respecting professor would let software grade papers for him.
Evolution in Action (Score:3)
But it makes sense. If teachers change the criteria for a passing grade to the criteria in some software - which can be inferred by analyising its behavior, or REing the program - than students don't have to learn anything. Just successfully spoof the program. Or learn only what the program needs them to know, and nothing more. (so if it tests on 1st, 16th and current presidents, who cares about 2-15, 16-the last guy)
It wasn't so bad when the students just had to spoof the teacher. At least crappy teachers (a good teacher's going to use this? schwa, right) were generally crappy in different ways.
I fondly remember my HS math teacher who didn't teach me a thing except that my vision had deteriorated to the point when I needed to get glasses to see the board. Taught myself the whole thing in summer school, got an A and bitched her out. Ah, the memories....
Sadly, there will be some problems, and they're the same sort of homogenization problems that are evident in computing. MS will practically cut off your arms to make sure that one size fits all. Now we'll be penalizing creativity because it doesn't match a statistical model of the ideal paper.
In the old system of course, bad teachers are just as bad as this software, and good teachers way better. There was a chance, at least, that you'd get the good teacher. Now there will be only one teacher - the program. Monocultures strike again....
Big Brother (Score:4)
Big Brother wasn't the final judge of course, but it was very good at picking up cheaters. At least I think it was. I'm sure it missed anyone smart enough to think about how Big Brother must work. So I guess it picked up lazy and dumb cheaters, but not lazy and smart cheaters. Which isn't a bad compromise if you think about it.
This is quite different from the sort of tool described in the cited article. I think it is an arguably good use of technology on the teachers' "side".
Good quote from Essay.org (Score:1)
The evolution of education... (Score:1)
It worked pretty well, she relieved pressure by not taking off (too much) for punctuation, grammar and especially spelling.
Hey, until we get celluar uplinks in our skulls, this low-tech approach may be the only way to make sure that the students are actually learning something and not just skating through school.
Shaken
Essay topics (Score:1)
Take Shakespeare for example. Everybody from my home province had to read one play a year from grade 9 on until graduation. In each Shakespeare play, there are countless topics that essays can be written on.
What happens is that the teachers get lazy. They keep using the same old "standard" topics year after year to make their job easier by marking the same thing over and over again. How many of you remember writing about "Fate vs. Free Will" in Romeo and Juliet? Or how about Ambition being MacBeth's downfall, or similarly Hamlet's procrastination?
And let's not even start by looking at the obvious clues that a student is cheating. The dumb jock who can barely talk, managed to pound out an in-depth 15000 word essay on Othello's relationship with his dog Fluffy. When student grades fluctuate and are strangely not in character, teachers should investigate. But then they would actually be earning their 30k/year, wouldn't they?
Re:Custom essays? (Score:2)
The point being that there's less demand for papers of any kind on obscure topics. You can make quite a bit of money writing custom papers about Shakespeare, virtually none writing papers on the design and implementation of role-playing games 1979-1983.
When was school ever about learning? (Score:1)
In this day and age, just like any other day and age, if you want to learn something useful before you go to college, you have to go out and do it on your own. If you're lucky, when you go to college they actually start to teach you how to think, but by then you have to break 18 years of bad habits that the lower educational institutions have drilled into you.
Re:Custom essays? (Score:2)
Pre-writens also have other drawbacks -- you can't say "oh, by the way, the prof really likes to see theme foo emphasized and thinks interpretation bar is a load of crap." You have to either customize the paper (so why bother buying anyway -- if you're doing it right, rewriting is the most intensive part of the writing process anyway) or read a whole bunch until you find one you think is appropriate.
Another application for this tech... (Score:2)
I'm half serious... It seems to me that most offtopic, flamebait, or troll posts could be flagged by a statisical analysis pretty easily. It would be a little harder to pick out the better posts, but probably not impossible. The archive of thousands of comments that have already been moderated by humans could be used to train the system.
The Bell Curve (Score:3)
So what does the bell curve have to do with cribbing essays and work from the net? It raises competition. The art of education becomes cheating. Whoever is the most innovative cheater, wins. Its because when the grades go up, the level of work must increase to equalize the grades. So, more people suffer from burnout and turn to cheating. Its a race.
I graduated in 1992 with an engineering degree and get to see where the cheaters are today. It brings me great joy to see that they are employed, but in places like Walmart, approving checks, and meanial jobs like that. Those who do not have a clue now will never get it on their own later.
Imagine students and teachers all battling it out and competing against eachother like a game of chess. Who's going to win? It doesn't seem a very efficient method of education to me.
Back when I was in school, it was paper and pencil. It was a drag without the calculators, but it was fair.
Re:Custom essays? (Score:2)
The ones that would likely need custom writing are the more unusual ones.
Johnny, what did you learn today? (RANT) (Score:3)
What passes for education today is what is going to make us a nation of fry chefs tommorow (no insult intended toward actual fry chefs.) It's like these Microsoft Certified Training Courses I'm attending - the focus is NOT on learning the subject matter, it's on passing the test so you can have a neat little paper signed by Bill Gates that says "I'm a Systems Engineer!" and in reality signifies nothing. But as long as you get the certification, degree or whatever, that is all that counts anymore - not your ability to perform in the real world.
Reverse-engineering. (Score:2)
--
Mike Hoye
--
What you get with a large "prestigious" school: (Score:2)
Re:This wasn't even new a long time ago. (Score:2)
For one thing, profs may let their TAs completely restructure and revamp the specifications and code. If your submission does not link or otherwise meet the specs, then you've got problems...
The other thing is that it's harder to automatically compare the structure of papers than it is the structure of programs. If you just rename variables and functions and so forth, it's not that unlikely it'll be noticed.
Re:... and it's just an implementation of old idea (Score:2)
Bad Analogy? (Score:3)
Assigning papers that call for the rote repetition of researched facts (I wish there were another r-word in there) is like forcing musicians to hand-craft their own instruments?
Serious Musicians building their own instruments (at least once) would be a good thing. Just like Serious Programmers writing their own programming languages or compilers or whatever else good Computer Science Courses have them do.
Now the author is right in that repeating the same facts on some assigned topic isn't all that useful. But isn't the point of education to teach people how to find things out for themselves? (Okay, maybe I'm an idealist.)
If I were a teacher, I'd rather have one student sweating over a stack of books in the library for the first time in his life turn in a list of facts he culled from that stack, giving credit to those books, than a dozen students (like I was) who can crack a book and write a nice essay that doesn't say much but winks at the author.
Yeah, essays for sale and graders for sale subverts this process... but plagiarism was always a problem with education, and graduate assistants have been doing the grading for a long time anyway. That's not a reason to get rid of meaningless assignments. At the very least, it prepares students for the Real World. (As I'm contemplating a two-hour meeting to present some guy with a box of cigars....)
--
QDMerge [rmci.net] 0.4 just released!