Become a fan of Slashdot on Facebook


Forgot your password?
The Internet Communications

Paying People to Argue With You 397

Bennett Haselton has written in with an essay on a strange experiment on-line. He starts When you first hear about's "Mechanical Turk" service, which allows "requesters" to pay "Turk workers" a few pennies to complete some task which is hard to automate but easy for humans, what's the first application that comes to your mind? The system has been discussed previously on Slashdot, but I'll bet a week's wages for a Mechanical Turk worker ($1.45, according to one of them) that I was the first person who used it to pay people to write rebuttals to one of my arguments. Keep reading unless you want to fight about it.

The interesting result was that some of the rebuttals were quite insightful, and resulted in me making changes to the argument that I would make if I had to present it again. Judging by the literacy and intelligence of some of the respondents, most of them probably wouldn't need Mechanical Turk as a source of income, so I assume most of them fit the profile of this writer and are doing it just for fun. Hell, you can find enough people on UseNet and Slashdot who will argue with you just for free.

But there were a few reasons I found this preferable to the conventional ways of gathering interesting rebuttals to your own reasoning. If you send out a sample argument to all of your e-mail buddies, you will probably get some useful replies, but they may start to think you're a little weird for asking them to evaluate your thought processes, especially if you do it over and over. Post an opinion on UseNet or Slashdot, and you may have to wade through a lot of crap to find the useful responses (while others may consider your post to be part of the crap that they have to wade through). And in both cases, there's the potential embarrassment of what you're asking for -- the risk of seeming so uncertain about your own opinions that you want other people to check your work for you. (I actually think that being uncertain about your own beliefs is a virtue, but it doesn't seem to be one that our culture prizes very highly.) Using Mechanical Turk addresses most of these problems; even though you're still admitting to total strangers that you might be wrong and asking them to shoot you down if they can, at least the evidence of your insecurity won't turn up when your next employer or Internet date does a Google search for your name. ("Damn it, I want a man who doesn't question his bumper stickers!")

So, while I didn't find it useful enough that I would run every opinion through the Mechanical Turk machinery to see what feedback I could get from it (I'm not paying a bunch of them to proofread this article), I did like enough to recommend it to people for certain arguments in certain settings. The main kinds of arguments that I would try out on the Mechanical Turk service would be about abstract philosophical or moral questions on issues that have been around forever, like abortion or the death penalty -- topics so explosive that you'd risk making your friends very uncomfortable if you test-marketed your arguments on them, and which would seem almost rude to post about in a public forum because the debate topics have been around for so very, very long. But on Mechanical Turk, $1 is apparently enough to get people to ignore the awkwardness and the exhaustedness of the topic and to focus on what you ask.

And what was the argument that I used to test it out? Perhaps the geek crowd will feel more sympathy with this than the general public does. Basically it was that the conventional wisdom behind allowing adults to smoke, but banning cigarettes for people under 18, is wrong. Either you can believe that smoking should be permitted for everybody, or that it should be banned for everybody, but there is no consistent set of assumptions that could lead you to conclude that smoking should be banned for people under 18 but allowed for everyone else. You have two groups of people under consideration -- people under 18 who smoke, and people over 18 who smoke. What possible reason could there be for wanting to protect the health of the people in the first group, but not the people in the second group?

The problem with the conventional reason for smoking age restrictions -- "Younger people have worse judgment, so they are more likely to smoke" -- is that if this is true, all that means is that the first group of people will be proportionally larger, relative to the total population of people in their age range. But even after that assumption, you're still left with two groups of people, who exhibit the same continued bad judgment with regard to smoking cigarettes. Treating the two groups differently, is a bit like saying we should have lighter sentences for female murderers than for male murderers, just because men are more likely to commit murder.

And yet this conclusion did give me pause, so this is a classic example of an argument where you'd want someone to check your work. Off I went to create a Human Intelligence Task (HIT) on Mechanical Turk simply asking people to read the argument and respond. In the first round, most responders missed what I thought was the point of the argument, and responded with some variation of "Minors are more likely to smoke because they have worse judgment", without addressing the question of why the two groups of smokers should be treated differently. A few people responded with variations of "We've always done it that way" (referring to similar restrictions on alcohol, pornography, etc.); fair enough, it just reminded me that if I asked the question again I'd have to say I didn't consider any argument valid that boiled down to "We've always done it that way".

But then came some more interesting responses. One worker replied that I was wrong to assume that the effects of a cigarette were "the same" on adults and minors because cigarette smoke has been shown to be more damaging to developing tissues. OK, that was worth a dollar. On the other hand, that just means that there is some number N cigarettes that would be just as harmful to an adult, as 1 cigarette would be to a minor, so you're still left without a consistent reason for why you'd let the adult buy those N cigarettes but prevent the minor from buying 1 cigarette. Then another user called me out on the opening line of my original argument, "There is no reason to ban cigarettes for minors but not for adults." He said, quite correctly, that I had only attempted to debunk the most commonly given reason, but it was wrong to conclude that there was no such reason.

So, this led me to another idea for how to present an argument and solicit feedback on Mechanical Turk: in the form of a series of mathematically precise statements, each one following from the previous ones. The new HIT was to ask users if they disagreed with the conclusion, and if they disagreed, then to identify the first statement that they disagreed with. The idea was that each statement would follow logically from the ones before it, so identifying any statement as the "first" one that they disagreed with, would be tantamount to a self-contradictory paradox.

Now, whether or not you want to use this format when running an argument past the Turk workers, depends on what your goal is. If you want to really find out if your own argument is valid, then breaking it down mathematically is one approach. On the other hand, if you already believe your own argument, and you're just trying to find the most persuasive way of phrasing it, then you may not learn anything useful by breaking it down into a series of mathematical steps, because that's probably not going to be the format of our final persuasive essay.

Anyway, the new mathematical format of the argument was (slightly reworked from what I posted on Amazon):

  1. Government should ban smoking by people under 18, because of the harmful health effects.
  2. If that's true for the entire group of underage smokers, then it's also true for each individual smoker under 18. In other words, even if only one person under 18 smoked in the entire country, it would still be justified for the government to ban them from smoking.
  3. Whatever bad health effects are caused by the average person under 18 smoking 1 cigarette, there is some number N cigarettes that would cause the same bad health effects in the average adult who smoked them.
  4. If banning 1 person under 18 from smoking 1 cigarette is justified (even if they were the last smoker on Earth), and the health effects would be the same for an average adult who smoked N cigarettes, then banning 1 adult from smoking those N cigarettes would also be justified (again, even if they were the last smoker on Earth).
  5. If banning 1 person over 18 from smoking would be justified, then the same logic would apply to every person over 18, which would imply banning smoking for all people over 18.
  6. Hence, if you believe that smoking should be banned for people under 18, then the same logic would lead to a ban on smoking for people over 18 as well.

The response from a lot of workers who responded to this HIT was that... I lost them. Each of them identified the first statement in the list that they disagreed with, as required by the HIT, but many commented that the whole thing was phrased confusingly. There was no clear winner for the first statement that people disagreed with, but several people picked #3 and #4, arguing some version of "People under 18 have less developed judgment." (I still say that doesn't matter, because you're talking about comparing a person under 18 who smokes, with a person over 18 who smokes, and their judgment in both cases is the same, etc.) So this particular experiment failed -- it didn't make it easier to persuade people by formulating the argument as a series of steps, and it also didn't lead to any agreement on what was the Achilles' Heel of the argument itself.

However I think the general idea, of using Mechanical Turk to find sparring partners, may be useful to a lot of people. If you were interested in publishing some kind of persuasive argument, you could use an Amazon HIT to have readers compare several different versions of the same argument and identify the one that they thought was most convincing. If you were feeling more philosophical and simply wanted to know if your argument was correct, you could pay people to look for flaws in it (and here is where the mathematical phrasing could come in handy). If you're crafting an argument for public consumption, you could even have HIT workers build up your argument for you -- start with a position and have them come up with reasons supporting that position -- although to me that feels like a cheapening of the debate process that crosses the line, because you're not even trying to reason your way to a conclusion, instead starting with the conclusion you want and then working backwards (not that this isn't what a lot of debaters do anyway!). My own interest would be to see next if certain types of arguments are more likely to persuade people who are more mathematically inclined (by asking respondents to indicate how well they did at math in school). Perhaps arguments with flowery language are more likely to appeal to people who were English majors, while arguments spelled out as a series of logical steps are more likely to appeal to people who look at things in a mathematical way (also known as the "real" or "right" way of looking at things).

Maybe my preference for the controlled, user-reimbursed process of "debating" that is enabled by Mechanical Turk, has to do with a lifelong focus on bottom-line results: Decide what the result is, and judge the process by how well it brings about that result. I don't think debate and discussion should be like soccer, valued for the fun and the exercise; I think a good debate should actually get somewhere, persuading the participants or the listeners of a new point of view that builds on their old one, or else the debate has failed. If paying HIT workers kills the "spirit" of a good debate but helps achieve the goal, then so much the better. On the other hand, we'll never run out of people who enjoy the process of debating and arguing for its own sake, and will continue to debate things into the ground without anybody paying them. Hey look, here come some of them now!...

This discussion has been archived. No new comments can be posted.

Paying People to Argue With You

Comments Filter:
  • Re:An Argument (Score:2, Interesting)

    by Fallus Shempus ( 793462 ) on Monday November 05, 2007 @01:24PM (#21243117) Homepage
    You can also add:

    4) The purpose of the ban is to stop people from starting smoking.
    Since most people start smoking in their teens a ban to the age of 18
    is there to stop these people from picking up the habit.

    As the parent states there is not a total ban as this is possibly a rights issue with an adult.
  • Sheep (Score:3, Interesting)

    by Blakey Rat ( 99501 ) on Monday November 05, 2007 @01:27PM (#21243167)
    This is a much better use of the Mechanical Turk service: []

    Get 1,000 random Internet users to draw you a left-facing sheep.
  • by blueZ3 ( 744446 ) on Monday November 05, 2007 @01:34PM (#21243283) Homepage
    You imply that adults "weigh" the long-term consequences of smoking and then make a rational choice to continue smoking. As a former smoker, I'm not sure that's an accurate reflection of what's happening with smokers (adults or children) nor a compelling answer to why we ban underage smoking.

    A clearer explanation, in my opinion, would be that we expect adults to accept the consequences of their actions as a matter of personal responsibility. In other words, adults could reasonably know that smoking is dangerous and choose to smoke anyway, but they have to live with the consequences; lung cancer, emphysema, etc. We don't necessarily hold "minors" to the same standard. If a child gets their father's gun, takes it to school, and shoots another child, the consequences (should) fall mainly on the parent.

    Of course, in our modern society, there are some really wacky things going on. People can sue corporations for the result of bad choices they've made, because we've pretty much abdicated on the idea of personal responsibility in non-criminal cases, while at the same time a five-year-old can be tried as an adult and sent up to the Big House if they use crayons to draw a gun in kindergarten.

    But my take on this is more that (historically) we've decided that kids aren't held to the same standard of personal responsibility that we've (historically) assigned to adults.
  • by Per Abrahamsen ( 1397 ) on Monday November 05, 2007 @01:38PM (#21243347) Homepage
    The benefit of a ban is possibly improved health in the group of people the ban effects.
    The cost of the ban is decreased freedom in the group the ban effects.

    Your argument address the possibility that the value of the benefit might differ with age.
    You argument fail to address the possibility that the value of the cost might differ with age.

    Since society value freedom for adults much higher than freedom for minors, the age discrimination is justified in this case. QED.

    (A totally different discussion is whether society should value freedom for adults higher than freedom for minors, but it clearly does).
  • Re:Dialectic? (Score:4, Interesting)

    by Bodrius ( 191265 ) on Monday November 05, 2007 @01:55PM (#21243597) Homepage
    Also, maybe then people would not consider it weird or awkward when their friends asks them for a serious discussion and their intellectual opinion on an issue.

    Perhaps people will even stop considering submitting their opinions to rational debate a sign of weakness and an "embarrassment" - which is a particularly disturbing trend.

    My 2 bills of monopoly money:

    - I really like the general idea, simply because intelligent argument is a valuable commodity.
    If that is the price of creating a transient forum with critical mass, having people take the discussion seriously, and not seeing it degenerate into highschool psychology, it seems pretty cheap.

    - Even more valuable by far: friends who would not consider it weird that you want evaluate and improve your thought processes, at least not any weirder than sharing youtube links, jokes, and all the other less purposeful conversations that friends carry over all the time.
    Having someone open to a meaningful discussion who is willing to contradict you and tell you when you're wrong is not impolite; that is what friends are for.
    It may be the thing that makes the difference between 'friends' and 'long term acquaintances'.

    This contemporary fear of argument is chilling, if it gets to the point that even publicly thinking through your opinions is a cause for embarrassment.
  • by bigattichouse ( 527527 ) on Monday November 05, 2007 @01:56PM (#21243615) Homepage
    I agree.

    I remember in a democracy seminar I took in college, society as a whole decides who has the right of franchise, and this age frequently denotes the point at which a person is responsible for their actions. This is usually for those who fall under a certain age, and within certain norms and other boundaries. Additionally this is why criminals lose their franchise in the democracy (they "gave it up" when they exceeded a social norm for antisocial behavior).

    The discussion came down to why "18"? or "16" (for some things in europe) or "33" (I recall something about Masai and having to travel for an extended period before being able to marry - but could be mistaken)? It was purely based on a social decision, that at a certain age/limit a person is able to enter into a contract with another, and will be held responsible for that agreement.

    Sure you can look at brain development to back up your argument after the fact... but its a bit like coming to a conclusion then looking for physical evidence to support it. I believe it is related, but I believe the primary focus is just more fuzzy.

    18 is an arbitrary age for responsibility in our society, that is all it is. By consensus, we all pretty much agree 18 is a pretty good marker. Personally, after watching others (and myself), I think 21 should be more appropriate.

  • A counterargument (Score:1, Interesting)

    by Anonymous Coward on Monday November 05, 2007 @02:41PM (#21244273)

    - Government should ban smoking by people under 18, because of the harmful health effects.

    The government should ban smoking by people under 18, because this is a cost-effective and meme-wise implementable way of achieving the true intent of preventing people under 18 from incurring harmful health effects from smoking.

    The ban incurs positive utility on average from the extensive lives young people on average have in front of them, however negative utility from the fact that it prevents people from doing what they would like to do.

    - If that's true for the entire group of underage smokers, then it's also true for each individual smoker under 18. In other words, even if only one person under 18 smoked in the entire country, it would still be justified for the government to ban them from smoking.

    I don't see any neccessity to induce from the many to the one - the ban could be intended to prevent the average person under 18 from incurring unhealth H by smoking a large number of cigarettes, and is simply enacted as a blanket ban, as mentioned above. If the single person under 18 was someone who did not want to smoke a lot of cigarettes, but only a very limited number that was not sufficient to incur unhealth, then a ban would not have been required. It feels more natural therefore to stay with the averages.

    - Whatever bad health effects are caused by the average person under 18 smoking 1 cigarette, there is some number N cigarettes that would cause the same bad health effects in the average adult who smoked them.

    There are no harmful health effects from banning the average person under 18 from smoking 1 cigarette, and that is not the true intention of the ban to prevent either. It is simply implemented because it is a useful proxy for the true intent of preventing smoking so much that it incurs harmful health effects.

    If you scale up 1 cigarette to 20 cigarettes, then certainly, there is a number N cigarettes that the average adult may smoke to incur a similar health cost. Or is there? Firstly, disutility of health should not be defined simply as being in a state of unhealth, but rather in terms of the deficiencies of life that this unhealth incurs. Because a young person has a longer life in front of them, then for any unhealth Y, the disutility of health that results from a certain state of unhealth Y must be scaled down in line with the life they would have left to live. Hence it is not only sufficient to take as your starting point an adult that smokes N cigarettes, but rather, you should say that a young person smokes Z cigarettes where Z is a number sufficient to incur a state of unhealth Y, while there is any number of X cigarettes that an adult could smoke that would incur a proportionally larger state of unhealth so that the disutility of health of the young person and the old person becomes equal to R.

    We take that as the starting point instead - for any number Z of cigarettes a young person smokes to generate a certain total disutility of health R over their life, there must be a number X cigarettes that an old person could smoke that would incur the same disutility of health R. If the intent behind a ban is to prevent a young person from smoking Z cigarettes, then this should also justify preventing an older person from smoking X cigarettes.

    That is however not the only adjustment that you can make - in the eyes of many people, preventing young people from doing as they feel like has a lower negative utility than preventing adults from doing what they feel like. Possibly justified by them with reference to the desires of young people being easier to change through external influence, being more prone to short-term views rather than long-term, and other fanciful observations they claim to have made. If you accept this, then the disutility from lack of health in an adult cannot simply be X cigarettes at state Y of unhealth, but rather >X cigarettes, (let's say T cigarettes) enough to incur an even greater health cost to create a positive utility
  • by PMuse ( 320639 ) on Monday November 05, 2007 @03:30PM (#21244993)

    Here's a hint: what tax burden is paid by those younger than 18?
    Answer: About $9085.6 billion + $1.4 billion/day. That's the amount that the generations currently in charge (including me) have spent, but not paid for. We show no signs that we plan to paying it, ourselves, any time soon. We had better hope those kids exercise better judgment than we have and pay that number down.

    Are you suggesting that our banning smoking by 18-year olds is a rational economic response to the minuscule amount of taxes we let ourselves pay? Horse-cookies! If we were rational economic actors, there are a lot of bigger-ticket items we would have eliminated first.
  • by drig ( 5119 ) on Monday November 05, 2007 @03:53PM (#21245321) Homepage Journal
    I know this is sort of off-topic, but here's my take on the children-smoking issue:

    1) It's true that children have less mature decision making abilities. The ban on children smoking is partially a ban on *advertising* to children. The advertising has a much greater affect, regardless of the volume of advertising. It is more likely to result in a child making a bad decision than advertising directed towards adults affects adults.

    2) The negative effects of smoking on children is more severe than on adults. When a child gets cancer or emphysema, it runs through the system quicker. The resulting illness is much more damaging. This isn't affected by the number of cigs the person smokes. Smoking more increases your chances of getting cancer, but it doesn't increase the damage the cancer does. Being younger both increases the chances and the damage.

    3) Right or wrong (I think wrong), our system of laws has always had 2 standards. A child simply does not have the rights an adult does. This means it's much easier to enact draconian laws that target children than adults. In reality, we should ban smoking for all citizens, or allow it for all. But, we also recognize the rights of adults to make self-damaging decisions is much more broad than that for children. So, it's not that the laws devalue the adult, but that enacting similar laws against adults would come up against resistance by the ACLU and others.

    4) Most of these laws were made before children were recognized as a valuable demographic. If we tried to make it illegal for children to smoke today, the tobacco companies would resist much more forcefully than they did in the 1970s when these laws were made.


You will never amount to much. -- Munich Schoolmaster, to Albert Einstein, age 10