Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
AI

Creating a Better Chatbot Through Crowdsourcing 49

Posted by samzenpus from the people's-bot dept.
An anonymous reader writes "MIT Technology Review reports on a chatbot built at the University of Rochester that is capable of high quality, human-level conversation — thanks to software called Chorus that turns to Amazon's crowdsourcing service Mechanical Turk to generate and evaluate replies to a human's statements and questions. No one person is ever acting as the bot, instead multiple workers suggest responses that are then voted on to select the best. The crowd workers contributing change frequently, but Chorus also has them keep a running list of important contextual information to give the bot a kind of memory of a conversation's history. The researchers say Chorus-style chat bots could out-perform fully automated assistants such as Siri, while being considerably cheaper than a true concierge service."
This discussion has been archived. No new comments can be posted.

Creating a Better Chatbot Through Crowdsourcing More

Creating a Better Chatbot Through Crowdsourcing

Comments Filter:

  • BREAKING NEWS (Score:5, Funny)

    by Anonymous Coward on Monday September 10, 2012 @04:33PM (#41293349)

    Humans better at chatting than computers, and by replacing computers with humans, you can almost have a natural conversation.

  • crowdsourcing != AI (Score:4, Insightful)

    by metageek ( 466836 ) on Monday September 10, 2012 @04:36PM (#41293393)

    Good try, but even if this passes the Turing test, it is not AI in any way. Responses are by humans and there is no intelligence in it. So it will be the collective human engine behind it that will pass the test. Not really any big achievement.

    • Mod parent up. (Score:4, Insightful)

      by khasim ( 1285 ) <brandioch.conner@gmail.com> on Monday September 10, 2012 @04:43PM (#41293469)

      And if this ever takes off you know that it will become a game of trying to submit/vote the most inappropriate responses to the questions. A never ending battle between the censors forbidding words and phrases and the people finding new ones.

    • Re: (Score:1)

      by Anonymous Coward

      The summary never mentions AI or the Turing test at all so you're attacking a strawman.

      It's not like they're claiming to have made an AI breakthrough or something. They're just saying "I made a chatbot out of humans. Perhaps it may be of some use."

      • If you follow the link on the summary you find an article with title "Artificial Intelligence, Powered by Many Humans"

        • If you follow the link on the summary you find an article with title "Artificial Intelligence, Powered by Many Humans"

          That about sums up my daily experiences...
          Everywhere I go there are many humans with an overwhelming lack of real intelligence.

          At least someone has finally found a use for their collective stupidity such that the whole is greater than the sum of the parts.

    • There can still be some value in this for advancement of AI and hybrid systems. They decompose the problem of keeping up a conversation into nice simple subtasks with clear interfaces. Some of these subtasks (suggesting replies, evaluating them, keeping notes) can then be further automated or assisted independently to a varying degree, gradually reducing the use of human brainpower. Also, there can be uses for adding such crowdsourced conversation support into otherwise automated systems. Perhaps entertainm

    • Good try, but even if this passes the Turing test, it is not AI in any way. Responses are by humans and there is no intelligence in it. So it will be the collective human engine behind it that will pass the test. Not really any big achievement.

      Totally agree.

      What's the point of this? Is it to make a better chatbot? Do chatbots have any use beyond annoying chatroom participants and surreptitiously inserting ads?

      Is it to study human language? Will useful insights come from this? Will there be results? A publishable algorithm? Will this inform future software packages?

      Does this tell us something about AI? Can this bring us closer to machines which actually think?

      I'm not even sure there is a purpose here. I don't see any academic value in this project

  • I can't imagine anyone fluent in the language being willing to work for the prices that could be paid. We're talking pennies per hour.

    • Re: (Score:1)

      by Anonymous Coward
      But I your question can answer good! Pennies please do be sending to me. My starve childs you help?

    • Comment removed (Score:4, Interesting)

      by account_deleted ( 4530225 ) on Monday September 10, 2012 @05:13PM (#41293829)
      Comment removed based on user account deletion

      • Even if the money offered was enough for people in the Third World, surely people with the English language skills required could find something better. It's unclear to be just what demographic MTurk is depending on now.

        They're mostly doing it for ratings these days, not actual money.

        I'm surprised a highly rated Slashdot user such as yourself didn't pick up on that.

  • Cleverbot isn't AI as it's claimed, and I think it's interesting how they try and pass it off as such. Maybe there's a bit of AI there, but from what I've seen it just connects two random users and then reshuffles every so often so you're getting responses from some other real person than the one before...
    • It feels that way since it has only like 2 lines of conversation memory/context, but to be sure, it is recording answers. If you repeatedly ask the same question, you can get the same answer twice. You can also check out things like its extensive knowledge of Pokemon. Random users are only going to occasionally know how to respond to Pokemon queries. Cleverbot pretty much always does.

      • oh, come on!
        EVERYONE played pokemon when we were young, when there were only 151...

      • Re: (Score:2)

        by Klinky ( 636952 )

        I don't know about that...

        Me:What are the names of team rocket?
        Cleverbot: Sufian Stevens and Elvis Prestly.

  • Kinda Like ThinkMosaic chatbot (Score:1)

    by Anonymous Coward

    I haven't made mine live as I keep tweaking the code but I've made many generations of chatbots. The concept I wanted (and still want) to open up to all is ThinkMosaic. It doesn't have the vote for the best response.. More like, chat with it and then if you don't like the response then edit the logic by which it chose that response... Basically, there is a user statement that is best-matched to a "recognizer" (a text pattern with named wildcards optionally in it). This identifies a "meaning". Each mean

    • Re: (Score:2)

      by msauve ( 701917 )

      The requested response format ("A system error occured. No response is possible until the error has been fixed.") is not supported.

      Hey, it's working great! I have people use that line as a conversation starter all the time.

  • "I feel like some Mexican food." - 10 minutes later: "40% of us say that there should be a Taco Bell somewhere around. Just look for the bell sign"

  • Of course chatting with real people will feel like chatting with real people. How is this different from, for example tech support where you get connected to lots of random people?

    • How is this different from, for example tech support where you get connected to lots of random people?

      You don't have an unintelligible accent when you type?

      • You don't have an unintelligible accent when you type?

        Um, hefe-a yuoo nut met zee Svedeesh Cheff? Hees vreettee vurds steell hefe-a qooeete-a un eccent. Bork Bork Bork!

  • Probably gets similar results.

    • Re: (Score:2)

      by arth1 ( 260657 )

      Yes, I think so too.
      Remember the old Demotivator(TM)? "Because none of us is as stupid as all of us".

      I also think the idea that this is cheaper than a concierge service is ridiculous. Instead of having to pay one person to answer your question, you have to pay many.

      And even worse, there's no "conversation" as such, because the people answering aren't necessarily the ones who provided the previous output. There's bound to be erroneous answers or WTFs due to a lack of continuity.

  • Just think: If this technology lives up to its promise, Slashdotters fighting in the great OS Flamewar could create their own automated sentries to fight back with phrases like "reality distortion field' and "rounded corners". Maybe we can finally end all this bloodshed!

  • Over in IRC, one of the denizens of the channel I frequent has neen running a chatbot that snarfs conversation from all over IRC.

    It is like Siri, but drunk and insane, and knows all your secrets.

    00:18 ? ascaris
    00:18 Ascaris lumbricoides, or "roundworm", infections in humans occur when an ingested infective egg releases a riot on facebook

    20:57 bmo, what is the survival length of cats in yogurt
    20:57 I don't know, I never tried

    --
    BMO

  • This may be a good example of a hive mind implementation. The first being the church, and the second was the corporation. The difference is that a machine might be the parser, instead of another person. There was talk of this kind of machine logic being used by companies in the form of schedulers, automated personal assistants, and management analytic engines -- all based on business policies. Once interconnected, they form a collective intelligence that drives the workers, probably like DNA drives a co
  • That which has both Cybernetic and Organic components... Their "Chatbot" is a Cyborg.

  • while being considerably cheaper than a true concierge service

    A "true concierge" is an experienced individual who knows his clients and can perform all manner of complex tasks requiring intelligence.

    Neither Siri nor choose-from-random-trollish-human-responses form a concierge service.

    It's like pointing out that a vacuum cleaner and a washing machine are cheaper than a butler.

  • I guess it's old news, but this sounds exactly like what was being described in Neil Stephenson's Diamond Age. Actors there were paid to read/act short pieces of text/commands to reply to a young girl's questions. In the story, the girl was asking a book to explain a concept to her. Not much different from what happens with a chatbot.

    I guess this might also relate to the earlier post on online math courses. Presumably grad students could be given micropayments to answer specific questions for an online

Slashdot Top Deals

The truth of a proposition has nothing to do with its credibility. And vice versa.

Close