Tech Companies Draw on 'Wisdom of the Crowds'

An anonymous reader writes "News.com is carrying an article on a 'mini-conference' held at Yahoo's HQ this past Wednesday. The get-together put representatives from Google, Microsoft, Yahoo!, and HP together to talk about their experiments with predictive networks. The 'wisdom of the crowds' allows these companies to make use of the collective knowledge their employees hold to answer important questions for the company." From the article: "David Pennock, a principal research scientist at Yahoo Research, said the company has created a currency called a Yootle. It's described as a 'scorekeeping system for favors owed.' Pennock offered as an example a programmer offering to write a piece of code for a few Yootles. Or, when organizing a dinner outing, one employee could use an internal SMS tool to bid 2 Yootles for Italian and 4 Yootles for Mexican. 'If you don't get to go to the restaurant you want to, you get compensation' in Yootles, he said. Related to Yootles is Yahoo Research's experiment with a fantasy prediction market for technology called the Tech Buzz Game. It's a modified version of software licensed from NewsFutures in conjunction with O'Reilly Media and features topics like Atlantic hurricanes and portable media devices. Winners are those who predict how popular a topic will be on Yahoo Search. "

I think this has a name

That sounds remarkably similar to the indifference vote [paulbirch.net] that Paul Birch likes to promote. You bid for your preferred option until the money you allocated toward it would make you indifferent, and if you don't get your way, you are paid that much (instead of paying that much). If you do get your way, you pay your bid. I makes you, in essence, indifferent to the outcome. His example:

Let's try a simple example. ... You and your friends want to go to a restaurant. But which one? The Peking Duck or the Spaghetti Italiano? Charles prefers Chinese, but you're a bit strapped for cash and Italian's cheaper. You bid 50p. Charles goes 60p. The girls join in. Amy is on a diet and bids 50p for the Duck, but Beth is always hungry and bids 70p for the Spaghetti; the score is now £1.20 for Italian, £1.10 for Chinese. Amy looks at Charles, who goes up 11p to 71p. You decide to bid another 2p. Charles shakes his head. Amy reluctantly adds another 2p for Chinese herself. The final bids are £1.22 for Italian, £1.23 for Chinese. So off you all trot to the Peking Duck. Amy and Charles fork out 52p and 71p respectively; Beth gets 70p and you get 52p. Amy and Charles get the fancier but less fattening fare they wanted; you get enough money to cover the higher prices, and Beth gets enough for a larger helping. The waitress gets the penny left over.

Re:I think this has a name

I prefer the system we use at my office: We stand around coming up with suggestions for 30 minutes. Invariably, every restaurant suggested is shot down by some other member of the group ("I'm not really in the mood for Chinese today..."). Eventually, we all sort of migrate out the door, get into someone's car, and magically end up at a restaurant no one can even remember suggesting, and we eat there.

Re:I think this has a name

The problem with Birch's scheme is that individuals are incentivized to lie about what they want in order to get more money from the group. In the example given where Charles is willing to subsidize a trip to Chinese food, you could extract extra money from him by pretending it's a huge disappointment to you.

For a strategyproof scheme, check out the Vickrey-Clarke-Groves mechanism [wikipedia.org]. Basically, everyone gives weighted votes about something, and the winners of the tally pay a penalty equal to the imposition they caused the rest of the people. Only trouble is, this penalty has to be completely discarded to prevent hyjinx...

I bid $10 for "eat out of a dumpster".

Damn, it looks like I lost again. Well, looks like everyone else is paying for my lunch again.

Yes, it is an extreme example, but it shows how you can "game" that system. Not a good idea.

Color me a tad cynical

"Or, when organizing a dinner outing, one employee could use an internal SMS tool to bid 2 Yootles for Italian and 4 Yootles for Mexican. 'If you don't get to go to the restaurant you want to, you get compensation' in Yootles"

Alex I'll take "most retarded use of the Internet for $1000".

It's amazing poeple could go out to lunch harmoniously for decades prior to this stunning reveleation.

No wonder Yahoo is not relevant these days.

I am surprised it took this long...

I am surprised it took them this long to implement as a similar project was implemented at select federal intelligence agencies through DARPA funding back in early 2002 to evaluate possible intelligence leads and threats to national security. Unfortunately the Total Information Awareness program developed out of this work and the true benefits of predictive networks using human intelligence have not really panned out due to an almost pathological reliance and worship of technology supplanting human intelligence rather then supplementing it. Only more recently have projects based on simple, yet tremendously technologies such as wikis been gaining more traction.

Re:I am surprised it took this long...

I am surprised it took them this long to implement as a similar project was implemented at select federal intelligence agencies through DARPA funding back in early 2002 to evaluate possible intelligence leads and threats to national security.

You may also recall that a particularly bad round of reporting on some related work (wherein people in the defense/intel world were "gambling" futures on which head of state, for example, would next come under attack from within, etc) resulted in headlines like "Government Officials Place Assassination Bets." They actually had to shut that one down because the media idiots got enough people to make congress creatures uncomfortable. I hope they just moved the research out of the bright lights and kept it up, but it just goes to show you that these slightly odd-seeming areas of research can be wildly misinterpreted by people who get all of their interpretation in 10-second sound bites. Um, or slashdot summaries.

Re:I am surprised it took this long...

Have no fear as "betting pools" work *has* continued and they are proving themselves to be very effective. Also, you are quite correct about your assessment of "odd-seeming areas of research", as they often, particularly in a basic research sense turn out to be incredibly valuable. One classic case was a member of congress bitching about the NIH funding studies to examine bird songs without knowing or understanding some of the neuroscience implications of that work.

Asimov's Foundation series discussed this in 1950s

'You can't predict what any one person will do but you can predict a crowd's actions.' - a rough paraphrase of Asimov

scorekeeping system for favors owed

My little sister always managed to keep score in our house without the use of technology. ("I took out the trash for you one day three weeks ago.") The rest of us (even Mom and Dad) found it really annoying... bordering on petty and selfish, and we're all glad she grew out of this (mostly). Doing favors for people shouldn't involve keeping score.

Re:scorekeeping system for favors owed

On the other hand, one should never perform labor for others for free or without expecting something in return.

What a cold and empty approach to life. Sounds like you don't have "friends" of even "family"... just "business associates".

However, there is nothing 'selfish' about expecting and/or demanding compensation.

No, that's pretty much the definition of it: what matters to you is yourself, not the needs or wants of others.

Yootle domains: Get 'em while they're hot!

A quick whois at gandi.net [gandi.net] shows that while yootle.com and yootle.net are taken, you can still get .org, .info, and several others.

That whois also reveals something else -- Yahoo! didn't get the .com and .net, as far as I can tell. You would think they'd have thought of that before announcing the name of their new online currency... checking Domain Tools' whois [domaintools.com] shows that the .com has been registered since 1999, and the .net since 2005. Neither is an active site.

Brilliant

For an example of the wisdom of the masses, just look here [slashdot.org]. If that doesn't convince you, I do not know what will.

"Yootles?"

Who came up with that name, My Cousin Vinny? [indiana.edu] "Two yoots..."

Gaming the system

So you just always bid for a restuarant that nobody, under any circumstances, will ever actually want to go to. When you actually *want* to go somewhere in particular, you can outbid anybody else.

Anyway, my point is...this is great, except human nature will always win out. The system only works if people participate. To get maximum participation, you need some sort of incentive. As soon as there's incentive, people will figure out a way to game the system.

Yootle is a lame name

I think Yootle is a stupid name for a unit of fake currency. How about... hmm... hey I know, "Flooz"! No wait, even better, "Beanz"!

More info on prediction markets

Prediction markets are a major interest of mine. I'm in a bit of a rush at the moment, so I'll have to make some more extensive comments later, but in the meantime here's some neat links on prediction markets:

* Tradesports, a real-money prediction market on political and news events. The 2008 president market [tradesports.com] currently gives a Democratic a 50% probability of winning the White House in 2008, Hillary Clinton a 55% probability of getting the Democratic nomination, and John McCain a 49% probability of getting the Republican nomination.

* Futarchy [gmu.edu], a system of government semi-seriously proposed by Robin Hanson which would use prediction markets as a means of government decision-making. People would vote on values, and use a prediction market to determine the optimal government policies to achieve those values, which would help get around some of the godawful stupid things democracies tend to do.

* Storage Markets [storagemarkets.com], a real-money (but limited access?) market on the computer storage industry

* The Policy Analysis Market [wikipedia.org], a proposed prediction market for policies in the Middle East. It was IMHO a great idea, and could have potentially prevented some of the stupid decisions which have been made in the Middle East. Unfortunately, the government ended the project after it was the media (including slashdot [slashdot.org]) had a knee-jerk reaction to it and demonized it. The funny thing is, after the project was cancelled and the media learned more about it, coverage of the project became much more positive.

I predict this story will be..

.. 23 popular

Wisdom of The Crowds

Indeed...

The companies mentioned have some very smart people working for them. It's a shame the PHB's pretty much kill whatever innovation is happening in the belly of those beasts.

The wisdom of the crowds is frequently spoiled by individuals that game the system. Microsoft astroturfers on /. is a good example. So-called climate science coming from the U.S. Gov't that doubts global warming is another. As a former Tech Buzz Game player I can tell you from personal experience the game was stopped and restarted with new rules because of cheating.

The end result is the wisdom a crowd was supposed to provide essentially evaporates.

Yootles?

I'm sorry, but I fail to see the wisdom in a name as gawd-awful as "yootles"! I mean, you wouldn't expect to see some guru walking around and complaining that his/her existential theory of quantum physics and intracellular electromechanical equilibrium & interstellar space travel (IANAG - I am not a guru in those regards) was upset by not having enough "yootles" to buy a cup of coffee. Yootles are yet another substitute for good old hard cash?

Ok, maybe it's a little bit interesting, but seriously folks..

-6d

It's pretty clear...

that yahoo is going down the shitter at top speed. Get out now before you pay check comes in yootels (or whatever) rather than dollars!

Wisdom of the crowds = part of the long tail?

I've been playing with a collaborative filter engine called CRITEO [criteo.com] that is completely blowing my mind in how it opens opportunities to gain that "wisdom of the crowds" bit for the average user -- not just huge companies like Amazon or these emergent venture capitalized corporations. Over the past 2 weeks I've been working on some Wordpress code to actually integrate this relevancy predictor (my results should be forthcoming by the first week of January) and it really seems like you NEED a predictive filtering engine to utilize the crowds to give each individual within the crowd relevant results as compared to just generic "ratings."

This Yootle system is interesting, but it doesn't go far enough. Just because the crowds skew towards a majority opinion doesn't mean that opinion is relevant to the majority (I know it sounds weird). Each individual will have certain likes and dislikes within that majority opinion. Without some sort of relevancy predictor, the "majority vote" is useless.

Hopefully we will see more people utilizing systems such as CRITEO's to actually take the input of the masses (thousands, millions, or even billions of decisions and ratings) and run them through a real-time engine to give everyone a unique view of what they might want/need/like/hate/etc. As I spent more time beating on trying to come up with my own quick/real-time solution, the more I realized that using someone else's services let me focus on what is best for my customer -- my content, generally.

The prediction system to rank Yahoo searches is very 2005 -- it really just capitalizes on the likes of the masses, which means it is hitting the top head of the long tail rather than the more important remaining 80%. I'd love to see a search engine that allows you to "rate" your search results or even individual search results in real time, maybe in collaboration with a system like CRITEO. Anyone interested in working on one? I'd be willing to bet that such an investment of time would give many of us a better search engine that actually returns results that are relevant to the individual's tastes rather than the masses' collective "favorites" which are usually way off base. It would also reduce the spam results greatly and open the door to the wisdom of the masses actually making a difference for each individual. What I like about collaborative filter is that 5 seconds per user can mean days or weeks saved for that user in the long run because of the 5 seconds "donated" by the million others.

Foresight Exchange

Just a reminder, as posted in many many slashdot posts before (see http://www.google.com/search?hl=en&q=site%3Aslashd ot.org+ideosphere [google.com]), the http://ideosphere.com/ [ideosphere.com] site has been running a (non monetary) experiment in freemarket intelligence for a long time. The more players the more fun (and better information extractable from the market). Go play. -math

Wisdom

As several authors have put it - the intelligence of a crowd equals the IQ of the least intelligent member divided by the number of people.

Beware: there's another phrase

The madness of mobs.

Yahoo Yootle...

Maybe they should learn Yiddish [wikipedia.org] to find more colorful words?

Wisdom

What does the Yootle have to do with a wisdom of the crowd concept? It has no predictive aspect, only a petty score keeping of how much work you do and how little you get your way.

Not a new concept, but loads of new applications

I've always been interested in ways of tapping into the crowd as a resource.

Slashdot and Digg [digg.com] got a lot of attention as news filters, but these things are now being used everywhere. Trusted Places [trustedplaces.com] for restaurant reviews, Crowdstorm [crowdstorm.com] for shopping recommendations, wine sites, health sites, etc., etc. I can't wait to see where this is all headed. What's the next logical step?

I can't be the only one

who reads that and immediately concocts a plan to rig the system.
Take for example the restaurant example. I may not be too bothered which one of those two we go to, but if I do choose the most vile restaurant I can think of and make that my choice, then I'll still get to eat where the most yootles wanted to go, but I get given Yootles as well.
This works for a while, until more people twig and junk on the bandwaggon - eventually nobody'll come out with a net-yootle amount - and you'll all end up eating in the foul restaurant.

Also...

Is that Yootles with stars or Yootles without stars?

Was Dr. Seuss the keynote speaker?

Wisdom of the Crowds?

Obligatory demotivator reference: "None of us is as dumb as all of us" http://despair.com/meetings.html [despair.com]

Microsoft believes in community efforts?

Isn't this the same company that scoffs at similar situations that result in source code being written, re-written, etc For example, open source which results in Linux? It's a bit of an irony.

Although, they've done this many times before. Take part of something well respected and has community involvement (Java, etc) and then consume their efforts later. I'd be very careful what I share with Microsoft. Not because I don't like them, but because of a very long track record.

Other than that little tidbit, it sounds like a great idea. I try and get my sister companies to help out each other, but everyone is always so focused on their own piece of the pie. Harvard Business Review (December 06) had an interesting write up ( Managing the Right Tension) which talks about the competing priorities of making collaborative efforts that help the community but hurt your individual unit. They use it in the sense of cross-organizational groups, but it's still a concept you could apply to industries. How often do you seen companies move on an opportunity to help the whole industry at their own expense?

"Bidding" system is kindof silly

What if there's a tie? Say my mother, who has horrible musical taste, wants to listen to Mike Reid. Say I want to listen to Pink Floyd. We both bid the exact same on each artist/group, and so we end up listening to Iris DeMent. Who wins? Nobody, 'cause now we have to listen to utterly crap music. Besides, calling your currency the "Yootle" is just _begging_ for 4chan or Fark to turn it into some kind of horribly unpleasant sexual innuendo.

Wisdom in signatures

My signature defines the true wisdom found in crowds. Moo

I prefer Quatloos

Right now the exchange rate is much more favorable for Quatloos. Most places that take the Yootle also take Quatloos anyways so it's not much of a problem.

Crowds do not possess wisdom...

They only possess mob crew behavior.

An humble suggestion...

I don't know; I think stupidity is a much more powerful force. Is there some way we could draw on the stupidity of crowds? In warfare, for example, you could draw on the stupidity of crowds, and when you "bomb somebody back into the stone age" you might do it without loss of life or property.

-Loyal

Hmmm

You know who would take this over? Prostitutes. As far as I know it's perfectly legal to exchange sex for Yootles...Not to mention improving the exchange rate by throwing in a little *coughs*something extra....and as no money is exchanging hands, that would get them out of entrapment. Where did that come from you ask?I don't know, but there it is.

What's the value of a yootle?

I ask because it takes a certain amount of physical time and work to perform a favour.

Anyway, who destroys the yootles? If you keep creating them but they're never destroyed, they'll become worthless very quickly indeed. The yootle inflation will be huge.

 

Whuffie

Sort of like Whuffie [wikipedia.org] from Down and Out in the Magic Kingdom [craphound.com] although that was based on general goodwill and esteem, not specific favors per se. Wiki's definition is probably better:

Whuffie is the ephemeral, reputation-based currency of Cory Doctorow's sci-fi novel, Down and Out in the Magic Kingdom. This future history book describes a post-scarcity economy: All the necessities (and most of the luxuries) of life are free for the taking. A person's current Whuffie is instantly viewable to anyone, as everybody has a brain-implant giving them an interface with the Net.

Wisdom of Crowds

Whenever I hear the phrase, "Wisdom of Crowds," I think of lemmings.

Predictive networks and markets reflect consensus

Predictive networks and markets may reflect the "wisdom of crowds", but this "wisdom" is just the current consensus view. There are times when the consensus view is very wrong, even when it comes to predicting what masses of people will do.

For some time leading up to the 2006 mid-term elections in the United States I followed the Intrade and Iowa Electronic markets which were real money futures markets for, among other things, the US House and Senate races. The idea is that because these markets attract thousands of people and because they trade real money futures, they may be accurate predictors of future political events. This speculation has been born out in the past.

Leading up to the mid-term elections the Intrade and Iowa Electronic markets had more or less the same prediction: that the Democrats would talk control of the House. But the chance given for the Democrats taking the Senate was only about 33%. There were a couple of close elections (Webb in Virginia and Tester in Montana), but the Democrats ended with with a very thin majority in the Senate. In retrospect the political markets seem to have underestimated the wave of discontent among voters.

The "wisdom" of crowds can be useful, but it is not always accurate, as the last election showed. People naturally tend to think that tomorrow will be like today, which was like yesterday. In general this view is correct. But extreme events take place (market crashes and extreme economic declines or bubbles or even wars). The "wisdom of crowds" is very bad at predicting drastic change. The "crowd" just goes along placidly while extreme events lurk on the horizon. In fact, if anything crowds tend to ignore evidence that change is coming, until the change is upon them.

Yootles?

Yootles? I thought it was called Whuffie [wikipedia.org]?!

What's Whuffie you ask? Well, click the link, and if it sounds interesting, go here [craphound.com] and download the book Down and Out in the Magic Kingdom. (it's free)

Yootle Compensation Fairness Critical

Yootle based exchanges had best be fair, or else expect problems as demonstrated by scientific research [upliftprogram.com].

Community currencies and 'yootles'

We've had one of the proponents of yootles (I'm not going to embarrass him by name) on the International Journal of Community Currencies (ijccr) mailing list (on Yahoo).

All I see was/is an attempt to create a currency to monetize favours, which (IMHO) ought to be freely given in an elightened society anyway. This is probably an attempt to talk up something that is not to do with the 'wisdom of crowds' anyway.

For anyone interested in community currencies and with a little patience, I'd suggest the above list and http://www.le.ac.uk/ulmc/ijccr/ [le.ac.uk] the journal itself.

Re:Extrapoling this + DRM

Sorry, Henry Ford [wikiquote.org] already has prior art on that concept...