Forgot your password?
typodupeerror
The Internet

Online Reputation Is Hard To Do 224

Posted by kdawson
from the to-say-nothing-of-trust dept.
Symblized writes "A new article from InformationWeek argues that not only does the Web need ways to verify identity, it also needs better ways to measure reputation . The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely. There's also a profile of Opinity, a company that tried to introduce a reputation system and didn't make it. Choice quote from a source in the article: 'The idea of a transferable, semantic reputation is identity nirvana.'"
This discussion has been archived. No new comments can be posted.

Online Reputation Is Hard To Do

Comments Filter:
  • by Raindance (680694) * <johnsonmx@@@gmail...com> on Sunday June 03, 2007 @05:20PM (#19374361) Homepage Journal
    "Trust is the currency of the participation age." -Jonathan Schwartz

    This is the $64,000 question. Building a reputation/trust system is very difficult. Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

    I'm volunteering at Citizendium, which is another possible datapoint. We're assuming that real names and respecting verifiable expertise will allow us to benefit in some fashion from existing scholarly reputation systems, and to build a more cohesive community.

    Eventually, I think it'll be feasible to layer reputation and credentials (for sites that care) on top of a system like OpenID. People will be able to choose what reputation/credentials to share with which site. Information that you want to follow you (e.g., "I have a BA in Math from UCLA" or "I have excellent karma on Slashdot") will follow you across sites.

    But yeah, it's a very difficult problem. Figuring it out is a big, potentially very lucrative issue.
  • by lawpoop (604919) on Sunday June 03, 2007 @05:26PM (#19374417) Homepage Journal
    I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles. I haven't kept up with the structural changes they've made in the past couple years, but a lot of the editing work seems to be undoing trolling and vandalism, and also participating in edit and revision wars. I could be wrong at this point.

    But if wikipedia had a reputation system ( other than just being banned or allowed ), they might automate contributions from reputable authors ( and check on the actual contributions later), while authors who are less reputable may have their contributions queued for review before they are published.

    Furthermore, a casual user would be able to have a more savvy understanding of the reputability of any article or section of an article if it is tagged with the reputation of its' author.

    Reputable authors might be able to also tag the contributions of others, such that the text or information itself gets a reputation. That would help users make a judgement about the validity of information on Wikipedia.

    Instead of pushing the mechanics of the actual editing of articles behind the scenes, and just presenting a 'final' article to the end-user, let's formalize the process and enfranchise users into the process of judging the validity of articles.
  • by FoxNSox (998422) on Sunday June 03, 2007 @05:44PM (#19374575)
    Even still, it is hard to rank someones reputation based on a numbers system. In alot of forums I post on, I am a regular poster with a high rank. However, alot of people have an issue with me because of my free speech (the beauty of the internet). Is there really a standardized way to determine reputation? It really has to do with the context. If you are on a programming forum, you may rank someone based on their aptitude for a specific language, or their problem-solving skills. Conversely, if you are on a political debate forum, ones reputation may be based on how fluently their opinions are expressed.
  • by khasim (1285) <brandioch.conner@gmail.com> on Sunday June 03, 2007 @05:52PM (#19374649)
    It seems that everyone trying to "solve" this "problem" doesn't know what they're trying to achieve.

    So what if you can make a perfect pseudonym identification system? What does that achieve for you? What do you accomplish beyond that?

    Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?
  • by Anonymous Coward on Sunday June 03, 2007 @06:00PM (#19374707)
    I don't align politics-wise/religion-wise on this site, so while I have excellent Slashdot karma (built up over a very looong period of time commenting on non-sensitive topics for this audience), I wouldn't otherwise be considered having a very good reputation here.

    There's that plus many sites like these are mostly just kids playing around and/or just mouthing off saying any ridiculous thing because they can. And practicing to be good little enforcers of Political Correctness for when they get older.

    So, an online reputation based on sites like this at least, is valueless. The value of a reputation is only as good as the people who judge it are worthy to judge.
  • heatware (Score:2, Insightful)

    by mscdex (774392) on Sunday June 03, 2007 @06:09PM (#19374757)
    Heatware [heatware.com] Most frequently trusted on many For Sale/For Trade forums because of their strong stance on scammers.
  • Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?

    It might. I think the best possible solution would be to let the destination site look at whatever pieces of reputation you choose to provide, and then weight them however it wants to.

    E.g., let's say I set up a blog. I know most traffic is driven to it by people clicking on my site's URL in my Slashdot comments; therefore, it might not be a terrible metric to utilize Slashdot-karma in approving comments on my blog. There's no guarantee that someone with Excellent /. karma isn't a spammer, but I suspect it's a better chance than "random person on the Internet" being a spammer.

    That said, there's no objective reason why Slashdot karma is more valuable than some other kind of Karma (Digg karma, whatever); it's purely a subjective decision on the part of the destination site, deciding what forms of "reputation currency" they're willing to honor. If a site was perceived as being easy to game, then its karma wouldn't be valuable elsewhere. Likewise, credentials that are hard to fake, hard to obtain, and easy to verify (if there is such a thing), would be worth a lot, probably at many places.
  • by mysidia (191772) on Sunday June 03, 2007 @06:35PM (#19374961)

    For one thing, it says you are probably not a spammer; if you have a very good karma on slashdot. (You didn't just run by slashdot to post a piece of junk)

    For the reputation system to really be useful for filtering purposes, a method needs to exist for your reputation to become "sullied" if you do spam somewhere, or abuse your reputation; in that case, your "reputation of good karma" on the pseudonym@SITE would need to have the bad part attached to it, I.E. if you spammed on some other site, and used your "Slashdot karma" as a credential to establish a special account, then the record of your spammage should become tied to the special credentials you needed to use in order for someone to trust you.

    And the bad part of your reputation would need to follow you if you create a new account on the site and link any of your old pseudonyms to this new account on the "reputation tracking" site.

    I see in the early posts suggestions that people should be able to pick and choose what they want to share. This is no good, for the reputation information to really be useful -- the system can't just let you pick all the good things and hide all the negatives.

    For information to really be useful for others to make decisions (about whether to trust a certain pseudonym for a certain action or certain transaction) it also needs to be impossible for you to start from scratch and create a new reputation and link it to only your pseudonyms that have a good rep.

    I think the way this would work is you link a number of pseudonyms on other sites to your profile. If one of the pseudonyms later spams, that "bad reputation link" will follow you if you link one of your old pseudonyms to a new account.

    A good reputation system also needs to be resilient against people maliciously/frivolously attaching bad marks to other people's reputation, for example: claiming person X is a spammer, who never actually spammed. I.E. it needs a method of verifying or authenticating any credentials and negatives/qualifiers on those credentials, that get recorded and presented. One possibility is to require corroboration, and to quickly expire negative reports that are not confirmed by a sufficiently trustworthy source.

  • by khasim (1285) <brandioch.conner@gmail.com> on Sunday June 03, 2007 @06:39PM (#19374987)

    it matters because if you have good rep on slashdot, chances are you're not a complete mumpty.

    So you're saying that it would help filter out a majority of the "complete mumpty".

    That's a possibility. But it would be even easier to just use Slashdot's reputation/moderation system on your own site. That would solve the "complete mumpty" problem while also solving the problem of someone with excellent karma for his programming knowledge posting his conspiracy theories on your site.

    And it automatically tunes itself to your audience.

    It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too.

    Not really. Check back on the "creationism vs evolution" stories here.

    What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

    You achieve all the same benefits without the problems just by having your own reputation/moderation system.
  • by zCyl (14362) on Sunday June 03, 2007 @06:46PM (#19375041)

    Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

    That's because Slashdot's system puts only minimal emphasis on individuals, and very high emphasis on selected adjectives of value. Comments do not simply get moderated up or down, but have to be moderated with a chosen adjective, such as "insightful", "informative", "funny", etc. This really helps keep people's heads on straight, especially with the presence of meta-moderation, because people then have to agree on what these words mean. The end result is that posts are usually moderated in close proximity to these labels.

    The karma attribute is used only as an accessory to this content-based moderation, to provide some inertia to the community's character. It's not really a reputation centered system.
  • by Colin Smith (2679) on Sunday June 03, 2007 @06:55PM (#19375093)
    It's why we have exams, professional organisations, CVs, brands, social networking etc etc etc.

    We use reputation all the time and no-one has come up with a single reliable, coherent way of measuring it. You just try to get a decent builder.

     
  • by Original Replica (908688) on Sunday June 03, 2007 @07:33PM (#19375351) Journal
    In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. Just like if you act like an asshat at the company picnic, it effects you back in the office and possibly gets back to your friends at home. And yes, I think if someone is a carebear in WoW then they are a more trustworthy eBay seller, and someone with intelligent /. posts is more likely to contribute intelligently to Wikipedia.
  • by dkf (304284) <donal.k.fellows@manchester.ac.uk> on Sunday June 03, 2007 @07:38PM (#19375387) Homepage

    What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

    You achieve all the same benefits without the problems just by having your own reputation/moderation system.
    I suspect that there will be a lot of correlation between sites, as people who act like asses in one place are more likely to do so elsewhere. On the other hand, I also think that what will actually emerge is that the sites that correlate cluster in groups: if you have a good rep on site A, you're more likely to have a good rep on site B and a bad rep on site C, and you probably won't frequent D at all. Given all that, someone with a good search engine and a lot of cleverness will be able to start mapping out what these virtual communities actually look like and other sociological/anthropological stuff like that.

    But will sharing reputation systems help sites? I think so, eventually, but not for many years yet. Until the reputation clusters have been found, sharing is as likely to introduce needless pollution of reputation systems as it is to enable reputations to be built up quickly. (Could there be a single reputation cluster? Maybe, but I suspect not; people are too inclined to divide the world into "us" and "them" for it to work out.)

    One thing that might come out of reputation research is that it might become possible to use the reputation clusters to predict, from someone's interests and interactions, which sorts of sites they'd like to visit. OK, that does sound backwards, but it should guide people to where they won't want to make a total fool of themselves on a regular basis (yes, even the griefing pranksters; after all, when amongst the fools the foolish are sages and the wise foolish.) It may also eventually be possible to join the reputation clusters up, but using negative links (so reputations on sites for followers of Xenu who believe in ID and the supremacy of feng shui of placement of feeding bowls for their chihuahua will negatively reinforce reputations here) but I doubt that will help any time soon. There's a revolution waiting to happen here, but since it really involves lots of people, it'll take time to brew.

    On the other hand, it is sensible to start working out what technological steps are required to enable specific bipartite reputation sharing, as well as looking at how to build sane single-site reputation systems. For example, slashdot's is pretty good in that it isn't easy to totally game the system while being mostly self-regulating, but can it be bettered without input of data from outside sites? If it can't be greatly improved, how difficult will it be to export the system to other sites? (It's late: I'm sure you can think of other aspects, but I can't right now.)
  • Well, some of us have been lucky that we've managed to get our own names (or pseudonym in my case) almost everywhere. But some folks don't use quite as unique names.

        I guess if there was a pseudonym reputation ranking facility somewhere, it would need to have every pseudonym for each resource listed. Ahhh, a tall order just got taller.

        But, our pseudonym is such because we don't necessarily WANT everyone knowing who we are.

        JWSmythe (my pseudonym) is kind of well known. I show up all over the place, including one of my own sites (jwsmythe.com). But, when it comes down to it, JWSmythe isn't a real person. The JWSmythe you converse daily with on another site isn't necessarily me. If you were to walk up to me in real life and say "Mr. Smythe?", I'd just give you a dumb stare and ask "Who?", just like almost everyone else would.

        Who cares what a ranking system on the net says about a pseudonym? Some people think I'm great. Some think I'm an ass. Hell, even here, I have an 'Excellent' karma. I have 137 "fans" in my fans list (thanks everyone!), and 10 "freaks".

        You can't make everyone happy all the time. I'd say more so in real life, but people tend to blast folks they don't like even more with the anonymity of the Internet. Me and my keyboard, you can't reach out and bitch slap me. :) But hey, 93% of the folks who bothered to pick a side seem to like me. (and 98% of statistics are wrong.) I can only pick up 10% of the women in any given bar, so there's a big difference in the Internet, and real life.

        (on that, if I were to try, I have a 99.9999% chance of not picking a lovely lady in that 10%)

  • by alphamugwump (918799) on Sunday June 03, 2007 @09:16PM (#19376093)
    Oh, the irony. The utter irony. The one man who speaks truth about moderation, gets modded troll. Fortunately, I score trolls +2.

    Getting excellent karma is indeed easy. I spent a whole weekend on slashdot, and now I have excellent karma. It's that simple. All you have to do is post something slightly different from the usual copyright arguments, gun-control arguments, and so on, and you'll get 5s all over the place. Even better: act like you know what you're talking about. I've made up complete bullshit about security, and the mods ate it up.

    Of course, someone who is capable of karma whoring in a creative way probably IS marginally more intelligent, but I'm not sure I'd want the system to respect someone like that.
  • by jollyreaper (513215) on Sunday June 03, 2007 @09:22PM (#19376137)
    This is a problem with real life. There's a truism along the lines of "you never really know who another person is, you only know what they've shown you." This goes hand in glove with another truism: "perception is reality." Every one of us creates models of people in our life to better understand what makes them tick, how they operate. Once we feel comfortable enough in our understanding of that person we can say things like "Oh, X is going to love this!" or "Just wait until Y hears about this, I know he's going to flip!" But do we really know the person? Not really. We only know what we see. And if we like someone, we tend to gloss over the negatives. After shit happens, you'll get an earful from friends as to why it should have been obvious but they kept their mouths shut beforehand for fear of alienating you. And they were right. Would you have listene to them? Of course, not. Bad relationships, can I get an amen in here?

    I tend to classify people into the following categories: good to have a beer with, someone I would introduce to other people I know, someone I can trust with something semi-important and people I can trust with things that are very fucking heavy. Most of the social drama I see comes down to someone mistaking a casual friend for someone that can be trusted with something heavy. "What, you're upset that Weasel slept in and forgot to drive you to the airport? Why are you so surprised? This is Weasel, the guy with the reputation as the heaviest drinker in the city. He's fun at a party but you actually thought you could rely on him for something more than a laugh? You made the mistake here, not him." Of course, that's exactly the sort of thing you don't say or the drama will shift to you.

    So this is all normal human social dynamics, web doesn't have anything to do with it. People were doing long-distance business with people they didn't know personally for thousands and thousands of years before computers were invented. It all comes down to trust. Does the person have a reputation? Can someone else vouch for him? Someone using a fake or young ID on the net is no different from someone using an alias in the past. And what does it come down to in real life and on the web? One simple question: "Is there enough incentive to fuck me over in this deal for the guy to throw away his reputation?" Because it takes time to earn a reputation, get seller feedback on Ebay, etc. Do people put time into pulling off a proper con? Of course. Just look at the con men who go after little old ladies. This isn't just scamming someone one the net, this is full on interaction, romancing, sleeping with, trying to convince the mark you're legit so you can get the means of fucking her over for the inheritance. But do you think that the high stakes con man is going to try fucking over your grandma for her social security check? I don't think so.

    Even when someone has good reputation, you don't know what he has going on in his life. Maybe something has come up that means something is more important. I've seen businesses with formerly stellar reputations go to pot because the owner has other things on his mind.

    Figure out a way to fix this problem in real life and you'll have a model for how to do it online.
  • by Sigma 7 (266129) on Sunday June 03, 2007 @11:15PM (#19376947)

    In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere.
    I'll play devil's advocate here...

    Let's say you comment on the theory concerning the use of demolition explosives on one of the world trade centers - pointing out that the collapse of the WTC doesn't look like other building demolitions [google.ca], or that the "symettric demolition" claim is incorrect.

    However, the conspiracy theorists on the site are extremely fanatic about their theory (as opposed to a more moderate site that tries to investigate properly.) As a result, you receive a large quantity of negative feedback that attaches itself to your online reputation.

    Other things that can affect you would be playing RTCW:ET, where you get kicked from a server for n00bism as you didn't dodge the three panzers that get fired into your local area (because another player thought you should have.)

    And my personal favourite - just claim you support Bush. Your reputation would instantly tank.
  • by tekrat (242117) on Monday June 04, 2007 @12:01AM (#19377217) Homepage Journal
    eBay's system of who you can trust and who you cannot doesn't work. Generally, even theives have a good feedback rating on ebay, and worst of all, the average joe who isn't an eBay powerseller or doesn't have his own ebay "store" is likely to get shafted because eBay only cares about fees, not feedback.

    For example, a user with a feedback of 1 can buy something from a seller with a feedback of 450, and then complain to ebay. The user of with the feeback of 1 can have the seller's account suspended despite the vast chasm of difference between the seller's feedback and the buyers, and ebay makes NO distinction over which party has the better reputation.

    It all comes down to who files the dispute first. Either that or eBay just assumes the seller is always at fault.

    Ebay's dirty little secret is that you can create an account, buy something, and essentially get it for free from the seller because the only way the seller has to resolve the suspended account is to refund the payment. There is NO other recourse.

    And "reputation" means nothing to ebay. You could have created the buyer's account a week ago, and take down a seller who has been a good seller on ebay for years.

    And when Ebay gets wise to you, create a new email account on yahoo, start again, buy something, register a complaint and get it for free, because the seller has to refund your money.

    So; how exactly does eBay's system "work"?

  • Sucks to be you (Score:3, Insightful)

    by SuperKendall (25149) on Monday June 04, 2007 @01:30AM (#19377737)
    Some of us have posted under nothing but our real names for years, knowing that every post was only adding to our online metaprofile, and if the bulk of what we posted was helpful or useful or at least sensible, we were building up years of easily accessed credentials to help people understand they could rely on information from us a little more than Joe Q. Internet.

    Why you would choose to be anon in a world where reputation is growing in importantance every day has always been beyond me.

  • by Eivind (15695) <eivindorama@gmail.com> on Monday June 04, 2007 @03:31AM (#19378493) Homepage

    Yes it matters. A lot.

    Online, a lot of clues that are present in the real world are absent. We instinctively assign some level of trust to people we meet, for different purposes, based on a lot of variables, some of them we're aware of, others are subconscious.

    You let your neighbour have your house-key to water the flower. But you wouldn't do that with *ANY* kind of neighbour.

    You let someone babysite your kids. You let a friend borrow $50 'cos his credit-card is broken. You wouldn't to everyone, it's a matter of trust.

    You trust someones judgement on some issue -- because you know that they are experts in the field and have a track-record of good judgement.

    Being able to build trust in a pseudonym, and being able to prove that you are that pseudonym is very useful. It allows people to trust you who wouldn't otherwise.

    To avoid abuse, it is nessecary that *you* have complete control over what aspects of your trust you share with which people and which companies.

    So, what do you want to achieve ? World Domination offcourse ! *grin* No seriously, a million little small things, each of which may be unimportant, but the sum could be huge. Some examples used *today* include:

    • If you've got excellent reputation, Ebay-buyers generally won't mind paying first, getting the item(s) afterwards. This would be quite risky -- except you know that the seller has sold 471 things before on similar terms, and -zero- of his customers complained.
    • Hospitality-club use a trust-system to allow you to let complete strangers sleep over at your house, or vice versa, with a much reduced risk of any unwanted problems. Sure, *you* may not know this person, but it helps if 50 other people do -- especially if 5 of those are your friends.
    • Slashdot use a trust-metric to let people with a track-record of sane comments be sligthly more visible.

    In a universally networked world (which we're rapidly approaching anyway) with strong trust-systems, you could stop a complete stranger on the street and ask to borrow his car -- and he'd actually consider it. He wouldn't know *you* but, he'd be able to know a lot *about* you -- if you choose to share it with him.

  • by Original Replica (908688) on Monday June 04, 2007 @08:45AM (#19380575) Journal
    You don't think the fact that you occassionally like to "flame and act silly" or play "an annoying lowbie ganker" is telling ? It tells me that when you can hide behind the veil of Anonymous Coward, you don't have much respect for the people around you. Because let's face it, when you do those things you are causing real frustration for real people, and you do it for fun.

    Don't be so gullible as to think someone's behavior is necessarily consistent no matter where they go on the internet.

    This would be a case of adding "but ONLINE" and thinking it's something new and different. I don't think anyone's personality is 100% consistent as they go from one social setting to another, but it is all facets of the same actual person.
  • Reputation is easy (Score:4, Insightful)

    by richardtallent (309050) on Monday June 04, 2007 @09:19AM (#19380915) Homepage

    Identity is really the easiest part of the problem.

    But reputation is easy as well. The problem with most proposals is that they are focused on organizational reputation rather than personal reputation.

    Reputation, however, is relative and contextual. We don't need Slashdot vouching for us, we need people in our own address book / social network. Then we can vouch for our friends/family in various ways ("this person isn't a spammer", "this person knows a lot about cars", etc.).

    But the real power of a personal reputation system is that it is transitive. If I trust that Alice is not a spammer, and Alice trusts that Bob is not a spammer, I can to some degree also trust Bob, and so can my friends, etc. A few degrees of Kevin Bacon there and you've got a real system.

    Such a system allows for anonymity as well. I don't need to use my real name if I can generate some other identity and foster trust in some other community. As long as the identity token itself is secure, they don't need to know my name, they just need to know I'm not a troll, I'm insightful (hint hint), etc.

    My vision of such a system would use SMTP as the transport mechanism for requesting and relaying trust between parties. Mail agents would handle the requests automatically, like calendar-enabled mail programs do now, and it is a fully-distributed system. Mail clients would also cache trust from their own "friends," like DNS, to better respond to requests.

    This degrades well, since the emails can contain manual instructions for those whose mail clients don't have this feature. Or their Internet providers can help with server-based responses, so the mail client doesn't even need to be involved in most cases.

    With such a system, spam would mostly be a thing of the past. I can limit incoming email to only people in my Address Book, people in theirs, etc. out to some limit of degree. Chances are, that will quickly encompass everyone likely to want to send me a legitimate email, and bounce away people with no legitimate friends (spammers). The system would self-correct when accounts are compromised or people unwittingly trust spammers, and if a friend of mine is too naive and adds spammers to his list constantly, I can stop trusting his list.

    We really do need a ubiquitous identity-trust system, something that uses existing protocols to share trust and integrates with IM, email, online forums, auction sites, etc. But the problem itself isn't that hard.

<<<<< EVACUATION ROUTE <<<<<

Working...