Online Reputation Is Hard To Do

Symblized writes "A new article from InformationWeek argues that not only does the Web need ways to verify identity, it also needs better ways to measure reputation . The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely. There's also a profile of Opinity, a company that tried to introduce a reputation system and didn't make it. Choice quote from a source in the article: 'The idea of a transferable, semantic reputation is identity nirvana.'"

Trust is the currency

"Trust is the currency of the participation age." -Jonathan Schwartz

This is the $64,000 question. Building a reputation/trust system is very difficult. Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

I'm volunteering at Citizendium, which is another possible datapoint. We're assuming that real names and respecting verifiable expertise will allow us to benefit in some fashion from existing scholarly reputation systems, and to build a more cohesive community.

Eventually, I think it'll be feasible to layer reputation and credentials (for sites that care) on top of a system like OpenID. People will be able to choose what reputation/credentials to share with which site. Information that you want to follow you (e.g., "I have a BA in Math from UCLA" or "I have excellent karma on Slashdot") will follow you across sites.

But yeah, it's a very difficult problem. Figuring it out is a big, potentially very lucrative issue.

And what do you buy with that currency?

It seems that everyone trying to "solve" this "problem" doesn't know what they're trying to achieve.

So what if you can make a perfect pseudonym identification system? What does that achieve for you? What do you accomplish beyond that?

Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?

Re:And what do you buy with that currency?

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty. On the other hand, if you have a dreadful karma on slashdot, you'll be saying the same old pants on other sites too.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too. What that means to a particular site depends on that site, for something like ebay that can matter quite a lot, at least it would allow good posters to be recognised as such, and then I think sites would start to implement policies on posting that restrict non-recognised 'nyms until they gain a good reputation.

Re:And what do you buy with that currency?

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. Just like if you act like an asshat at the company picnic, it effects you back in the office and possibly gets back to your friends at home. And yes, I think if someone is a carebear in WoW then they are a more trustworthy eBay seller, and someone with intelligent /. posts is more likely to contribute intelligently to Wikipedia.

Re:And what do you buy with that currency?

You don't think the fact that you occassionally like to "flame and act silly" or play "an annoying lowbie ganker" is telling ? It tells me that when you can hide behind the veil of Anonymous Coward, you don't have much respect for the people around you. Because let's face it, when you do those things you are causing real frustration for real people, and you do it for fun.

Don't be so gullible as to think someone's behavior is necessarily consistent no matter where they go on the internet.

This would be a case of adding "but ONLINE" and thinking it's something new and different. I don't think anyone's personality is 100% consistent as they go from one social setting to another, but it is all facets of the same actual person.

Re:And what do you buy with that currency?

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere.

I'll play devil's advocate here...

Let's say you comment on the theory concerning the use of demolition explosives on one of the world trade centers - pointing out that the collapse of the WTC doesn't look like other building demolitions [google.ca], or that the "symettric demolition" claim is incorrect.

However, the conspiracy theorists on the site are extremely fanatic about their theory (as opposed to a more moderate site that tries to investigate properly.) As a result, you receive a large quantity of negative feedback that attaches itself to your online reputation.

Other things that can affect you would be playing RTCW:ET, where you get kicked from a server for n00bism as you didn't dodge the three panzers that get fired into your local area (because another player thought you should have.)

And my personal favourite - just claim you support Bush. Your reputation would instantly tank.

Re:And what do you buy with that currency?

Regardless of what kind of eBay seller you'd be, this system would let people shun you for being an asshat somewhere if you used an important identity to be an asshat with. This would relegate most asshattery to anonymous identities, which would mean that sites that want to eliminate asshattery would simply require that all participation come from an identity with a decent reputation. I'm not sure if this is a good thing or not.

Re:And what do you buy with that currency?

You'll buy one. No, seriously. You'll see a whole host of forums, sites, etc, spring up where you either "have a good reputation with iDentify", or "Paypal/MC/Visa $15" for membership.

The merit of such a system is not particularly high. Neither is the probability of those sites making a profit.

And then you'll have reputation farms, like we now have link farms, that spammers will use to build reputation.

To clarify that ...

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty.

So you're saying that it would help filter out a majority of the "complete mumpty".

That's a possibility. But it would be even easier to just use Slashdot's reputation/moderation system on your own site. That would solve the "complete mumpty" problem while also solving the problem of someone with excellent karma for his programming knowledge posting his conspiracy theories on your site.

And it automatically tunes itself to your audience.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too.

Not really. Check back on the "creationism vs evolution" stories here.

What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

You achieve all the same benefits without the problems just by having your own reputation/moderation system.

Re:To clarify that ...

What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

You achieve all the same benefits without the problems just by having your own reputation/moderation system.

I suspect that there will be a lot of correlation between sites, as people who act like asses in one place are more likely to do so elsewhere. On the other hand, I also think that what will actually emerge is that the sites that correlate cluster in groups: if you have a good rep on site A, you're more likely to have a good rep on site B and a bad rep on site C, and you probably won't frequent D at all. Given all that, someone with a good search engine and a lot of cleverness will be able to start mapping out what these virtual communities actually look like and other sociological/anthropological stuff like that.

But will sharing reputation systems help sites? I think so, eventually, but not for many years yet. Until the reputation clusters have been found, sharing is as likely to introduce needless pollution of reputation systems as it is to enable reputations to be built up quickly. (Could there be a single reputation cluster? Maybe, but I suspect not; people are too inclined to divide the world into "us" and "them" for it to work out.)

One thing that might come out of reputation research is that it might become possible to use the reputation clusters to predict, from someone's interests and interactions, which sorts of sites they'd like to visit. OK, that does sound backwards, but it should guide people to where they won't want to make a total fool of themselves on a regular basis (yes, even the griefing pranksters; after all, when amongst the fools the foolish are sages and the wise foolish.) It may also eventually be possible to join the reputation clusters up, but using negative links (so reputations on sites for followers of Xenu who believe in ID and the supremacy of feng shui of placement of feeding bowls for their chihuahua will negatively reinforce reputations here) but I doubt that will help any time soon. There's a revolution waiting to happen here, but since it really involves lots of people, it'll take time to brew.

On the other hand, it is sensible to start working out what technological steps are required to enable specific bipartite reputation sharing, as well as looking at how to build sane single-site reputation systems. For example, slashdot's is pretty good in that it isn't easy to totally game the system while being mostly self-regulating, but can it be bettered without input of data from outside sites? If it can't be greatly improved, how difficult will it be to export the system to other sites? (It's late: I'm sure you can think of other aspects, but I can't right now.)

and...so what?

I've got mod points, but I think I'd rather participate.

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty. On the other hand, if you have a dreadful karma on slashdot, you'll be saying the same old pants on other sites too.

And so what? Is all of this really so important? I find it fascinating that so many people on so many sites care more about their "reputations" than what they post.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too.

Does it? Sometimes I don't WANT my "good" reputation to follow me. I like acting like a goon on something awful and like a lolcat-loving ding-dong on fark and like a...well...never-you-mind-like-what on consumptionjunction and 4chan.

When (and where) I want to be serious, I am. Others see it quickly enough too. It doesn't take long at each site I join for people to realize that I'm a "good poster". Honestly, it isn't complicated. Stay on topic, write well, be helpful, and the rest follows. Such has been my pattern over the years at sitepoint, namepros, webhostingtalk, and even here.

\Perhaps it's because I'm old
\\And still use slashies
\\\(reversed because slashdot doesn't like 'em forward for some reason)

Exactly

I was going to post much the same thing, so it's refreshing to see someone else thinking that way.

The thing about Slashdot's karma is that it creates groupthink. As you've said, too many people care about it instead of just posting what they think. So they post what they perceive to be the popular opinion, even if that's not what they really think, or even if it's contrary to what they really think.

Frankly, I think groupthink is a bigger problem than even the goatse links. Groupthink is where rational information exchange dies. If you look at the worst bible-thumping communities, or at rabid theocracies, or at the worst excesses of history, and some of its biggest mistakes too, almost all were based on groupthink. Take a million people who individually think "jeez, X is stupid and evil" and put them in a big group where they think that everyone else is fundamentally and rabidly pro-X... and watch them all start chest-thumping for the very thing they secretly despise. Just to get brownie points with the rest of the gang.

When a whole village went and cheered about one of them being burned at the stake as a witch (for bonus points when everyone knew it's a bogus charge and the real reason is something like: widow without sons inherits some land, some rich guy wants her land), that was groupthink. "OMG, I can't let the other ones even think I'm not a rabid fundie. Why, my popularity would go down."

At the risk of tempting Goodwin's law (although it's not a comparison): when a few million Germans cheered about invading the USSR, that was groupthink too. "OMG, I can't let the others think I'm not patriotic."

And in our own times, when you look at such things as bible-thumping communities, or at the broken high-school culture where being smart is uncool and being an airhead is the apex of fashion... guess what? That's groupthink too. Once the ball got rolling, even kids who do understand that their future job does depend on it... still go and insult the nerd, because that's what brings them karma points with the rest of the group.

So, to cut a long story short, I actually _don't_ want that kind of global karma. I actually _want_ people to come forth and say what they think, and not what they think would be popular in that community. I want people to actually come forth and say stuff like "this war is bogus" or "the PATRIOT act is unconstitutional" and not devolve into sheep thinking "OMG, I can't have it follow me for the rest of my life that I'm not patriotic or that maybe I have something to hide". Even if it's something as unimportant as a games forum, I actually want people to come forth and tell me the bad parts about it, so I can make an informed decision. I don't want more of them to think "OMG, if I say anything bad, I'll come out as a troll." Etc.

Re:And what do you buy with that currency?

Yes it matters. A lot.

Online, a lot of clues that are present in the real world are absent. We instinctively assign some level of trust to people we meet, for different purposes, based on a lot of variables, some of them we're aware of, others are subconscious.

You let your neighbour have your house-key to water the flower. But you wouldn't do that with *ANY* kind of neighbour.

You let someone babysite your kids. You let a friend borrow $50 'cos his credit-card is broken. You wouldn't to everyone, it's a matter of trust.

You trust someones judgement on some issue -- because you know that they are experts in the field and have a track-record of good judgement.

Being able to build trust in a pseudonym, and being able to prove that you are that pseudonym is very useful. It allows people to trust you who wouldn't otherwise.

To avoid abuse, it is nessecary that *you* have complete control over what aspects of your trust you share with which people and which companies.

So, what do you want to achieve ? World Domination offcourse ! *grin* No seriously, a million little small things, each of which may be unimportant, but the sum could be huge. Some examples used *today* include:

• If you've got excellent reputation, Ebay-buyers generally won't mind paying first, getting the item(s) afterwards. This would be quite risky -- except you know that the seller has sold 471 things before on similar terms, and -zero- of his customers complained.
• Hospitality-club use a trust-system to allow you to let complete strangers sleep over at your house, or vice versa, with a much reduced risk of any unwanted problems. Sure, *you* may not know this person, but it helps if 50 other people do -- especially if 5 of those are your friends.
• Slashdot use a trust-metric to let people with a track-record of sane comments be sligthly more visible.

In a universally networked world (which we're rapidly approaching anyway) with strong trust-systems, you could stop a complete stranger on the street and ask to borrow his car -- and he'd actually consider it. He wouldn't know *you* but, he'd be able to know a lot *about* you -- if you choose to share it with him.

Re:Trust is the currency

I have some nasty comments about this, but first let me switch to AC.

Re:Trust is the currency

Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

That's because Slashdot's system puts only minimal emphasis on individuals, and very high emphasis on selected adjectives of value. Comments do not simply get moderated up or down, but have to be moderated with a chosen adjective, such as "insightful", "informative", "funny", etc. This really helps keep people's heads on straight, especially with the presence of meta-moderation, because people then have to agree on what these words mean. The end result is that posts are usually moderated in close proximity to these labels.

The karma attribute is used only as an accessory to this content-based moderation, to provide some inertia to the community's character. It's not really a reputation centered system.

eh?

Why is The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely in a different colour to the rest of the text?

online resumes

As mentioned in the article, online resumes are one easy way of verifying credentials. But even that's not perfect, as they can be faked quite easily. Heck, people have been fudging information in their resumes even before the internet was invented!

Maybe somebody like Google can use their search engine technology to develop an improved algorithm that would perform multiple searches across multiple websites and databases to come up with some type of score rating an individual's credibility. But even this has drawbacks; do we really want to give Google that kind of power?

Wikipedia needs reputation system

I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles. I haven't kept up with the structural changes they've made in the past couple years, but a lot of the editing work seems to be undoing trolling and vandalism, and also participating in edit and revision wars. I could be wrong at this point.

But if wikipedia had a reputation system ( other than just being banned or allowed ), they might automate contributions from reputable authors ( and check on the actual contributions later), while authors who are less reputable may have their contributions queued for review before they are published.

Furthermore, a casual user would be able to have a more savvy understanding of the reputability of any article or section of an article if it is tagged with the reputation of its' author.

Reputable authors might be able to also tag the contributions of others, such that the text or information itself gets a reputation. That would help users make a judgement about the validity of information on Wikipedia.

Instead of pushing the mechanics of the actual editing of articles behind the scenes, and just presenting a 'final' article to the end-user, let's formalize the process and enfranchise users into the process of judging the validity of articles.

Hear, hear!

I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles.

Indeed! Here's my own anecdote on that: I recently tried editing an article on a certain Posada Carriles, a man whom the Cuban government and Wikipedia call a "terrorist".

I was browsing the Cuban government site Granma [granma.cu] where they had a list of what they called evidence against Posada. One item was an AK-47 rifle, another item was a box of 5.56mm ammo for that rifle. It doesn't take much of gun expertise to know that NATO ammo doesn't go into an AK-47, and I tried to put that in a paragraph criticizing the accusations against Posada. I don't know the guy, for all I know he could really be a terrorist, but you aren't going to convict anyone in a civilized court of law with that kind of "evidence".

I was thoroughly flamed by someone about that. It seems that Cuban government sympathizers are carefully patrolling any critical statements about the dictatorship. If Wikipedia had a reputation system, the commies would mod me down for presenting a balanced view in their rant against Posada, but I would recover my karma through my other contributions. OTOH, fanatics would find it too troublesome to fake an interest in subjects other than their favorite and their karma would suffer from that.

Hmm

The only reason online reputation is hard is because online identification is hard. Once you're past the identification and privacy issues you could go Google: your single/central/one point rated identity, linked with all your accounts from all over the place which should give you some sort of a global and more specified ranking (karma on ./, trustworthiness on ebay, whatever rating/googlerank on google/amazon) for people to search for.

Web design

Companies will often try to maintain "reputation" by using a rigid, non-interactive web design. With Flash, for example, a website becomes just a commercial to sell the perception of quality, or the illusion of branding.

(Isn't that right, all you "Hubsters"?)

What about LinkedIn?

LinkedIn has a reputation model that works in a limited sense. It tells you the people on the chain between yourself and an individual, up to a certain length.

Since you know (by definition) the first person on the chain you can ask them to make enquiries along the chain about the person you want to know about. It lists up to 3rd degree associations, ie your friend knows them or your friend knows a friend of theirs. Surprisingly effective for finding out about someone you want to hire, in a general sense at least.

I'm still waiting for a P2P system that works the same way - you create encrypted connections to your friends, and can pass requests for content that are spidered out across the network automatically.

Because you only ever talk to your friends, and because no ultimate destination information is passed on, it should be very good at preserving anonymity - in exactly the same way as Freenet is, in fact.

The difference would be that because you manually tell it which nodes to locally connect to there is no danger of encountering a poisoned node... as long as none of your friends are spies for the MAFIAA.

Linking RL with the IL

Even still, it is hard to rank someones reputation based on a numbers system. In alot of forums I post on, I am a regular poster with a high rank. However, alot of people have an issue with me because of my free speech (the beauty of the internet). Is there really a standardized way to determine reputation? It really has to do with the context. If you are on a programming forum, you may rank someone based on their aptitude for a specific language, or their problem-solving skills. Conversely, if you are on a political debate forum, ones reputation may be based on how fluently their opinions are expressed.

Credentials

One of the solutions Wikipedia co-founder Larry Sanger suggests is using "credentialed" experts who have college degrees or an institutional email address. This not only smacks of arrogance, it is completely fallacious. It is like suggesting that a person with a drivers license is a good driver.

This process also effectively eliminates the diligent amateur who may very well have good very good methodologies of investigation.

Expertise can be got by many methods. Certifications merely prove that you paid the price, and passed the tests. After that it's all tenure.

Instant Mashed Repuation!

The problem is that everyone wants to know right now who is trustworthy and who is not.

Building a reputation takes time, often a lot of time. Amazon's reputation is built on several years of good service, good web design, and overwhelmingly positive customer experiences.

Facebook and Digg don't have that track record, and until they do will not enjoy the same level of trust.

Any system designed to give a stamp of approval needs only one mistake to become untrustworthy. Unless it can be nearly 100% foolproof it won't be effective. And given the number of supposedly trustworthy businesses who are anything but, I'd say that rating reputation is not likely to happen soon.

measure reputation among who?

I don't align politics-wise/religion-wise on this site, so while I have excellent Slashdot karma (built up over a very looong period of time commenting on non-sensitive topics for this audience), I wouldn't otherwise be considered having a very good reputation here.

There's that plus many sites like these are mostly just kids playing around and/or just mouthing off saying any ridiculous thing because they can. And practicing to be good little enforcers of Political Correctness for when they get older.

So, an online reputation based on sites like this at least, is valueless. The value of a reputation is only as good as the people who judge it are worthy to judge.

The First Law of Cyberspace

There is no such thing as negative trust.

(Once you accept that, the rest isn't so hard.)

heatware

Heatware [heatware.com] Most frequently trusted on many For Sale/For Trade forums because of their strong stance on scammers.

What about well-prepared people?

I haven't used my name for posting on the Internet since 1997 when I realized that dejanews.com would keep my newsgroup postings forever (even that was with a somewhat random e-mail address). I literally don't have any internet presence with my real name unless it's inadvertent (ie. a news release from my employer) but the good thing is that my name is so common I would be hard to find anyway.

So in it's place, I created a whole shitload of false identities that I post under, one of them about 10 years old now. Mainly on forums and newsgroups for work purposes, etc. If you searched for this particular identity, you would probably fine hundreds of posts (including many on slashdot) some of them truthful, some of them fake, with various opinions of topics.

Every few years I will discard an identity or create a new one, for various reasons. I even have a fake lj blog that I've created just for the purpose of having that sense of "credibility", just in case I need it. I usually update that every few weeks, with something that I read on someone else's blog, but changing the words around just enough so that I can't be googled and exposed as a fake. I make sure each identity has a different way of typing, different levels of typos or capitalization, etc. I don't think you would be able to properly gauge the "credibility" of this person at all.

I doubt I'm unique and there are probably scores of people doing the same thing. As internet users get more and more sophisticated, how will internet credibility really be gauged unless you actually meet someone face-to-face? I was even contemplating getting a pay-as-you-go cellphone with no traceability (paid with cash at a store in a different city than where I live) just in case I needed to talk with someone offline. I'm doubtful you can really establish credibility to the point where it's better to just assume that everyone is lying and be on the guard all the time.

RapLeaf

What about RapLeaf [rapleaf.com]? Although it's centered around ratings for conducting transations, I have to believe their system would be pretty effective across a broad spectrum of reputation and ratings needs. Plus, they offer a set of APIs, which is always handy.

Advogato

Well, there's Advogato's Trust Metric [advogato.org] system.

I've been of the opinion for a while that a similar system could be devised using PGP or S/MIME certificates to combine identity verification with "web of trust" reputation evaluation. Under such a model, every user would import the public certificate of authorities that they trust. For example, consider a consumer review web site, where I decide to trust the site's admin. The admin trusts its editorial staff, and their certificates are signed by the admin. Any of the editorial staff may trust one of the site's frequent contributors, based on the quality of their work. That editor may sign the contributor's certificate. Now, my level of trust for that contributor can be established as a function of the proximity of that user to the admin in whom I placed trust. This differs from Advogato's system in that the "Master" certificates are simply those whom I've decided to trust.

The same thing can be applied to social networking sites, as well. I can trust my friends by accepting their certificates, and gain insight into social relationships by examining the signatures in their keys.

Reward and punishment

Online, and in real life, we reward bullying and punish honesty. How do you make a reputation in that environment?

This is the Internet; who are we kidding?

The reputation system? Maybe something that, yes, we'd like try, but, this being the Internet, there always will be some sites somewhere that put out misinformation. Politically-motivated sites are the first ones that come to mind for me.

The idea of a transferable, semantic reputation is identity nirvana

No; Kurt Cobain, Chris Novoselic, and Dave Grohl are Nirvana.

Reputation vs. identity

The article claims to be about reputation but mostly talks about the various "identity" efforts out there. Yes, a reputation is associated with an identity, but most of the identity systems being promoted focus on real identity rather than pseudonyms which you can choose to associate with yourself or not.

There is a paradox to those systems -- the easier they are to use, the more they will get used -- and demanded. We'll go from a web where most web sites can be used casually, with no "sign on" (single or otherwise) to a web where far more sites demand you use the single sign on and thus have an account, because it's easy for them to ask.

This paradox is described at http://ideas.4brad.com/paradox-identity-management [4brad.com]

Online reputation? All reputation is hard to do

It's why we have exams, professional organisations, CVs, brands, social networking etc etc etc.

We use reputation all the time and no-one has come up with a single reliable, coherent way of measuring it. You just try to get a decent builder.

 

Why bother?

There is access via the web to some sources of information which exist independently and so are fairly spoof proof. For instance, someone wanting to research me could look for publications with my name on them in PubMed. Sources like that would be valid, web or no.

But as long as the web remains a place where anyone can say anything, rightly or not, anyone who relies on it for supposedly objective information that they can use to measure a person's reputation is coming in dead last in the old "Arguing on the internet" poster (http://www.argaste.com/img/arguing_on_the_interne t.jpg).

And that's exactly what should happen. The web *should* remain open to anyone for anything. An individual has no better defense against boneheads that would take whatever they find on the web seriously than to prove their faith in all things computerish as truthful misplaced by pointing out absurdities in what's available. In my last position a colleague came to me with the breathless warning that someone in the department had found some less than flattering stuff about me on the net. I responded by posting a picture of me from the net on my office door (not the same as http://www.subgenius.com/bigfist/fun/devivals/X-Da y98/POST-X-DAY/X-DayPhotos/portraitsTN/_dynasoar.h tml [subgenius.com] but from the same stage show). I never heard another word from them.

That being said, as long as anyone can post anything, a person's best defense against misinformation is disinformation. The latter invalidates the former in any rational mind. If someone still can't see the joke, that's not someone whose opinion matters to me anyway, even if it's a potential employer. I want to know when they're so informationally inbred so as to take this garbage seriously, as I want to steer clear of them. These are the same people, in spirit if not in truth, that would believe anything they read on paper if it was green and white lined line printer paper printed with dot matrix, since it obviously "came from a computer" and so must be right.

There's a real world, and the web does not reflect it any more than what's printed on that green and white lined paper.

OpenID Not A Good Candidate For Trusted IDs

I've been doing some research on OpenID, which seems to be considered as a possible substrate for building trusted identities (OpenID, as its proponents are quick to point out, only establishes an identity token, not trust). However, I hope that trust does not get built on the OpenID model because the OpenID protocol for identification is very poor from a security standpoint, a blindness to which the OpenID proponents have (I think) because of the vocabulary used.

Basically, the way OpenID works is that you connect to some site, give it your OpenID, which is just a URL, and the site you connected to (in the initial 1.0 standard of OpenID), gets the content at the URL. From the URL, it finds the URL of an identity server and redirects your browser to that server. Here's where the problem lies: In OpenID terms the site that you connected to is called the 'relying party'. However, you are really the 'relying' party because you are relying on the site you just connected to to send you the correct URL of your identity server. If instead they send you to a machine that merely proxies your identity server, they get your identity server password as you authenticate and your identity is compromised.

Now, there are various ways that OpenID proponents say this can be handled, but it's a fundamentally dangerous security model when you rely on an untrusted site to direct you to your identity system. The use of encrypted keychains (keyword bing encrypted here) with browser autofill, albeit not perfect, is much more secure system and works well enough that I'm not sure what the real savings of OpenID are (OpenID proponents will point to all kinds of other uses that OpenID could *potentially* be used for, but the process it was designed for and only practical purpose to date is to log into web sites using a URL instead of a username and password). Is saving having to fill in a password really worth this much complexity: http://openid.net/pres/protocolflow-1.1.png ?

When it comes to trust, we need to figure out a less complex methodology for identity before we can start establishing trusted identities. We also need to make an identity valuable, and right now an OpenID identity doesn't really represent something of value to general users who already have keychains and autofill. In fact, OpenID proponents often defend OpenID by saying that it should be used for low risk logins like blogs and the like. In that case, non-encrypted browser auto-complete is already superior from an end user perspective, and generally enabled by default or by clicking yes on one dialog window with first use of browser.

Privacy?

Most people don't like to be identified or even their synonyms used on different websites to be linked; otherwise you would see more people registering with their real name in the first place and OpenID would be pretty common too by now. I guess the next newsitem on this topic will probably be filed under YRO.
Imaging logging into your workaccount from home and your boss looking up the IP in Google Identity Search or something and seeing every website you visited and every comment you made. Google could probably really do something like that, since they know every website using Adsense you visit anyway and that's a lot.

Also, I couldn't care less if someone writing informative and insightful comments here on /. is trolling on some other website. It's the content of that one particular comment that matters and not what he wrote a year ago on some other website. Next thing will be that people don't even read comments anymore because everyone gets automodded based on his karma anyway? ;)

Rules of Reputation

Back in March, I sketched out some "Laws of Reputation" (along the lines of Kim Cameron's "Laws of Identity") on my blog. As I thought about it, I came to the same conclusion. It's *hard*. eBay has a good system, but they only have one context to juggle and it's hardly free. I've seen people posting about the need for context & anonymity, which I touched on, but there's a big difference between saying "it needs to" and "it does". At this point...there hasn't been anything done to implement them, and I'm not 100% sure it would be commercially viable.

An online reputation system should be...

Anonymous: The system needs to protect anonymity. OpenID uses URLs for identity, with the knowledge that a person can register for a LiveJournal account under an assumed name and get an OpenID with it. On the other hand, a site like csixty4.com is my OpenID, and I have my name all over the place.

Contextual: We have different facets of our lives, and the should have relatively little ability to influence each other. I could be the most helpful person on technical message boards, but a complete jerk in Star Trek chatrooms. My negative behavior in one venue should influence my reputation in other forums, but to a small extent.

Forgiving: A reputation system can't hold people accountable for stupid things they did ten years ago. A reputation claim against you from a third party should lose relevance with time, and be completely ignored after a "sunset" date.

Automatic: People are quick to complain about bad behavior, but less motivated to laud good behavior. The system should support default, automated "good" ratings, which would carry less weight than a positive reputation claim, and would be in effect until a positive or negative claim is made. In other words, a message board post that isn't marked as "good" or "bad" by other posters would, by default, rate as "half a good" or "1/4 good" as long as nobody said otherwise.

Free: Second Life's reputation system lets users make reputation claims about other people, if the party making the claim pays for the privilege. This is to cut down on people gaming the system, but also acts as a disincentive to say nice things about other people. It shouldn't cost anything to make a reputation claim about another person, and it shouldn't take much effort, either.

Trustworthy: The system should be set up so it's hard to "game". A reputation system is pointless if someone can run a script that creates 1000 fake accounts which all sing the praises of a main account.

What would this reputation system do? On Slashdot, a person's "karma" can modify their posts' scores, and low-scoring posts (trolls, flamebait, etc.) are hidden from view. A similar phenomenon leads to troll posts on Digg to be "dugg down". Similarly, programs and sites could let users set a reputation threshold, and statements from anyone below that threshold would be ignored.

They're just blogs.

When I first saw the title, I thought this was about reputations for web sites or online businesses, but no, it's about reputations for, well, bloggers. Where it doesn't really matter all that much. It matters for eBay, but most of the sellers on eBay are businesses. It's been a long time since eBay was individual to individual.

Dating sites have struggled with this. True [true.com] wants to see an image of your driver's license. With the controversy over Myspace, we may see them going that way, at least for parents.

Wikipedia doesn't care much about identity, except as regards vandal blocking. Even admins and ArbComm members are anonymous. All Wikipedia needs is some way to slow down unlimited generation of new identities. I once suggested that one way to do that would be to require some easily available, no-cost, unique, verifiable physical token to register. Like an AOL disk.

One approach to identity verification, which I'd like to see used for domain registration, is simply mailing out a card by postal mail. When you register a domain, a letter should be sent to the address listed for the domain. When you get the letter, you type in the password printed in the letter postcard, and the domain registration completes. That would really improve WHOIS data quality and cut down on scams. The cost of sending out customized mailing pieces is under about US$0.50 each when you have a bulk mailing house do it, so it's quite feasible at current domain prices.

Collaborative filtering

Reputations are relative. They depend on the person making the recommendation. If the person doing so is a numpty, the information the recommendation is based on isn't worth much. Expanding this to everyone would be what, an N^2 problem? Where N is potentially the population of the planet. Thankfully not everyone knows everyone.

It would require a centralised registry though rather than distributed in order to calculate the effect of the relationship to the person making the recommendation. And it would of course all have to be based on a reliable identity system.

Good luck with that.
 

This isn't just an online concern

This is a problem with real life. There's a truism along the lines of "you never really know who another person is, you only know what they've shown you." This goes hand in glove with another truism: "perception is reality." Every one of us creates models of people in our life to better understand what makes them tick, how they operate. Once we feel comfortable enough in our understanding of that person we can say things like "Oh, X is going to love this!" or "Just wait until Y hears about this, I know he's going to flip!" But do we really know the person? Not really. We only know what we see. And if we like someone, we tend to gloss over the negatives. After shit happens, you'll get an earful from friends as to why it should have been obvious but they kept their mouths shut beforehand for fear of alienating you. And they were right. Would you have listene to them? Of course, not. Bad relationships, can I get an amen in here?

I tend to classify people into the following categories: good to have a beer with, someone I would introduce to other people I know, someone I can trust with something semi-important and people I can trust with things that are very fucking heavy. Most of the social drama I see comes down to someone mistaking a casual friend for someone that can be trusted with something heavy. "What, you're upset that Weasel slept in and forgot to drive you to the airport? Why are you so surprised? This is Weasel, the guy with the reputation as the heaviest drinker in the city. He's fun at a party but you actually thought you could rely on him for something more than a laugh? You made the mistake here, not him." Of course, that's exactly the sort of thing you don't say or the drama will shift to you.

So this is all normal human social dynamics, web doesn't have anything to do with it. People were doing long-distance business with people they didn't know personally for thousands and thousands of years before computers were invented. It all comes down to trust. Does the person have a reputation? Can someone else vouch for him? Someone using a fake or young ID on the net is no different from someone using an alias in the past. And what does it come down to in real life and on the web? One simple question: "Is there enough incentive to fuck me over in this deal for the guy to throw away his reputation?" Because it takes time to earn a reputation, get seller feedback on Ebay, etc. Do people put time into pulling off a proper con? Of course. Just look at the con men who go after little old ladies. This isn't just scamming someone one the net, this is full on interaction, romancing, sleeping with, trying to convince the mark you're legit so you can get the means of fucking her over for the inheritance. But do you think that the high stakes con man is going to try fucking over your grandma for her social security check? I don't think so.

Even when someone has good reputation, you don't know what he has going on in his life. Maybe something has come up that means something is more important. I've seen businesses with formerly stellar reputations go to pot because the owner has other things on his mind.

Figure out a way to fix this problem in real life and you'll have a model for how to do it online.

Reputation at SXSW 2007

Was part of panel at SXSW 2007 "Every Breath You Take: Identity, Attention, Reputation and Presence" - Quite an interesting topic.
My presentation here: http://www.slideshare.net/ted.nadeau/sxsw-2007-rep utation-20 [slideshare.net] ... The internet is still more infantile than infinite.

eBay's system doesn't work

eBay's system of who you can trust and who you cannot doesn't work. Generally, even theives have a good feedback rating on ebay, and worst of all, the average joe who isn't an eBay powerseller or doesn't have his own ebay "store" is likely to get shafted because eBay only cares about fees, not feedback.

For example, a user with a feedback of 1 can buy something from a seller with a feedback of 450, and then complain to ebay. The user of with the feeback of 1 can have the seller's account suspended despite the vast chasm of difference between the seller's feedback and the buyers, and ebay makes NO distinction over which party has the better reputation.

It all comes down to who files the dispute first. Either that or eBay just assumes the seller is always at fault.

Ebay's dirty little secret is that you can create an account, buy something, and essentially get it for free from the seller because the only way the seller has to resolve the suspended account is to refund the payment. There is NO other recourse.

And "reputation" means nothing to ebay. You could have created the buyer's account a week ago, and take down a seller who has been a good seller on ebay for years.

And when Ebay gets wise to you, create a new email account on yahoo, start again, buy something, register a complaint and get it for free, because the seller has to refund your money.

So; how exactly does eBay's system "work"?

the key problem

Is that no one wants to hand anyone person all of the neccessary things to verify your identity. think about it. Do you really want to give the company an SS#, address, email, phone, birth cert, etc. To get a digital certificate of some sort?

Especially in light of the TJMAX and credit card company break in, as much as I personally have faith in Verisign and such, no one wants to give someone ultimate identification power. And I don't think the solution will ever be a "one stop shop" thing.

Basically we are going to have to expand upon the key signing initiatives that exist currently through digital certs, and make them less expensive. Then you could have your bank sign your cert, your employer sign it, your friends sign it, etc. Each additional signature adds more "trust" but trust will never be 100% absolute I don't think. Just as we have fake id's in the real world, they will exist here to.

The problem is how to hide the big red button from the idiot; because inevitbaly there will be a button, and as we all know there are plenty of idiots out there.

Reputation has weaknesses

1. If an account gets hacked, the hacker gets to control its reputation. You can't tell know whether I'm really giafly, or just some /b/tard who just hijacked this account, except that the latter would be funnier.
2. A routine psyops technique is to regularly start up accounts, karma-whore for reputation, then burn each account in an orgy of inappropriate posting. This is often more efficient than using n00b accounts to troll.

Doesn't this already exist?

Employers search Google about their candidates, partners about their potential dates. Who needs a centralized database - or even a format convention - for online reputation when it is already possible to get a good first impression of any person by the web trail they leave?

If anything, leaving this without a formal standard or central authority heads off problems with privacy or manipulation.

Formal reputation systems are useful at a local level. It works on Ebay, where there is only one type of transaction (a sale), and all reputation and references stems from how the party behaves on either end of the transaction. When transactions and relations become more complex, it might be best just to provide the raw information and let people make up their own minds about each other.

This is a good article, although it only touches on the topic in one section late in the text:

A Group is its own worst enemy, by Clay Shirky [shirky.com]

Reputation scoring

This is just a random Monday morning thought... I used to work for one of the major credit reference agencies. When I told people who I worked for, they would almost inevitably say 'oh, can you sort out my credit rating then?', and of course the answer was 'no, it's an immediate sacking offence to do anything with live data' and 'what the agency says about doesn't necessarily mean that you'll get a loan or a credit card or whatever. That decision is down to how the querying party chooses to interpret the data that the agency returns.' By then they had normally got bored and gone away.

It strikes me that reputation could also be a similar agglomeration of data, from which it would be down to a site or an individual to make a decision on how much to trust a poster/seller/relationship/whatever. Of course, the difficulty would be getting the key services such as Google, eBay, Amazon, Yahoo, LiveJournal etc to provide data, and indeed give a metric by which to report (number of posts with more than 2 karma on Slashdot?). This would be presented as bit of XML/RSS (depending on how corporate the application needed to be) and would be then presented to a site's user either based on preset conditions, or on the user's own conditions. The user would not see the raw data. This type of system does add a level of complexity to web interaction but it's going to be essential soon, and far more than 'just for blogging'.

Reputation is easy

Identity is really the easiest part of the problem.

But reputation is easy as well. The problem with most proposals is that they are focused on organizational reputation rather than personal reputation.

Reputation, however, is relative and contextual. We don't need Slashdot vouching for us, we need people in our own address book / social network. Then we can vouch for our friends/family in various ways ("this person isn't a spammer", "this person knows a lot about cars", etc.).

But the real power of a personal reputation system is that it is transitive. If I trust that Alice is not a spammer, and Alice trusts that Bob is not a spammer, I can to some degree also trust Bob, and so can my friends, etc. A few degrees of Kevin Bacon there and you've got a real system.

Such a system allows for anonymity as well. I don't need to use my real name if I can generate some other identity and foster trust in some other community. As long as the identity token itself is secure, they don't need to know my name, they just need to know I'm not a troll, I'm insightful (hint hint), etc.

My vision of such a system would use SMTP as the transport mechanism for requesting and relaying trust between parties. Mail agents would handle the requests automatically, like calendar-enabled mail programs do now, and it is a fully-distributed system. Mail clients would also cache trust from their own "friends," like DNS, to better respond to requests.

This degrades well, since the emails can contain manual instructions for those whose mail clients don't have this feature. Or their Internet providers can help with server-based responses, so the mail client doesn't even need to be involved in most cases.

With such a system, spam would mostly be a thing of the past. I can limit incoming email to only people in my Address Book, people in theirs, etc. out to some limit of degree. Chances are, that will quickly encompass everyone likely to want to send me a legitimate email, and bounce away people with no legitimate friends (spammers). The system would self-correct when accounts are compromised or people unwittingly trust spammers, and if a friend of mine is too naive and adds spammers to his list constantly, I can stop trusting his list.

We really do need a ubiquitous identity-trust system, something that uses existing protocols to share trust and integrates with IM, email, online forums, auction sites, etc. But the problem itself isn't that hard.

How Can Trust be Established Over the Internet?

We have created an authentication system, "Trusted Node Authentication", that answers many of the questions raised in this article. We we can answer the most fundamental question, "How can trust be established over the Internet?"

http://www.forbintechnologies.com/forbinTechnologi esWeb/trustedNodeAuthentication.do?ref=109401245 [forbintechnologies.com]

This system has has now been reviewed by several academics and authors with an expertise in security. While most have pointed out the challenge of convincing ISPs to join the consortium, no one has found a conceptual flaw in the "Trusted Node Authentication(TM)" system.

Here is a brief overview:

A major consolidation of Internet Service Providers (ISPs) over the last five years provides the opportunity to create a new authentication system. (In the U.S., twenty-two of the major ISPs now control 75% of the market.)

Forbin Technologies is a startup company located in Austin, TX. We have designed a, "Trusted Node Authentication(TM)" system, that will solve many of the systemic problems of the current Public Key Infrastructure system, especially the problems of certificate revocation lists (CRLs) and on-line status checking.

This system could also provide the "holy grail" for single sign-on. We believe the simple and efficient single sign-on process for "Trusted Node Authentication(TM)" is far superior to the "federated identity" process provided by the Liberty Alliance framework.

Because of its non-proprietary nature and its ease of use, "Trusted Node Authentication(TM)" could become the cornerstone of a new identity management structure popularly referred to as Identity 2.0. "In Identity 2.0, usage of identity more closely resembles today's offline identity systems, but with the advantages of a digital medium. As with a driver's license, the issuer provides the user with a certified document containing claims. The user can then choose to show this information when the situation requires." (Burton Group)

Microsoft's CardSpace and the open-source Higgins project (supported by IBM) are the two leading Identity 2.0 technologies. OpenID and Sxip are also players in this space. None of these systems has an effective authentication mechanism built in or expressly defined. Without authentication these systems are simply profile management systems and not identity management systems. (Higgins and CardSpace provide a workflow for authentication from an "Identity Provider"; however, they do not answer the question as to why a "Security Token" from an "Identity Provider" should be trusted.)

The basic question is, how can trust be established over the Internet? If you and I have never met and I come to your website, how can you be confident that I am who I say that I am? "Trusted Node Authentication(TM)" answers this basic question regarding the establishment of trust.

Sincerely,

Mike Duffy
CEO / CTO
Forbin Technologies

Identity != reputation

The assumption that reputation management should be tightly coupled with identity management is often non properly stated .. (no matter what Dick Hardt keep saying in his beautiful speeches ...) I recently wrote few posts [clipperz.com] on this topic on the Clipperz password manager [clipperz.com] blog.

Thinly Veiled Disguise

These aliases are truly, and only, thinly veiled disguises. Unless you use aliases like a one time pad.
The proof is all through my post, at this point I must stop this charade.
I shall at some time be found out for who and what I truly am.
A very well trained Great Pyrenees. [I work for a Shepherd]
Fravia says it best [fabulous site]:

http://www.searchlores.org/noanon.htm [searchlores.org]

http://www.searchlores.org/fobegano.htm [searchlores.org]

BONUS !

http://www.searchlores.org/trolls.htm [searchlores.org]

I really like the idea of OpenID

http://openid.net/ [openid.net]

I think it would further a type of "vetting" or track record for such discourse that requires it / be enhanced by reputations. A HOSTS file of sorts like the academics of ARPANET lore. [I was but a pup]
So, tell me whose advice would you trust concerning sheep herding? Hmmmmm? I chase them for a living, hence, you could trust my advice.

Otherwise:

http://digg.com/ [digg.com]

For those of you that think me just another lap dog, I say, ... If only!

http://www.gailgiles.com/Jack.jpg [gailgiles.com]

[I like CATS (Broadway version), long walks in the meadow, old shoes, Slashdot, ...]

Re: The bigger question is WHAT you trust them for

So the question then becomes: is the public qualified, as a lowest-common denomintator, to do determine who should have the reputation points?

The public is as qualified to determine who should have reputation points as they are qualified to determine who should hold elective office.

Take that as you will...

Re:"reputation" is a bogus concept

Just give me an ID card and wiretap my house.. Reputation piffle I'd not be wanting whatever 99.9% of the populace would label me as spread around anyway as it isn't a true reflection of self but garnered through the many hats that we wear. Plus reputations are made to be destroyed just ask the media or any "good" politician....