Google TrustRank

Philipp Lenssen writes "Google registered a trademark for the word "TrustRank", as Search Engine Watch reveals. Is this a sign we can expect a follow-up to Google's PageRank? An earlier, possibly related paper on TrustRank is available; it proposes techniques to semi-automatically separate good pages from spam by the use of a small selection of reputable seed pages."

Will this have anything to do

with the newly proposed AdSense plans?

more censorship, unimpressed

so when google desides what's trusted for us, what is good content and what isnt, are they still not being "evil"? additionally, how are the pages seperated? on what criteria? man or machine (potential for flaws on either side)?

Re:more censorship, unimpressed

Its not censorship. Google couldn't censor even if they wanted to. Rather than explaining to you what censorship means, let me just tell you that what Google is doing is siply doing their job better. I don't want to find spam when searching for anything, and neither does anyone else. Ergo, eliminating spam from the search results makes everyone (except spammers) happier.

Re:more censorship, unimpressed

Yes, this is always a problem. How can you possibly know whether or not a site is spam just by looking at who's linked to it? A lot of great sites have very few external links to them and often they're from blogs and other sites that will likely be identified as spam prone.

This is a basic problem of filtering web-content. How do you avoid throwing out the baby with the bath water? I'm running into that problem in designing a custom filter to keep my son from inadvertently seeing pornography as he looks for his "r0mz," but that's peanuts compared to Google's dilemma.

The fact is, spam filtering is inherently censorship. This kind of interference will always have a negative impact on the marketplace of ideas that is the modern internet. On the other hand, as a side effect, removing blogs from search results (as this trust metric very likely will) may increase the usability of Google overall. I suspect there will be some people who are not as happy about that as I am.

Re:more censorship, unimpressed

You fail to understand that google is incapable of actually censoring anything. Them not displaying a webpage in their results does not, indeed, remove it from the web.

Google's primary responsibility now is to it's shareholders, which means increasing the chance that you and I find exactly what we are trying to look for, and not to unabashedly display every peddler that serves up content over http.

Re:more censorship, unimpressed

Considering how much market share Google has, them not displaying a web page in their results (or dropping it a few hundred places) effectively removes it from the web.

Google's primary responsibility now is to its shareholders. Google makes money from advertising. If Google can encourage you to patronize its advertisers instead of trusting its index for everything (which right now is pretty easily gamed), then Google makes more ad revenue and shareholders are happy.

Re:more censorship, unimpressed

"Freedom of Speech"

First of all, the only protection that is guaranteed you here is that the gov't will make no law abridging the freedom of speech.

Google, as influential as they might be, are not the government (insert 'Do No Evil' joke here). Therefore, they are not bound to this "Freedom of Speech" argument.

Secondly, "Freedom of Speech" is not this universal, higher-being ordained preserve at all cost idea that we have transformed it into.

Freedom of speech does not give you the right to spray-paint your slogan all over my front door, nor, in this case, does it give you a 'right' to be listed on Google. Nor do you have a 'right' to have your name printed on the front page of your local paper in 36pt font.

Not being listed in Google does not amount to censorship in any definition of the word. The net existed before google, and people still managed to find web-sites. Google gives (through PageRank or whatever mechanism they choose) free advertisement to 'good' sites. They have every right to only display sites that pay money, if they so desired. You have absolutely zero (0) 'rights' to be listed for free on Google.

Trotting out the Freedom of SPEECH argument is nothing more than whining about Big Brother coming to get you because what you have to say isn't worth hearing. Guess what? If you want to be heard, say something that's worth listening to. All that glitters is not gold, and much that is said (or printed) is worthless drivel. Much like this post.

Re:more censorship, unimpressed

Two points: 1) Any new system Google implements will run along the side of PageRank; they've invested too much to completely switch all of Google running to TrustRank. The system might even augment current PageRank by running an algorithm over the data that PageRank returns. We can only speculate as of now. But I can assure you that one will not replace the other, and there will probably be a way to use both systems in the future if you like. Hell, using your Gmail account, you may even be able to specifically tune PageRank, making certain pages more relevant to you appear higher in search results.

2) You have the option of not using Google. Yahoo is a completely independent search engine now.

Re:more censorship, unimpressed

"so when google desides what's trusted for us, what is good content and what isnt, are they still not being "evil"?

Yes.

Why is it that everyone is constantly striving to find Google's evil? Ranking the relevancy of pages to a search is Google's job. By ranking spam as relevant to my search they have failed. Using the concept of a web of trust to establish relevancy is a fairly obvious solution and has well established analogs in other fields (e.g. PKI).

If you're looking for evil, try GE, GM, or Unilever. Google doesn't even begin to rank on the evil-o-meter.

Conjuction?

Are these going to be used in conjuction? It would be very nice to be able to sort out those pages that have nothing but a long list of keywords on them. It's probably all in vain, as somehow will sooner or later find a way to get around this, as well.

Re:Conjuction?

It should be noted that the Slashdot user No More Free Stuff [slashdot.org] catalogs such links, and by adding this user as a friend and assigning a negative bonus to foes of friends, you can lower the moderated value of any such posts.

Potential abuse?

This is a step in the right direction conceptually, but giving a smaller number of "seed sites" more rank influence increases the potential fallout from any rank cheats that may be found in the future (see Google Bomb [outer-court.com] and Google 302 exploit [searchenginewatch.com].

Google may be better off as they are currently leaving all sites initally equal in influence before the Pagerank calculation.

Then again, Google has a great track record for testing their ideas before committing them to general service...

Re:Potential abuse?

With Google's "portal system" they're developing, the trust comes from within; the company trusts its users because they are clicking into an agreement of terms. That being said, hacks that would make this new TrustRank unreliable would probably just lead to the termination of services of the account.

This to me keys that Google's trying to become a more involved company; instead of just sitting back, caching and searching the internet, they are now trying to serve you best and give you the results you are looking for. I would imagine with TrustRank, you will see a little star or something near a link on Google's home page, and the star would indicate if it is something in your field that you would be looking for. For example, if you were a Biologist and searched for a certain kind of fish, say "Blue Tuna", it would put stars next to sites with the fish's breeding habits, etc., but if you were a general consumer, it would provide links to the local fishery.

The internet is an extremely powerful tool, and search engines have simply evolved to the point that they are now "dumb technology". Without more user invervention (and not simply by throwing in more keywords and praying), they will continue to be as they are now. Once the company better knows what we'll be looking for, they can better serve us. And that's all I see this new tech as being.

psst, wanna buy some TrustRank?

I've got a TR7 site with four links available...

Questions

How is this different from applying a weighting to PageRank?
Will the owners of the pages / sites deemed to fall within the set of trusted seed sites get any money for all their hard work (i.e. hand-maintaining pages of links)?
What if such an owner decides to link to a page of commercial or spam links - will they get any money from the owner of the linked site? Is this a possible method of abuse?
Will that cool poster of links between websites now become 3D to give trusted links more prominence?

Re:Questions

I just finished reading the paper. All these questios are pretty well answered by the text. To save you and others the trouble of reading it, I'm gonna take a stab at these. Feel free to actually read the paper and tell me if I misunderstood.

How is this different from applying a weighting to PageRank?

It attempts to detect clusters of pages which have few inbound links, which also propagating "trust" scores to all other sites by using their linking structure. For sites that have many inbound links (high scroring in pagerank), the authors claim this modification tends to classify spam and reputable sites differently.

Will the owners of the pages / sites deemed to fall within the set of trusted seed sites get any money for all their hard work (i.e. hand-maintaining pages of links)?

No.

However, they will get better search engine visibility, which is quite valuable.

What if such an owner decides to link to a page of commercial or spam links - will they get any money from the owner of the linked site?

The paper suggests using only highly reputable organizations with long-term stability for the seed pages. Government organizations, universities, very well known companies.

The analysis in the paper is based on a per-site graph, not per-page, by the way. They lacked the resources to try these computations on such a large data set.

Is this a possible method of abuse?

Presumably, the small set of seed pages/sites will need to be monitored by staff employed by the search engine company. If one of the trusted seed sites "went bad", they would need to be removed from the list.

Will that cool poster of links between websites now become 3D to give trusted links more prominence?

Probably not.

I can already imagine this...

So, links from pages of bad reputation give your page bad reputation?
I can see this already....

This page contains very objectionable content.
If you are easily offended, don't enter.
Blah, blah, blah.
Blah, blah, blah.

Do you agree to these conditions?
Yes [goatse.cx] No [disney.com]

Probably the other way.

No, probably the other way round.

If you're linked by a trusted page, then your rank goes up, but there's no negative for being linked by untrusted pages - your pagerank stays the same.

Similar to Advogato's?

This sounds very similar to Advogato's trust metric [advogato.org], which uses a "seed" of trusted accounts to filter out trolls/spammers. The difference might be that it should be even easier to implement in the case of web pages, because they already have links to each other, avoiding the reliance on users to manually "certify" other user accounts in order to build the graph.

TrustRank link broken, session expired

To see Google's TrustRank Trademark info on the USPTO site, click here [uspto.gov] , click "New User Form Search (Basic)", and search for "TrustRank".

A good sign

Google, as we all know, is a reputable service provider; they get the job done efficiently and innovatively. Now they are continuing their attack on the ails of the internet which was started by Gmail spam filtering. By developing this tool, Google is helping to clean the Internet up and enable it to become the massive source of pure information it has such potential to be. The "negative" sites on the Internet, such as keyword sites with no real content which invade search results, and the like are a bane to the community and by helping get rid of them, Google is yet again doing us all a favour. Google, I salute you.

Re:A good sign

Yes, this is a good thing. It might result in wiping out search engine spam, maybe. If the "search engine optimizers" don't find creative ways to cheat.

Let's not get overly optimistic about what this is going to do for the web... such as:

By developing this tool, Google is helping to clean the Internet up and enable it to become the massive source of pure information it has such potential to be.

What exactly is "pure information" anyway?

Consider my little website [pjrc.com]. Lots of pages about how to design electronic stuff. But we sell components that support those activities, so it's not 100% "pure", is it? You could consider all those pages as a giant ad for the stuff on the store section of the site. But most people would consider my pages on the more informational side (and the vast majority really are).

About once every 2 or 3 weeks, I get a call from one of these search engine optimiztion companies. Not sure if it's the same couple companies... I usually just say "no" and ask to be on their do-not-call list. They're mostly a bunch of slimey people and probably don't honour such requests.

But sometimes, the idea is tempting. I resist because I believe it's unethical, and ultimately a bad long-term investment. Still, to anyone selling via the web, even a tiny little 2-person company like me, the sales pitch is quite compelling. Pay some fee, traffic goes up, more sales, increase in revenue offsets the cost for the SEO's work. Maybe it's not so bad if they don't stupe to cheating.

Still, I resist because I know it's not a black and white distinction. It's a fuzzy line between the obviously good techniques (improving site structure, rewording page titles, etc) and the obviously bad (cloaked pages). I also just don't trust them.

But even the distinction between "pure information" and "spam" is fuzzy. I'd like to think I'm leaning towards the "pure information" side, but we do indeed sell products. It wasn't always that way... in the mid-90's, the site was smaller and hosted at a university and no products were sold. I had several people begging me to sell them a few of the parts needed for a project. Eventually, a friend started selling some stuff (prices were high, service poor), and so I took it over. Satisfaction with the site has improved dramatically since then!

Still, it's a fuzzy area between pure information and purely commercial, or advertising or spam.

I can tell you it's a lot more work crafting really good web pages than just writing a check to a seedy SEO company. But if these ranking algorithms really do improve to perfection, the response is probably going to be more and more pages appearing in that gray region. Increasing sales can pay for a lot of man hours to author more material that's compelling for visitors and truely does help them to solve their products (especially if they buy the described products).

So, in a best case scenario, these algoriths reaching perfection (seems unlikely) is probably going to lead to a lot more very good content, but content that revolves around pitching products (eg, infomercials), and not "pure information".

Ignore them Lord Google

your loyal minions know not what they do... Okay yeah. I think it's a smart idea. I hope it works. But there IS room for human error. (just as always)

I always find it annoying to find irrelevant pages. If this works I'm happy, else I wont be mad at my Lord. Just a little disappointed.

Is the paper even the same thing?

Given that one of the authors of the referenced paper is an employee of Yahoo, I have to wonder if whatever Google has in mind has anything whatsoever to do with the trustrank scheme we're talking about here. I mean, all we know is they trademarked the word, nothing more.

Trustrank explained

Trustrank is basically the same as resetting pagerank.

What happens is, that humans select some webpages which they trust. The idea is, that these trustworthy webpages only links to good sites. So, the trustworthy webpages are used as seed into a regular webcrawler.

At first glance, this looks like a low pass filter to me. Ie the same result could be achieved by cutting all PR 5 sites.

in toolbar

bah, if they could include a green-orange-red light in their toolbar... Go to your bank's website to "verify" your password and a little red light starts flashing in your toolbar? Could be good.

Yahoo behind trustrank

The funny thing is, that one of the authors of the Trustrank paper is from Yahoo.

Bring lawyers, guns and popcorn

Since an entire industry of sleezeballs has grown up around tweeking* Google page rank, I expect that we'll see quite a few lawsuits over Google changing how they figure out what order to present search results. (There have been a few over previous adjustments. [pandia.com]) Whine and cheesed sleezeballs. I'll stick with the popcorn and beer, thanks!

* When I say sleezeballs and tweeking, I mean the people who will try outrageous stunts to game the system, rather than the consultants who will help you increase rank by the stunning tactic of actually improving your site. Radical, but sometimes it works.

Gmail spam filter?

I read another post speculating that gmail users could be used as voters to choose trusted sites. Something that would probably actually work would be tagging domains that are received by a certain percentage of the gmail population and NOT marked as junk, and then giving them weight according to their percentage.

Becase we gmailers are picky.

It would probably have to be integrated with something else, because I bet there are a few pr0n mailing lists that lots of people have.

PageRank is already no more what it used to be

The google-watch page on PageRank [google-watch.org] already mentions how pagerank, over the years, has switched from an actual score of popularity (number of links to a page), to a trustrank-like index, based on the reputability of the links to a page. This makes it much harder for the newbie to get a good pagerank, and empowers way too much the owners of old web sites and corporate pages.

Even though it contains way too much rant for my taste, google watch [google-watch.org] is worth a full read by all /.ers.

WTF?

Are we now reporting Google news from the future?

Semantic Web...

I read a very interesting article on the possible outcomes of a semantic web, and a google "trust rank" actually appeared in it.

If "Google trusts fooPage" becomes a standard, recognised triplet, I see no reason why this won't be extended to "Google trusts userX", which becomes "ebay trusts userX" etc.

It's very possible they're looking to the future, and have more in mind than "there's probably no pr0n on this page"...

Question.

If I search for "stoned whores" what sites should be considered trusted?

This info is not intended to be read by a human.

This [64.233.187.104] is wierd. It is the 19th hit (on the second page) of a google search for "trustrank" [google.com] It requires a login from google's results page, but a google's cache reveals a directory including the paper linked to by /.

I guess we weren't supposed to read this. And you shouldn't have read *this*!

Personalised trust metrics

It would be amazing if Google gave us the ability to assign trust values to sites that we ourselves trust. This way, for example, I might give Wikipedia or the BBC a 10/10 trust rating for all their off-site links (and set it so that links off the linked sites are at 50% of their parent trust rating etc.). If we could also subscribe to someone else's trust ratings then technically illiterate people could hand over the responsibility of managing their trust database to someone else. From first thoughts, this looks like it could solve the problem of malicious SEO.

A possible system?

Couldn't they just look for links in gmail messages and use those as
weights in a trust system?

Links in messages identified as spam could be given a negative
weight. That weight could be determined by the number of people
identifying messages with that link as spam. Links from those sites
would being given less trust than a completely unknown page, unless they
are positively weighted themselves or linked to by a positively weighted
site. Links found in non spam messages could be given positive weights
by the same rules.

This would also have the advantage of offering spam filtering rules
based on trustrank weights. Setting a minimum trustrank would allow the
system to weight the email by checking the links in the email, and using
their trustrank for the message itself. The automated spam filtering
gmail offers could thus affect trustrank, increasing the impact of both
systems (email and searching) and possibly allowing it to be extended
to google groups/Usenet filtering.

Potential Examples

(moving each weight given by linking 1 point towards 0)

site1 [+5] - url found in 5 non spam messages
site2 [-5] - url found in 5 spam messages

site3 [+4] - url linked to from site1 (5 + -1)
site4 [-4] - url linked to from site2 (-5 + 1)

site5 [0] - url linked to from site1 and site2 (5 + -5)
site6 [3] - url linked to from site1, site3, and site2. (((5 + 4) + -5) + -1)

Email1 [-5] - contains links to site2, site4, and site6 (((-5 + -4) + 3) + 1)

Not perfect perhaps, but workable and easy to combine with a simple
rule set for weighting parts of a url to create an 'intelligent' system
guided by user preferences.

My vision for trust...

This isn't an original idea, but I can't remember where I most recently read about the concept, so I'll go ahead and say it's mine:

Trust for things like email senders and web sites shouldn't be centralized. My web of trusted entities, which should be easy to maintain (unlike, say, blacklists or whitelists) and should evolve semi-automatically, should be based on the interaction of my trusted sites/entities, and, in turn, their trusted sites/entities. Sort of like TrustRank, but where each person determines their own initial seed of trusted sites/entities. Of course, if you didn't want to deal with choosing seeds, you'd just pick Google as your trusted site.

This is of course a horribly abstract idea, and I have no idea how I'd implement this for 1 or a million users, but hey, you gotta start with the vision.

Sounds like a confused algorithm

I've read it but is sounds mixed up. Isn't the ideal result from a search engine:

Matches - spam - offtopic, sorted by relevence

not

Matches sorted by f(pagerank,trustrank)

Google used pagerank+on page text as a measure of how relevent a page is but thats not reliable anymore because the set contains spam pages.

The 'trusted' value tells you nothing about relevence, it only gives the likelyhood of the page being spam or not spam. If its spam you want it removed, if its not spam, then its page rank determines its relevence not some function of pagerank and trustrank.

i.e. they should not promote or demote pages because on trust rank, they simply define a cut off value K, if the trust is less than K then its likely spam and should be removed.

Since spam follows money terms, they should have K(keyphrase), so they can change the value of K on each keyphrase to remove the spam. Otherwise they will filter non money terms where no spam exists and their algo can only do harm!

maybe for Gmail

I think TrustRank would be more useful in Gmail to give a reading on how "spammy" an email is. They already have something like it, where a box shows up warning you that the sender may have spoofed their address.

Censorship?

Seeing a large number of replies so far, it appears that most people seem to see this as some kind of censorship.

I haven't read the article, but the name suggest they will do something similar to how pagerank works, not actually trimming the results, but re-ordering them. It doesn't hide any content, just displays the content that is more likely to be what you want, higher up.

Or am I the confused one here?

hmmm ......

... would be nice if you could use adblock style filtering on Google search results, then if you wanted to get rid of certain results, (i.e. from blog or "sales" sites), you could block their domains.

Probably wouldnt be that difficult to get around it but might help a bit

t

Vipul's Razor?

This sounds very similar to the Trust sytem used by Vipul's Razor and Cloudmark software. I have used the Spamnet product since 2001 and run Vipul's Razor on my mailserver, it is the most accurate filter that I've found (and believe me, I've tested them all). Kudos to Google!

certain domains given a preference?

Does this still mean that .edu, .gov, and .org domains will be given a bit more trust over .com and other domains? (just like pagerank)

Bayesian

When they said "trusted sites" I was all geared up for some sort of Bayesian analysis. After all, it worked for spam [paulgraham.com].

CollaborativeRank

This may be of interest: http://collabrank.org [collabrank.org]

Google and Semantic Web

Sounds like Google is gearing up for the sematic web, much like that mentioned in this Slashdot article. http://slashdot.org/article.pl?sid=04/08/01/205820 6&tid=217&tid=133&tid=95 [slashdot.org]

Re:Cheeseh...

That's why us open source programmers always throw out and completely rewrite our programs from version 2.6 to version 2.8

Re:Cheeseh...

You know, new solutions are most often patched old ones.

Should Google just throw away their many years of research, and start from scratch?

I find this trust-based approach interesting, but I wonder how it's gonna work for smaller sites (Which the few trusted seeds will not ever link to), but I guess the smaller sites don't really have a problem as it is, because only specific search-terms are targeted.

There's also the problem of allowing new websites into the game, but I guess that's for the Google developers to figure out. :)

Re:Cheeseh...

You fail to see beyond your nerd reality. This is not a "bit of a hack". If you are assuming so because of the name, it is obvious you know nothing about marketing. Even if it has nothing to do with Pagerank, Google will continue the xRank naming convention, as it is known and trusted. RTFP (paper) before you spout off that this is a "hack". It is a whole new methodology.

Re:Cheeseh...

The fact is, we really don't have enough information as of yet to conclude whether this is a patch to PageRank, or a secondary system, running along side PageRank. One can assume it to be the former, but the latter could work just as well with Google's new corporate concept.

Imagine going into your Gmail account settings, adding a string of a few websites you deem to be "superior" or of better quality, and then let TrustRank grab the collection of all of these, note where the highest votes go, and use these as more "Trustworthy" search results. Or, using PageRank, it simply adds an option "Vote these sites higher because they are linked to the user defined site settings."

Both schema make Search Engine spamming more controllable by Google (Simply by terminating accounts linked to spammers), and could have an interesting effect. Can't wait to see what happens with TrustRank.

Re:Cheeseh...

We need new solutions, not patches on old ones.

Hey, we're not talking about MS here, so drop the cynical patches line thank you. And "old tech" can still bring you the best general web search results out there, no matter av,yah,msn,dp,whatever. Ad 1, we don't exactly know what they will use this "new" word for, what solution will it cover under its terminology. Ad 2, if Google seems to work on something, that's always a bit of joy :) as we constantly get something from them, so it would be a cause to worry if they weren't on to something :)

Re:Google argghh!

Who cares if they change staff, it's not like it's going to impact on world peace/trade or anything else as a matter of fact.

Might impact the development of one of the most critical tools on the Internet, but you're right, that'd just be news for nerds.

Oh, wait...