Google's Site Ranking Secrets

vivin writes "Ever wonder how Google's site ranking works? Wonder no more. Google recently filed United States Patent Application 20050071741 on March 31, 2005. This patent reveals a great deal of information about Google's site ranking algorithm and makes very good reading. For example, one of the criteria that they use is the number of years that your site has been registered. If your site has been registered for less than a year, then it counts against you. A site registered for a longer period of time means that the owner is probably serious about the site, and the site is probably legitimate. Google's Site Ranking algorithms reveal how hard they are making it for spam sites to get listed (on Google). This information will also make it easier for you to make sure that you get listed well in Google."

Note

Note that there is no guarantee that Google uses everything in the patent or that they don't use other methods not described in any of their other patents.

Re:Note

Then what is the point in patenting the method if they aren't going to use it?

Uhhh...to prevent others from benefiting from it? That's what patents are for. They say it there to promote innovation. It protects the owners exclusive control in the hope that he might reveal his idea to the world. More often than not, what really happens is that the owner will put the invention on the shelf because a)it competes with other inventions the owner may have on the market, or b)like a land or commodities speculator, he's holding out for an exorbitant price. Note that also more often than not, the owner of the IP privileges is not the creator. Patents are bought and sold like poker chips. While the actual device rots. Only the paper pushers benefit.

what are they trying to protect then?

Their advantage over everybody else.

or

or conversely how spam website can get higher :)

Real Explanation

I prefer the official Google explanation:

http://www.google.com/technology/pigeonrank.html [google.com]

Speed of gaining links?

'Google record the discovery of a link and link changes over time. The speed at which a site gains links and the link life span.' I fail to see how this would be helpful--if something's new and briefly popular, you only want to give it a high rank for a brief period and forget it once people stop linking. But if something's new and popular for a duration, you want to keep it well ranked.

Re:Speed of gaining links?

If you have a good site with valuable information, then, over time, news of it will get around, and you will keep getting new links over time.

However, if you have gotten 1000 links at once, and for the next months noone else is linking to you - then you have probably bought the initial links, but nobody real considers the content worthy of attention.

Spammers killing Google

Could someone explain how other crap search engines are getting high rankings in Google search?

Sometimes when I search for something specific, I get a bunch of useless links that have results of other "search engines" that invariably show something similar to "0 results for your search terms 'sheep+barn+slashbot+erotica'"

How do these sites get on the first page of Google results?

Re:Spammers killing Google

Yes, they're really annoying. They often have genuine looking summaries in Google's results, inticing you to click on them expecting to find useful information, but all you find is a page of links, often completely unrelated to your Google search. I wonder why Google hasn't got on top of them yet. All it would take is a second robot identifying itself as Internet Explorer slowly crawling the web looking for pages that give completely different results than the google spider.

Doesn't work, see explanation

This type of spam (showing a page to the crawler and another to the user) is called cloaking. Cloakers have anticipated this sort of move and can detect a search engine's crawler by not just the user agent but also the IP address range it comes from and other heuristics. In order to beat them, search engines would have to crawl from unpredictable IP addresses and behave like regular users.

A while back I proposed a distributed approach like this in the Nutch mailing list [mail-archive.com]. The problem is that it would be hard to implement and it may not be worth the effort, since there are cheaper ways to fight spam.

Re:Spammers killing Google

They are scaper sites.

They get to the top through link spamming, 302 hijacks, "scaping" content from other sites, search engine optimisation etc etc etc.

They are sites "made for adsense" as its called, whereby they exist for the sole purpose to be highly ranked in google and get ad clicks from people looking for something else. Effectively 'doorway' pages, which make a shitload of money, as people that land on such pages don't find what they really want, so click through on the ads in hopes of finding it there instead.

The crap of the internet, many hundreds of thousands of such sites run by only a hanful of thousand very rich people.

Re:Spammers killing Google

Could someone explain how other crap search engines are getting high rankings in Google search?

That's not as bad as getting mailing list archives. You search for a particular Linux error message, and what you get is archives of mailing list messages of guys who ask precisely the same question, with a lot of "me too" follow-ups, but no definite answer to your problem, that is if you can manage to find the link that leads to the follow-ups... Because heaven forbids mailing list archive software offers standardized navigation...

Re:Spammers killing Google

Try figuring this [ruby-talk.org] wonderful navigation out and I'll give you a cookie!

Hey, there's a help button... *clicks*... Oh God...

Its a patent... and a laundry list...

They've thrown every technique they could have thought of into the patent purely as a defensive mechanism to prevent other major engines from patenting them. Some of the techniques are thrown in as defensive FUD to prevent newbies from using them.

Some of these techniques are just plain old bizzare and might be way too difficult to approach algorithmically.

Oh well .. what do I know ..

Re:IMNAPL

Each claim in the patent can be invalidated independantly. Most patents start off with an all encompasing claim 1 that would almost certainly get invalidated if it went to court, and define subsequent claims more narrowly, often in terms of the preceding claims.

SEOs make me barf

Argh... quit trying to game the system! If you read the article, it's entirely from the perspective of someone trying to corrupt the rankings for financial gain. Here's an idea: make good, useful web pages, rather then spending all your time an energy creating these BS link farms. The SEO world is the modern day equivilent of snake-oil salesmen.

Re:SEOs make me barf

I agree they can be evil...but one thing Google lacks is giving new sites some priority....say i come out with the best tech site ever, but I have no money to advertise with, how do i get it popular? Ok i submit it to Google. I appear on page 5000 of the results. I have to beg people to link to my site, maybe spam a couple of blogs, i dunno...the thing is without the tricks, its almost impossible to get your new site to appear in the search results. And even with them its still pretty difficult. I think maybe google should have a special section of "new to the web" or whatever to give these sites publicity. In the old days, the yahoo directory kind of put all decent sites on even ground.

Not that simple

No, its not that simple. Lets say I have a small business, I sell garden tools, lawnmowers,etc, in a certain region. And yet I do a search on google for garden tools + region, I am nowhere to be found. What do I do? I optimise the hell out of my site, caking it with region name + garden tools information, and I set up a links exchange program, getting in links left right and centre from related sites. This is SEO, and it will only affect people that enter a search for "garden" "tools" "my region". In other words, those that actually want to find my site.

Theres a distinction between SEO and spamming; if I was to optimise for a garden tools site and set up a poker site there, that would be spamming.

Re:Not that simple

I hate SEO with a passion (and I have to do it as part of my job), but you're right.

The way I see it, SEO is a tool - nothing more, nothing less. It isn't inherently evil or inherently good - it's how you use it and what you use it for that matters.

If you've got a good site on... i dunno... aardvark polishing for fun and profit, then you should rank highly on Google. If you don't rank well on Google, it's probably because your site is lacking one of fame, content or clean code. All of these are necessary for (or inevitable side-products of) a good site that does what people want.

Conversely, a good site will probably have many inbound links, clean semantic markup, well-focused pages full of good content and so on. This is simply good site design (or, like the links, a side-effect of it), but it's also the very ethical end of the SEO spectrum.

Now, you also get evil scumbag fuckwits-for-hire who specialise in link-farming, keyword stuffing, cloaking and other black-hat techniques, and sell their services to shitty pr0n or spam sites. This is spam - no doubt about it - but it only represents the black-hat side of SEO.

The black-hat SEOers, it must be admitted, are the one which gets all the attention. They're the ones advertising like mad, making overblown claims, spamming search engines with crap listings and generally getting in people's faces. However, just because these people use SEO doesn't make SEO bad. Before SEO they were likely sending e-mail spams until that got too hard, but you don't unilaterally brand professionally-looking e-mails or people who sell mailing-list managers as evil, do you?

As Google et al. get their acts in gear and revamp their algorithms, "SEO" is increasingly overlapping with "good site design" - this was always the intention, and even now "white-hat SEO" and "good site design" are pretty much synonymous.

SEO isn't the problem - the problem is a combination of shithead black-hat SEOers, Search Engines inadequately assessing a page's worth and ill-educated types who shortsightedly blame the gun or bullet instead of the guy who fired it at them.

Re:Temporal data.

Umm, you spelled 'genius' wrong, genius.

I thought so .. changed my site from .ro to .com

I always suspected this... When we've started our business, we used the domain www.interakt.ro [interakt.ro] (we're from Romania). However, because we sell software tools mostly to the USA and Western Europe, we've decided to go to www.interaktonline.com [interaktonline.com].

Instantly, our ranking went from number one (for "Dreamweaver Php" for example, we were number one there instead of Macromedia itself a long time), to page 10.
Now, we're working hard to promote our site, we have links all over the place, but still our site don't get up again to page 1 (search for "dreamweaver extensions" - we have to pay to get our site in the first position). I even thought that they do this on purpose for us to continue to pay on Google Ads :D

Probably they say it too in the patent, but the best ranking tool is to use the right "title" tag in your pages. It's invaluable how well this scores as compared to the page content.

Alexandru

Re:I thought so .. changed my site from .ro to .co

And another small note... Initially, we have used an HTTP 403 (Permanently moved) from interakt.ro to interaktonline.com. This caused us a MASSIVE degradation of our position, so right now we just do a transparent redirect from interakt.ro to interaktonline.com, without the Permanently moved headers (and this is how we've reached page 2...)

Alexandru

Foiled Again Google!

Nothing in the patent nullifies my pagerank defeating technique - put lots of links to my homepage [paradise.net.nz] in slashdot posts modded to +5 funny!

PageRank

The article dedicates only a couple of paragraphs to PageRank, the main algorithm that Google uses, and about 2.5 pages to the rest. If anyone wants to know more about PageRank, here's Page and Brin's original paper: http://www-db.stanford.edu/~backrub/google.html [stanford.edu]

About the autor

bout the Author
How-to-make-money-online.info is a site focused on Making Money Online and Internet Marketing, listing the many and varied ways of making money online. Featuring, resources, thousands of Internet Marketing articles and useful links.

This article comes with reprint rights. You are free to reprint and distribute it as you like. All that we ask is that you do not make any changes, that this resource text is include, and that the link above is intact.

So that explains a lot. What a crappy article, I wonder if the submitter is the same as the Author?

Registration Age vs. Registration Duration

Just to clarify, from the summary:

one of the criteria that they use is the number of years that your site has been registered

is not the same thing as (from the article):

How many years did you register your domain name for?

Though the summary suggests that older sites do better, the article is stating that, in order to improve one's Google ranking, domain owners should purchase longer domain registrations.

Impossible? Spyware?

Some of the tatics detailed in the article require a spyware (google toolbar?). It is not possible for google to know when you came back to the search engine from your site, or another one (unless you have a link in your site to google). It also impossible for google to know if you have a bookmark.

Google does have a click-through engine attached to the results, but many people find this in adition to the single identifier cookie that googles push into you abusive already.

We all thing google is doing a good job, and it did managed to incorporate adds and an add service that is well accepted by the people. (I wonder why people still think it is a good idea to make blinking and noisy flash adds?) The point is how much we trust google? I personaly don't mind very much the click through, but do not accept the cookie and will not install a toolbar.

more on the subject

The story is so old I can't believe it made it to slashdot.

Some more on info the subject:
1. U.S. Patent Application [uspto.gov] - it's best to read what's exactly been patented.
2. interesting discussion on webmasterworld [webmasterworld.com]

Personally I think that while some of the stuff is interesting, most of it is made up rather to confuse SEOs (google doesn't quite like them, you know that, right?). Before that, they had couple factors to think about and work on. Now, there's a shitload of stuff that just makes their work harder. Also, more factors influencing SERPS means it's much, much harder to make a trial-an-error research on what works well and what doesn't.

Spammers

Google's Site Ranking algorithms reveal how hard they are making it for spam sites to get listed (on Google). This information will also make it easier for you to make sure that you get listed well in Google.

Won't this information now make it easier for spam sites to get listed?

Just look at the patent application

Jeez, news for nerds, and the story was a badly edited blurb referencing a badly edited blog that didn't reference the patent application.

Just look at the patent application yourself [uspto.gov].

I haven't read the whole thing, but just having taken a quick look at it, I have to agree with the posters who said that Google purposefully tried to cover any conceivable technique to index and rank pages. The application discusses multiple implementations of the various techniques that could be used to rank a page. Therefore analysis of the patent application is probably of limited utility for those trying to game PageRank (which was certainly a factor that Google's very competent IP lawyers considered before prosecuting the patent).

For those who are worried that Google is doing evil with this patent application, given the breadth of the patent and the fact that it discusses a plethora of techniques which Google may or may not be using, I will be surprised to see Google try to use this patent (or be able to use this patent) to push another search engine out of the market. More likely, I think, is that this will constitute prior art to enable Google to withstand challenges from other patent applicants for infringement. Of course, if you know anything about PageRank, you know that it was getting published in Scientific American long before Google was the dominant search engine. So this patent application is probably more to prevent allegations that Google infringed by adding on all the other checks and balances to the original PageRank technology to discourage spam sites.

Moiche

I find this quote funny:

There's a lot to take onboard here and consider. But you can't go far wrong with your SEO if you try to grow your site as organically as possible.

If any of you have worked in a small online shops you know what a fucking holy war this is between marketing and pretty much everyone else. I specifically remember saying at one point, "Do we have to make ALL of the money RIGHT NOW?"

Good for Google for coming forward and telling peole they won't be a part of that slimy shit.

Bad for Google for saying all of this to drive up prices on their AdWord sales.

Doesn't mean it really works this way

Remember, this is a patent which requires no working model. In other words, this could be how Google *envisions* their search working as much as it indentifies any of the things it does do.

My experience with FunWithHeadlines.net

I started Fun With Headlines [funwithheadlines.net] a few years back, and with no advertising on my part I was suprised how quickly Google picked me up. Right now I'm about the 5th or 6th result when you search for "fun headlines" and (obviously) the 1st when you search for "fun with headlines." At times I have been the 1st for "fun headlines," and at other times I have been around 10th.

OK, so there aren't that many sites like mine, let alone sites that update daily over a period of years and include their entire archive on the site that grows daily. On the other hand, to my knowledge from doing searches on Google, I have very few site that link to mine, and I thought that counted highly with Google. So basically without trying to game the system, let alone advertise my site (other than incidentally in comments like this), I've been treated really well by Google.

In my case, it must be the longevity issue coupled with the scarcity of sites like mine. It sure ain't the links to my site.

So...

...that whole pigeon thing was a joke? I can't believe it. Maybe this filing just a way to divert our attention?

Uh oh, Google applied for a patent!

Alrighty folks, you know the drill! Google filed a patent, ready pitchforks!!

Other, better approaches to search engine spam

A stronger approach would be to find out who really owns the site.

For example, let's search Google for "london hotels", a common search phrase. The first return is LondonNights.com [londonnights.com]. "Whois" returns "Worldview Ltd, 16 Marine Road West, Morecambe, LA3 1BS, Lancs, GREAT BRITAIN (UK)."

That's a UK company, so we look it up at Companies House. [companieshouse.gov.uk], where we find "WORLDVIEW LIMITED, 16 MARINE ROAD WEST, MORECAMBE, LANCASHIRE LA3 1BS, Company No. 04588973". So we have a match on a registered company.

We check further with Dun and Bradstreet [dnb.com], which has a worldwide database of companies. We find "WORLDVIEW LTD 16 MARINE RD WEST MORECAMBE , UK Type of Location: single"

So they pass company validation, and we can get financial information about them.

Now let's try a domain that just appeared in a spam: "fleagroups.com". "Whois" gives us "Flea Market Groups. 126 73rd Ave N., Coral Springs, Florida 34992. US" So we go to Sunbiz, the Florida State Division of Corporations [sunbiz.org], and search. No "Flea Market Groups" under fictitions names. No match on address under anything beginning with "Flea". No "Flea Market Groups" under corporations, and no "Flea Market *" address matches.

Looking in Dun and Bradstreet, there are "Flea Market *" hits, but no exact match and no address match.

So they fail company validation. Add to probable spammer list, drop search engine ranking.

This is a reasonable test for any site that appears to be selling something.

Number of years the site has been registered

For example, one of the criteria that they use is the number of years that your site has been registered. If your site has been registered for less than a year, then it counts against you.

So I get the following:

Date: 2 Jun 2005 11:42:45 -0000
From: Bettina Jensen <[e-mail]>
To: [e-mail]
Subject: [#17922] Buying your domain: vad1.com

Dear Webmaster

I am interested in buying your domain vad1.com for $400.
I'm only interested in the domain not in your content, so
you can sell your domain and move your content to another domain.
If you are interested please respond to this e-mail.

Regards,

Bettina Jensen

cat got my tongue

Google's Site Ranking algorithms reveal how hard they are making it for spam sites to get listed (on Google).

And provides a list of techniques for spam sites to use that guarantee them positions on every search engine but Google (in fact, if you use these techniques it's illegal for other search engines to penalize you for them.

This could be an especially evil technique for spammers.

1. Set up search engine.
2. Build some spam sites using search engine optimization techniques.
3. Modify your search engine to penalize people using your optimization techniques.
4. Get a patent.
5. Profit, either from your increased search results in Google, or from suing Google for violating your patent.

Re:ATTENTION

You're right! You're so right! I was lured in at a young age by a friendly smile [thewag.net] and a promise of free software, but now I realize my error. I want to convert; I've been led so far astray, I don't know how to get back. Can someone send me a copy of Windows ME so I can regain my sense of self independent of a prescriptive, prohibitive monoculture of an operating system?

I want to change. Please help me--I don't think I can do it on my own.