Is Google Breaking Their Own Rules?

flood6 writes "Threadwatch is carrying a story about Google getting caught doing things they ban other websites for. Here is a page as viewed by the public and the same page as viewed by a search engine (their cache)." Note that the titles in the cache are employing classic keyword stuffing, presumably to improve rankings.

So what?

For now, the implications are simple - If Google can do this on it's own pages, why can ordinary webmasters not? Google's keyword stuffed, cloaked title would be hard to describe as anything other than an SEO tactic not so much frowned upon, but full on hated by the Search giant itself.

Why? Because it's their site and they are in no need to follow their own rules. They aren't going to ban themselves but they will ban you. If you want to be listed on *the* search engine then follow their rules. If you don't care if anyone finds you then you can modify your page during crawler indexing and other sites can pick you up.

Re:No, wait

They're stuffing results for the internal search here: https://adwords.google.com/support/?hl=en_US

All I see is people talking about how "dumb" they were to use such "obvious" cloaking techniques. Hello people, they were teaching their own search that is to be used on the adwords site. You don't tune your own internal search pages to help people find what they're looking for?

Sounds like a lot of people upset over nothing.

Re:No, wait

Sounds like a lot of people upset over nothing.

Welcome to the internet!

Agreed.

Yup. Looks to me like they're using the technique internally to file things orderly, since they're generating content that directly populates the database. The nice, handy newline between the keywords and the actual title in the HTML source also makes it trivial for scripts to strip it out later. If they were trying to hide something, they'd teach their cacher to delete the "secret" keywords.

In contrast, for ad hoc "discovered" content, such as what a web spider crawling the rest of the web might find, such practices are hardly benign. Google can trust its own vision of how it wants its database to look, but not the intentions Mr. XXX HardCore Anal Sluts, or the guy that has Ad0be Ph0t0sh0p for 75% off, or worse yet, the guy who wants to "verify your account-holder information"...

--Joe

Re:So what?

Remember the golden rule: He who has the gold, makes the rules.

Re:So what?

I prefer the other golden rule: He who smelt it, dealt it.

Re:So what?

I prefer the other golden rule: He who smelt it, dealt it

Obviously you are unfamiliar with the great flatulence fallacy. This rule is nothing but a ruse to distract from the real source of the foul odor.

The shit sandwhich rule

Whoever has lots of bread, doesn't have to eat shit.

Re:So what?

Absolutely. People tend to forget that google is a corporation. They can do whatever they want with their search engine. Their goal in life is to keep you looking at their pages and using their> search engine so that they can show you more ads! Its all about money. Google is not making a search engine out of the goodness of their heart.

Re:So what?

Absolutely. People tend to forget that google is a corporation. They can do whatever they want with their search engine...Google is not making a search engine out of the goodness of their heart.

Right. Other people tend to forget that Google is not immune from oversight and criticism because they are a private corporation, and it is fully justified to call them on their activities if and when they pursue questionable avenues. No one, at least to my knowledge, is calling for government intervention, but are merely spreading the word of potentially hypocritical activities. As a user of search engines I want to hear this public criticism as it may eventually make me switch to whatever the new search engine is.

As a sidenote, I find it remarkable how defensive the general Slashdot community is about Google. Let's try your post in a slightly different light and see what you think about it.

Absolutely. People tend to forget that Microsoft is a corporation. They can do whatever they want with their software. Their goal in life is to keep you buying their goods and using their software so that they can lock you in and sell you more! Its all about money. Google is not making software out of the goodness of their heart.

Re:So what?

" Absolutely. People tend to forget that Microsoft is a corporation. They can do whatever they want with their software. Their goal in life is to keep you buying their goods and using their software so that they can lock you in and sell you more! Its all about money. Google is not making software out of the goodness of their heart."

I hear your point but I would have used Sun instead of Microsoft. Since Microsoft has been convicted of abusing its monopoly power, they can't do whatever they want - hence the conviction.

Now come all the posts that say...

"Well, Microsoft has been identified as a monopolist, so they have to play by a different set of rules."

I see people trot out this line every single time someone suggests that Microsoft be allowed something resembling the rights of other corporations. It's a broad, sweeping statement which essentially says that since Microsoft was designated a monopolist, the government can arbitrarily restrict their practices as they see fit, with barely adequate explanations.

It's also completely irrational. Yes, Microsoft was identified as a monopolist. The result? They've had to change some of their practices and submit themselves to an increased level of oversight from various government institutions. It does not mean that they have given up all normal, reasonable corporate rights that are in the possession of every other company. The vitriolic hatred for Microsoft on Slashdot makes some people think that any restriction on Microsoft is a good one - that they should be hampered in the course of normal business as much as possible, and screw any idea of fairness. Some might say that this was only justice, since Microsoft presumably didn't allow fairness to competitors and that's why they were convicted. Well, it may fit your personal sense of justice, but legally it's not. The legal system has already meted out its brand of justice, which, materially, is the only one that matters. And the legal system didn't say that Microsoft must be obstructed in business whenever possible, at every turn. They still retain the right to play by established legal rules - and, being a paranoid, highly successful company, they're going to exploit those wherever they can. You might not like it, but it's their right.

Re:So what?

Absolutely. People tend to forget that Microsoft is a corporation. They can do whatever they want with their software. Their goal in life is to keep you buying their goods and using their software so that they can lock you in and sell you more! Its all about money. Google is not making software out of the goodness of their heart.

Honestly, if I were using Microsoft's search engine to search for information contained on Microsoft's own site I would certainly hope that they made the most relevant results show first.

We aren't talking about Google and Microsoft dominating the world. We are talking about Google forcing high rankings on their own content on their own search engine.

Re:So what?

If Google can do this on it's own pages, why can ordinary webmasters not?

Quite. In fact, other webmasters can do it, as much as they like. Google aren't the web police... you won't be arrested or have your DNS removed. You won't rank highly on Google, but that reward is in Google's gift... and you have no right to dictate what they do.

Re:So what?

Someone sue Ford then since their website is promoting their products over the likes of GM, Toyota, etc.

First off

I am not sure that it is true. But leaving that aside, Google does not have a monopoly esp one that is forced or supported by illegal methods. They are large, but they do not really control the industry. Yahoo carries a lot more weight than others credit them for. When Google fails to pick up something relavent then I go to Yahoo.

Re:So what?

WTF. I don't pay for my searches on Google. I just assumed that nobody else did either. So, what exactly do you think Google has a monopoly on, giving away search results?

Re:So what?

The reason this doesn't work is because Google can't "pay themselves" to use AdWords to pimp...AdWords. Besides, they own the company and make the rules. And maybe they should follow their own rules, but they most certainly don't have to. Besides, they don't block out competitor's results, they simply bring theirs to the top (Think: on-site searching, THEN offsite).

I don't see Google in the wrong here.

Re:So what?

I don't think Google should give preference to their own pages in their search engine.

The page was a google cache page. Have you ever been served a google cache page as part of a Google search? I am fairly certain I haven't so I don't believe that this page would be a 'preference' in their search engine.

Second, does anyone have ANY evidence that this page only has the keywords in the title BECAUSE it is cached. This could very easily be what the page WAS when it was cached, and someone changed the title at some point.

• I don't see this effect generated in other google caches:
• http://64.233.161.104/search?q=cache:desktop.googl e.com/ [64.233.161.104]
• http://64.233.161.104/search?q=cache:64.233.161.10 4/ads/ [64.233.161.104]

The whole article sounds like FUD to me.

By the way, to quickly get to a Google cache, try this bookmarklet:
NAME:
::Google Cache for this page
LOCATION:
javascript:document.location.href= 'http://www.google.com/search?q=cache:' +document.location.href.replace(/http:\/\//,'')

Re:So what?

There _is_ _no_ _evil_ here, they index their own internal pages with keywords because it's not going to have sufficient links for pagerank to work normally, It's gone now, it's probably a weekly/monthly process so that searches for AdWords comes up with relevant answers.

Geez, people love Google when they're small, then they start looking for a reason to hate them. This isn't it folks, keep looking.

Re:So what?

because they pretend like they're a NICE corporation that wouldn't do that sort of thing?

that being said, there's a lot more fishier things google does without giving any explanation at all(with googleads etc..).

basically they got the same stance as everyone else who's big enough: "we can do whatever we fucking want including not giving you your money and you can't do shit about it, read the fine print that says 'all your base are belong to us'."

Re:So what?

"Why? Because it's their site and they are in no need to follow their own rules."

No, because of the public relations and potential litigation. The public relations are bad because the public has a low tolerance for hypocricy. Google's main asset is the user-base. If they public turns against them it could do major damage.

IANAL, but just because it is their site doesn't give them free reign to do anything they want. Since they have such a large market share of the search services there may, perhaps, be anti-competitive laws that come into play for taking advantage of their market share to artificially promote their own services above those of competition, as was the case with Microsoft and a few other similar cases we've seen lately (e.g., VoIP blocking). These might not be the case exactly here, but it is inching closer.

Relevancy is the key

More to the point: ignorning the fact that this is google sending keywords to google; ignoring the fact that this their site and they can do whatever they want; ignoring the fact that people are using words like "evil" to describe something that affects exactly no one; ...

The key point here is relevancy. The keywords are relevant and accurate. You might say that this breaks Google's style guidelines, and that's a good reason for them to bug-fix it. But, I fail to see how this is some great transgression on Google's part. This is USEFUL INFORMATION that they are putting in the title. Ugly, sure. I hate when eBay does the same thing. It's still not keyword spam, and it's still not cloaking. Cloaking is when you pretend to the search engine that you're a different kind of site so that you get ranked in with that kind of site. It's not putting keywords in ugly user-visible places when they are relevant.

Please return to your useless ranting about Microsoft or something.

Re:Could be keyword stuffing...

It's the fact that the title is different when a search engine views it versus when a person views it. Feeding different information to a search engine (with more keywords) is currently frowned upon, as people have abused it.

Check the *title* of the two links. One has a comma separated list of keywords.

--
Evan

Re:Could be keyword stuffing...

Next check the robots.txt of their site. First item is Disallow: /search? The evil URL in question is starts with /search? Thus google does not want that page crawled by other websites. How does one stuff keywords on a page they don't want indexed?

Re:Could be keyword stuffing...

RTA (http://www.threadwatch.org/node/1774) - the same pages used to be normal, now they're stuffed with keywords.

Re:Could be keyword stuffing...

Did you even READ TFA? Google gives different pages for users and its own search engine. The user's pages are NOT stuffed with keywords, while the ones for its search engine are. This is OBVIOUSLY keyword stuffing and cloaking.

first cache?

insightful interesting insightful interesting insightful interesting.

for fun...

just for fun, insert 'Microsoft' in this discussion everytime someone writes 'Google' and see if you feel the same way.

There IS a difference.

Google's business model DOES NOT rely on trapping users and forcing - practically blackmailing - it's victims to make exorbant payments for upgrades, Google DOES NOT have a death grip monopoly on the consumer Search Engine market, and the page in question does not further any political, social, business, economic, or other goals.

Is it shifty and underhanded? Indeed, but Google has had a history of being a benign company, and as such do not deserve the same treatment as an actively malicious company.

By the same logic which you have applied here, what would you be feeling if the names "Mother Teresa" and "Osama Bin Laden" were transposed?

Only one long term solution:

WikiSearch (or something like it). Long term we need an open peer-reviewed crawling and serving mechanism as bad as we need free OSs and browsers. How this is developed or funded I am not sure, maybe it will be the next breakthrough in P2P that obviates the need for the massive datacenter.

Until there is a free and open search engine, you are beholden to whatever these firms wish to do.

Re:Only one long term solution:

You mean dmoz? Or deli.cio.us? Or any other distributed referral system?

-theGreater.

One thing I'd point out

The keywords Google added to their title are limited in number and relevant to the actual page. This is rather different from the practice of a lot of SEOs of stuffing with several dozens of keywords and stuffing keywords that have nothing to do with the content of the page itself. And I notice that a lot of the SEOs squawking about this issue are among the worst offenders for high-volume irrelevant-keyword stuffing. Something to think about.

Re:One thing I'd point out

Couldn't have said it better myself. I'm glad someone else did, I wasn't quite sure how to word it.

Adding keywords like 'traffic monitor' and such to a page about traffic monitoring is very different than stuffing 'PARIS-HILTON-XXX-TAPE-FULL' into a page about home equity loans.

Google doing this in-house also gives them tighter control over what is stuffed where. Of course, this could be used to ensure quality hits, or simply elevate pages THEY want to the tops of the ranks. Searchola anyone?

Anyone else notice how there seems to be alternating instances of Google-scandal articles and Google-innovations every single day? Tomorrow we'll find out that they've made the internet obsolete, and in doing so, they've skewered a number of kittens on spikes. Oh the horror!

Re:One thing I'd point out

Or maybe, just maybe, this whole thing is much ado about nothing:

bash-2.05b$ curl http://adwords.google.co.uk/robots.txt
User-agent : *
Disallow: /

User-Agent: Googlebot
Allow: /
Allow: /support/
Disallow: /*?
bash-2.05b$

In case it's not inherently obvious, that means no other search engines will even see the page. So that means that Google's results are being skewed by.... (wait for it) Google.

Re:One thing I'd point out

Yes, and what's stupider still is if they wanted to skew their results, they could just modify their own engine to do so, and no one would ever know. On the other hand, see that little box at the bottom that lets you search the support site... and the fact keywords in the googlebot cache version are all related to the page in question. There couldn't be any... connection... could there? Like making their search page more accuract for their customers?

NO! This must be a conspiracy by Google to destroy all other search engines by polluting them, while boosting their own pages. There's no other possible explanation! *sigh*

Stop the presses

News at 11 ! Google is promoting themselves on their own website!

Hrmmm

Tools -> Chrange browser Identification -> Other -> Googlebot.

Nope... no change here.

Isn't it possible that the TITLE entry in the google cache database got corrupted for this page?

Re:Hrmmm

Changes for me. [img44.exs.cx] (Shown with Firefox User Agent Extension settings).

so....

...this means when I search google.....for things related to google, google pages will make it higher in the search results?!?!

I feel so betrayed!

Note that they've done this

just in order to have high rankings on the other search engines...

What?

Are they suggesting that google has to resort to keyword stuffing on cached pages to get a higher ranking on their own search engine? Is it me or is this unbelievably stupid? Surely, if they wanted too, they could just have their own pages rank top of whatever searchs they wanted- keywords or no keywords? Just some find of google flag in the ranking algorithm and they'd be done.

It's global

It's also done globally [google.com] as the article pointed out. Sneaky sneaky google.

(This still isn't evil by googles definition because "Evil is what Sergey says is evil." [style.com] and this tactic propably adds some additional millions of dollars to Sergeys pocket)

Big Fat Deal. Live with it.

Google doesn't need to stuff keywords for their own site - they could make their own Adwords page the only thing you ever see if you search for "traffic estimator." And why should Google care about stuffing keywords for Yahoo or Microsoft's earch engines? They don't control what you do for other search engines, either (if Google knew that your site only keyword-stuffed for MSN and Yahoo crawlers, would they care? No, they'd probably high-five you for screwing with their competitors' relevancy). There's no hypocrtical behavior here.

Brittant Spears

Their next step is to put on bottom of google.com front page, in font size 1, white foreground on white background:

"britney spears
brittany spears
brittney spears
britany spears
britny spears
briteny spears
britteny spears
briney spears
brittny spears
brintey spears
britanny spears
britiny spears
britnet spears
britiney spears
britaney spears
britnay spears
brithney spears
brtiney spears
birtney spears
brintney spears
briteney spears
bitney spears
brinty spears
brittaney spears
brittnay spears
britey spears
brittiny spears
brtney spears
bretney spears
britneys spears
britne spears
brytney spears
breatney spears
britiany spears
britnney spears
britnry spears
breatny spears
brittiney spears
britty spears
brotney spears
brutney spears
britteney spears
briyney spears
bittany spears
bridney spears
britainy spears
britmey spears
brietney spears
brithny spears
britni spears
brittant spears
bittney spears
brithey spears
brittiany spears
btitney spears
brietny spears
brinety spears
brintny spears
britnie spears"

Re:Brittant Spears

Actually they really did it... I got the list from Google itself =p

http://labs.google.com/britney.html [google.com]

Google hasn't done a very good job...

Searching for "search engine" only brings up google in 5th place. They're certainly doing a shoddy job of being unfair.

I find it funny how everyone is so pro-Google

If it was MS (if you use msn search or whatever their search engine is) made their products come out on top using such tactics then everyone would be trying to join the lynch mob. However if google does it its fine.

What I do find interesting is that they needed the keywords, and didn't just raise their rank artificially. Does the google algorithm not have such a feature in it (or not have it easily accessible)? Potentially it does but google chose to not use it. In either case this is nicer than what I'd see other companies doing in such a case, since I doubt they'd bother with keywords on their own search engine.

huh?

I don't get it. The two pages look the same to me.

Is it the highlighting? They always do that for pages that you find in the cache.

Are you joking?

I think this is actually handy. Google is simply altering the title "with the keywords you searched for". So that you can see the Google cache page in your title bar without seeing "Google cache", blah.

I don't see this as anything sneaky just something to help people. Why would Google want to alter the page rank of a cached page anyways?

Seems like a post to grab some hits on http://www.threadwatch.org/

Lame

obligatory Beastie Boys

You pop caught you smoking, and he said, "No way!"
That hypocrite smokes two packs a day.

In other news...

Google bans google webpages from google search engine.

It's been changed!

A lot of people are asking, what's going on here? The pages look the same now, but they were different before.

The original article said:

If you look at this Adwords page on Google you'll see at the top of your browser, the title:

Google AdWords Support: How do I use the Traffic Estimator?

That's what normal visitors like you and me will see when visiting the page.

Now have a look at Google's cache of the same page - Notice the change in the title? It now reads:

traffic estimator, traffic estimates, traffic tool, estimate traffic Google AdWords Support ...

But now, the links point to a different page. It is no longer about "Google AdWords Support: How do I use the Traffic Estimator?". Now the page is, "Why do traffic estimates for my Ad Group differ from those given by the standalone tool?" It's a completely different page on a completely different topic. And for this page, there is no difference between the cached and direct views.

That's why people are scratching their heads.

I don't know whether Google did this to cover up their actions when they got caught, or whether it was a simple and routine rebuild of their help database which caused page numbers to change so that the links no longer point to where they did before.

Who is more evil?

Critics have been pointing out little bits of evil at Google for quite some time now. Autolink and self cloaking are merely the latest blips. If you don't like it, stop supporting Google. Every time you use Google, you are casting a vote.

Begin using other engines and break the homogenization of the search engine market. We are better off with competition and multiple viable search services.

One important fact left out of the article...

There's just one thing the article and the summary fail to mention:

bash-2.05b$ curl http://adwords.google.co.uk/robots.txt
User-agent : *
Disallow: /

User-Agent: Googlebot
Allow: /
Allow: /support/
Disallow: /*?
bash-2.05b$

(Try it yourself if you don't believe me)

What that says is "Prevent any user agent from indexing anything below the root hierarchy, unless it's Googlebot, and then only allow the root level and /support/"

So, no other search engines should ever be seeing this page. Basically, Google is using their own search engine to also index their own support information. And this is a problem because.... why?

Not too evil...

Google can't be that bad...

After all, they're not censoring this from their pages [google.com]

Only affects their own searches

It's not a case of showing different results to other search engines. It's a case of showing different results to just themselves only. It doesn't affect Yahoo or MSN search - they still see the same page everyone else does.

Google's complaint with other websites is, basicly, "When we hit your site, please show us the same thing you show everyone else." Thus they aren't breaking their own rule, because they ARE doing that to the other search engines out there. They are only 'lying' to themselves.

Let's say MSN did the same thing, and rendered keyword-stuffed results for their own searches on their own sites, but still showed the same page to all external visitors, treating google no differently than an interactive user. Then it wouldn't harm Google's search in the slightest (and in fact google's search would end up being better than MSN's search on their own site because it wouldn't be tainted by the keyword stuffing). Similarly, what google did doesn't harm the other search engines in the slightest, and in fact makes them a tiny sliver more accurate than google is.

No, this is not the same thing that they are complaining about. They don't mind in the slightest if other search engines lie to themselves, so long as they don't lie to google, and google can lie to itself so long as it doesn't lie to other search engines.

Why bother?

If google really wanted to boost their own pagerankings, why go to the trouble of making keywords for specific pages? Wouldn't it be easier to tweak the algorithm so that google pages automatically get a certain number of points (or however they do it) bonus?

Re:Have I missed something?

This is NOT keyword spamming.

Keyword spamming is when you put UNRELATED keywords in the title or "keywords" headers of a page.

For example, if your page is a pile of ads for random stuff and your keywords are "tequila, mp3, oscars", then that's keyword spam. Putting the keywords in the title was a way to get around anti-keyword spamming techniques for a while. Many have said that putting keywords in the title is a bad thing because it results in unreadable titles, which is true.

Google has no circumvented that by putting readable, usable titles in the pages served to users and relevant, but verbose titles in pages served to crawlers... and this is related to keyword spamming how?!

Re:Grammar Dork Says...

That should be, "Are Google Breaking Their Own Rules," not "Is Google..."

Sure, that agrees now, but it still sounds bad. "Google are really cool!" WTF? Just because a corporation consists of multiple people doesn't mean it's plural. The headline should have been, "Is Google Breaking Its Own Rules?"

Oddly, this is the ONLY thing I get pedantic about when it comes to grammar.