Mr Anti-Google 531
MrNovember writes "Salon is
running a story on some guy named Daniel Brandt who they call "Mr. Anti-Google." Mr. Brandt runs a sort of anti-establishment database of citations called NameBase as well as Google Watch. He claims that Google's PageRank system is undemocratic primarily because it doesn't rank his NameBase information very highly. He also points out that Google maintains a log of all you've ever searched for associated with a long-term cookie. Google's system seems to work the best if you ask me but, on the other hand, link popularity may not provide the most intelligent top rankings."
better? (Score:2, Informative)
Article (Score:2, Informative)
http://www.salon.com/tech/feature/2002/08/29/go
Re:Article (Score:5, Informative)
google is VERY democratic in nature... (Score:2, Insightful)
Re:google is VERY democratic in nature... (Score:3, Funny)
To Salon's discredit, they appear to agree with this idiot, evidently finding some sort of nostalgia in his fact-averse crusade.
Why isn't there a link to the ARTICLE? (Score:2, Redundant)
Re:Why isn't there a link to the ARTICLE? (Score:2)
Nobody reads the articles anyway, right?
Actual Link to Story (Score:2, Redundant)
Slon Article [salon.com]
Oh, I see (Score:5, Funny)
Google Cookie Management (Score:2)
Good thing I search for p0rn with cookies, Java and JavaScript turned off! I also wipe my disk cache between sessions.
Reality Remains (Score:3, Insightful)
we'll fix that (Score:5, Funny)
Gee wow (Score:2)
If a popular site links to yours, that has more weight than some one-off site that links to yours.. google takes this into account.
The guy can argue all he wants.. google does not pruport to have the best stuff at the top all the time.. but if this guy's site was so good, then more people would link to it, if more poeple linked to it, it would be more popular.
Re:Gee wow (Score:3, Insightful)
If for instance, some posted decided post a single link [copacetic.nu] to some obscure worthless website that nobody's ever heard of, let alone linked to in a comment such as this one, it will be ranked accordingly to the total of slashdot's calculated 'popularity' based weight.
Google does not, and probably can not distinguish between actual content on this site and inane comments made by people such as myself.
Is this a large flaw? I really can't say, and I certainly don't have a solution to propose. I still say google is the best thing out there, and beats the hell out of inktomi's paid listings, which power an ever growing number of search engines.
Re:Gee wow (Score:3, Funny)
So this link from /. should help him out quite a bit? Hooray for everybody!
I don't know about everybody. It might draw him here and the next thing you know, he'll have a site called slashdot-watch.com to complain that his posts aren't being modded up to 5.
Paul
Chances are... (Score:5, Funny)
Link popularity works in the long term (Score:2, Interesting)
Compared to the other search engines, Google is great, and that's what matters. Is it possible that someone could make a better search engine? Maybe. Please, try. Competition is good for everyone.
I HATE GOOGLE (Score:3, Funny)
1) When I type in my name IT DOESNT SHOW IT!
2) My websites are not listed #1 NO MATTER WHAT YOU TYPE IN!
3) There image search doesn't have PHOTOS OF ME!
4) I hat all other search engines for THE SAME REASONS!
wa wa wa......
Hmm, that's odd... (Score:2, Funny)
Oh, the humanity (Score:2, Funny)
Caching has been disabled for the site.
Google Cookies (Score:5, Insightful)
As for the point made that this guy thinks that Google is "undemocratic," give me a break! Google is not a government - it is a search site! They exist to make a profit. They will make money by providing a quality search result, thereby attracting users. They are not in the business of being the arbiter of democratic principles on the web.
Re:Google Cookies (Score:2, Insightful)
Capitalism and Democracy are rarely congruent.
Re:Google Cookies (Score:3, Interesting)
As in this example search for snowboard retailers [google.com], Google even tags the top results as "Sponsored Links" so even the searchers know that those sites are ranked first because they paid Google to be ranked first. If it is what the searcher wanted, it doesn't matter.
IMO, this is no different from a company purchasing a large ad in the yellow pages of the phone directory. Does/should anyone think that the ads are bigger for certain companies because those companies are better? People know that companies buy those ads. Searchers should also know that "sponsored" = paid. I don't see anything inappropriate. In fact, I credit Google for being above some of the slimy companies on the web and staying above the board with its business practices. Google's ability to charge for ranking will be nil if its search results reduce to the point of being roughly equivalent to random advertising.
Re:Google Cookies (Score:3, Insightful)
I think the reason this guy's being told to shut up is that everything he says sounds like propoganda. When he talks it reminds me of that "If you download MP3s, you're supporting COMMUNISM" ad I saw a while back.
Re:Google Cookies (Score:3, Interesting)
Here's the quote because people obviously read my post before reading the article:
Mr Anti-Google said that, not me.
Buried on the next page of the Salon article, Google responded with basically what you said:
Mr. Anti-Google probably mistook the history bar for a cookie heh.
The point I was making was that even if Google did store search terms in the cookie, so what? There's a big difference between storing that locally on your machine and storing it on their own servers. Even if this guy was absolutely 100% right about how it works, his point carries no weight.
Re:Google Cookies (Score:5, Insightful)
More than that, says Brandt, Google is a careless custodian of private information. When you search for something at Google, it saves your search terms and associates them with a cookie that is set to live on your machine for 36 years. Brandt fears that law enforcement officials could muscle Google into divulging all the terms you've ever searched for. Those terms could be "a window into your state of mind," and are therefore a clear violation of your privacy, he says.
Uh, Does Brandt even properly understand how Cookies work? If the Feds go to Google and say "Give us all the cookies you've stored on people's computers" Google is going to say "Uh.. see, that's the thing about storing them on other people's computers.. we don't store them here."
And as for Google recording every search term I've searched for, let's be realistic here, even if Google did have that kind of storage space available (every term, for every user, with a link between each?) why in the heck would they use it for that when they have the whole freakin' 'net to try and store?
Re:Google Cookies (Score:4, Interesting)
Re:Google Cookies (Score:3, Insightful)
I think that once you have a judge consent to a search, getting him/her to sign off on asking Google is a minor hassle.
Re:Google Cookies (Score:4, Insightful)
Google is not a government - it is a search site! They exist to make a profit. They will make money by providing a quality search result, thereby attracting users.
Google has become one of the most important gatekeepers on the net, and they literally can make or break businesses by playing with their database (I wonder if they have checks and balances to ensure that Google workers aren't doing favours, returned by some $, for people by tweaking their rankings). Your claims that they're just some business is about as valid as saying ICANN is just some business that can do what they want. Uh huh.
In any case, the Salon article was pathetic. As much as I might disagree with this guy's opinion that Google sucks because it doesn't rank him highly, there is no doubt that we need to be vigilant that the net isn't usurped by any one group or individual. The Salon article did a classic right winger technique of refuted everyone of this claims with some absurd parallel claim: It's hard to get too upset about search privacy at Google when, all over the Web, other sites are increasingly playing fast and loose with private data....Yahoo, which requires sign-in for portal services, has already announced a plan to e-mail ads to people based on what they've searched for. (The plan, called Yahoo Impulse Mail, is "opt-in.") If you wanted to be a watchdog for the privacy of search, wouldn't you start by attacking that program?... Uh huh. "Well, sorry that the police raped and beat the kids walking down the street...but in Afghanistan they behead them too! Go pay attention to them, there's nothing to see here! {YOINK} (running away)". It's a pathetic, and dangerous, technique of disqualifying a complaint.
And what's with the ridiculous Google-love on here? You'd think that every Slashdotter was a majority shareholder. Google is my search engine of choice, but when Doubleclick tracks what you do there's an outrage on Slashdot. When Google technically has the capability to pull up every search you've ever performed (errr "genital warts"), it's a non-issue? Uh huh.
Re:Google Cookies (Score:2)
It works, and you dont have to use cookies if you dont care to.
siri
I like google (Score:3, Funny)
Trying to get on Google (Score:2, Insightful)
Since you need to have links to your site from other sites to get rated highly in Google, it is almost impossible to get them, as people who may be interested in linking to your site won't find it on Google.
Vivious circle, anyone?
Goblin
Re:Trying to get on Google (Score:2)
Once in, decent content will mean you will eventully get links as people find you - even if you start on page 10 in the results to start with.
Re:Trying to get on Google (Score:3, Funny)
popularity to increase (Score:2)
although strangely enough, apparently so will links from subdomain sites like geocities, etc.
so now he merely has to complain about his monthly bandwidth allotment getting used up, and his serving crashing due to /.
He can't win
Re:popularity to increase (Score:2)
Says more about Salon than the guy (Score:2)
Fear not, they'll soon be gone...
It seems to me... (Score:4, Insightful)
-brian
Re:It seems to me... (Score:3, Redundant)
According to the article, his complaint is twofold: Google favors popular, established sites over young or unpopular sites. Also, he fears the cookie.
I am Slashdot's complete lack of interest in his problems.
I Disagree (Score:4, Interesting)
I think, to be quite blunt, that this is a crock of shit.
One of the most important things in a civil society are the checks and balances critcism offers on any service, any government, any individual, indeed, any endeavor undertaken. These checks and balances, and the importance of public criticism, because of vastly greater importance when the perceptions and lives of many people are impacted.
This is true whether one is criticizing GNU, Linux, Richard Stallman, our corporate masters in the form of George Bush, Enron, WorldComm, Microsoft, Apple, Sun Microsystems, Red Hat, or whomever else happens to be in the hotseat at any given time.
If Google really were stacking their search results, criticism and a 'holy crusade' as you so snidely put it, would be a very important counterbalance in offsetting the corruption and distortion inherent in such a thing, particularly given how trusted Google is.
I disagree with the guys criticism, for what it is worth, and am an ardent user of Google. But I agree whole heartedly with the need for such criticism to keep the likes of Google honest, and to call them on the carpet when they do something shady or wrong (like they did when the caved to the Cult of Scientology's pressure to censor the search results revealing critics of that particular organization).
This "if you don't have something nice to say, don't say anything at all" is a fine creed for slaves or submissive corporate drones, but it has no place at all in the marketplace of intellectual thought or debate.
Now, on the other hand, if you'd like to argue for civil discourse instead of flame fests and random insults, I will be the first to add my voice to yours, but lest we forget, civil discourse can and must include criticism, sometimes vehement criticism. Indeed, such can often be the most important civil discourse being conducted.
Brandt misses the point (Score:5, Insightful)
Google's PageRank system isn't supposed to be democratic. It's supposed to be effective. When he starts arguing that Google doesn't consistently return pages that meet the search criteria, then I'll listen to him.
Google PageRank is democratic: links = votes (Score:5, Informative)
Re:Brandt misses the point (Score:2)
Amen!
In fact, any search engine that displays results relevant to your search terms won't be "democratic", because you'll get (wait for it)
I mean duh.
If I search for VPN configuration info or the GNP of Luxemborg in 1997 (I assume neither of which are on his site), and his site doesn't come up, is he going to complain about that too?
This page contains no Data (Score:3, Funny)
Not any more he doesn't...
Read the whole story... (Score:2, Flamebait)
Dumb dumb dumb dumb... but hell, what do you expect from something that ran in Salon anyway?
link popularity (Score:2, Insightful)
That's probably true from the standpoint that link popularity may not offer the most relevant information for what you might be looking for, but on the average, I'd say it's by far the best way to go. The main problem I see with that type of determination is that by having a link at the top of your search results, its likely that the site will continue to stay more popular than it might deserve. As a result, the system becomes somewhat self-defeating.
I think I know why his site doesn't get Googled (Score:4, Interesting)
<meta name="Googlebot" content="noarchive">
I can't help but think that this doesn't help his Google rankings.
Re:I think I know why his site doesn't get Googled (Score:4, Funny)
Re:I think I know why his site doesn't get Googled (Score:5, Informative)
Re:I think I know why his site doesn't get Googled (Score:4, Funny)
Actually, judging from how quickly his site buckled under this /.'ing, the good folks at Google are doing him a favor by not increasing traffic to his site.
Boo Hoo (Score:3, Redundant)
To his agenda perhaps.
However Google isn't used by most folks as a directory - it's a search engine. It simply pulls up entries according to a formula (see pigeonrank [slashdot.org] for the inside scoop) and gives those back. No bias beyond what smart webmasters can impart, no artificial clustering, etc.
If Google were to start doing as Brandt wants it would quickly run into endless battles, loose it's searching edge, become just another pay(or agenda)-for-play roadkill.
No thanks.
Re:Boo Hoo (Score:3, Interesting)
I agree, and I wish Mr. Brandt would suggest a workable alternative, rather than whining. He clearly has a monkey on his back.
However, I do wonder about the efficacy of google's formula. My concern is that google's popularity turn it's page rankings into self-fulfilling prophesies. It's a positive feedback loop: a site w/ a high google rank gets more views and more links, which increases its google rank, ad infinitum.
Like you say, I'd rather not have search engines be driven by agendas or money. But I believe anything can be improved upon. Personally, I believe perhaps a bit of randomness might help. Instead of recieving an absolute page rank, pages should recieve a probability of being listed higher or lower. Just a thought.
Re:Boo Hoo (Score:3, Interesting)
As opposed to summaries that typically look like:
inside
namebase can't be THAT popular (Score:2)
For crying out loud, my PERSONAL web site can handle more traffic than that.
What's he hosting it on, a dialup?
PageRank discriminates? (Score:2, Interesting)
Slashdotted... or something more siniester... (Score:3, Funny)
Or have the Google gods turned their clusters towards more sinister deeds, silencing their critics.
We may never know.
Boo Hoo (corrected) (Score:4, Insightful)
To his agenda perhaps.
However Google isn't used by most folks as a directory - it's a search engine. It simply pulls up entries according to a formula (see pigeonrank [google.com] for the inside scoop) and gives those back. No bias beyond what smart webmasters can impart, no artificial clustering, etc.
If Google were to start doing as Brandt wants it would quickly run into endless battles, loose it's searching edge, become just another pay(or agenda)-for-play roadkill.
No thanks.
unwarranted criticism (Score:3, Insightful)
Google is like a popularity contest (Score:2)
PageRank works. If your page is linked to by a large number of well trafficked sites, then you get ranked higher. If your some crack pot whose site no one cares about, you don't get a high rank...
In other words, Brandt recognizes that there has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site critical of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorable Donald Rumsfeld."
Don't blame Google for equating accuracy and usefulness with popularity. It's either that or resort to subjective measures.
Sour grapes (Score:5, Insightful)
Okay, so I did a search on Nixon on Brandt's site. Here are the first couple of results:
(1) How the Vatican conspired to hide Nazi war criminals.
(2) How various activists were persecuted by the CIA and FBI.
Nowhere did I even SEE Nixon's name in these abstracts. The only relevance is that Nixon was alive at the time, or maybe president when some of them took place, but hardly the man personally responsible for all of them.
When I type "Nixon" into Google, I expect to see biographical material, both good and bad, not totally unrelated rantings. Google is doing its job, in my opinion. It is giving low rankings to Brandt's irrelevant materials. His complaints are pure self-centered sour grapes.
Let me get this straight (Score:5, Insightful)
Google must be doing pretty well if this is the worst criticism they can find about them.
His view, capsule summary (Score:2)
I've never seen... (Score:2)
This chap seems to be little more than someone who is holding a grudge against google because his website isn't as high on the list as he wants.
Well Tough @#$%, life sucks doesn't it.
What this guy needs to learn that what helps out with your score on Google isn't just the content, but how many people link to your site for that information. Thus having a page on Rumsfeld isn't as helpful as being a webpage on Rumsfeld that 50 sites refer to you.
If this guy wants a higher ranking then he has to make relationships with other websites to get his rankings up. It's not that hard as most webmasters know this and a link sharing helps them as much as it would him.
He's just a whiny person who happened to catch the attention of some person who needed to fill out todays news space on Salon.
Ignore him and hopefully he'll go away.
Phoenix
Democracy? (Score:2)
There's one in every crowd.
Namebase: pretty handy (Score:2, Interesting)
Invasion Of Privacy? (Score:5, Insightful)
Maybe I'm missing something here, but how is this a violation of your privacy? I mean, the whole thing is that you are using their service for free and willfully sending them the data that you choose. Everyone gets to choose what they search for in a search engine. This isn't private information in any real way. Google is providing you the free service of looking up words that you have intentionally provided. You don't like them being associated with a cookie? Refuse the damn cookie! Really paranoid? Go wander the web on your own without a search engine!
At what point were you guaranteed the free and anonymous use of a search engine? You're not being forced to use it. The world doesn't discriminate against people who do not choose not to efficiently search the web.
People like this are blurring the privacy issue and focusing attention away from legitimate privacy issues.
slashdotted already. sigh (Score:4, Insightful)
hope you like your servers toasty, bud.
Moderation of hits? (Score:3, Interesting)
If letting Google rank the pages is undemocratic, what about a system in which, when you go to a page from a Google search, Google adds a frame at the top of your page that let's you vote on how useful this page was on a scale of 1-10?
Then, the most popular hits for a given set of search words would have their Google ranking rise. Now that's democracy.
Re:Moderation of hits? (Score:2, Interesting)
Are you serious?
Do you think someone might think to abuse this system? Automated form filling, anyone? Even if that were prevented, it wouldn't be too hard or expensive to hire hundreds of low paid data entry people to vote a site up.
The google alogorithm can be manipulated to some extent but it has stood up pretty well so far. A voting system could be manipulated much more easily.
Whine, whine, whine (Score:5, Insightful)
What this guy wants, by abolishing PageRank, is a return to the free-for-all of early search engines, where the loudest voice rules. If one page has more keywords, it's ranked higher -- whether or not those keywords appear in the context of relevant content.
Here's his real problem: he thinks that linking to "Donald Rumsfeld" should bring his site's page to the top, despite the fact that he has no actual content -- just a list of links to other pages with content.
He calls this a failing of PageRank. I call it whining. If he wants more links from Google, he should get the word out about his site (preferably without manipulating Salon.com into doing it for him) and add some actual information about the people he's archiving by hand, instead of just building a big hotlist about them.
Basically, he wants to be the tyrant he imagines Google to be. Well, let him want all he likes. Google's popular because it's good and it's relevant; the fact that a tiny tiny minority think it's not isn't a good reason to overthrow the whole system.
People who live in glass houses shouldn't throw stones. He should start by making changes on his own site, not insisting Google make changes on there.
In other news (Score:5, Funny)
The problem with this is.. (Score:3, Interesting)
Also, this guy claims that Google keeps a record of what everyone searches for.. what proof does he have of this? That Google sends a cookie? That cookie is more likely than anything just used for tracking how often most people use the site, so they can create aggregate numbers of unique users, etc. Sure they could be tracking every search term, but why would they, think how much storage space that'd waste for no return. If the FBI ever wants to find out what this guy searches for, they'll just contact his ISP and have him monitored that way.
Cranky, but not entirely off... (Score:4, Funny)
In the meantime, anyone who would like to cover their tracks can use my cookie:
.google.com TRUE / FALSE 2147368045 PREF ID=111439b95052c72a:TM=1030056425:LM=1030056425:S= v7T9QSFKEkI
;)
Of course, if it turns out that Google is planning to give a prize to the most active user, or they have some kind of search engine green stamps, you're screwed.
Re:Cranky, but not entirely off... (Score:3, Funny)
You're right, this is terrifying!
I can see the google conversations now:
Employee: Sir, our servers have indicated person #111439b95052c72a has a really interesting search pattern! I think we should send this info to the FBI for investigation!
Boss: Good work! Tell me his name and address, and we'll send that info over to the feds right away.
Employee: Ummm, name and address? How about a few dozen IP addresses from AOL's proxy servers instead?
Boss: Doh.
His real problem... (Score:2, Interesting)
So it seems that this guy's real problem isn't with how Google ranks his site, but rather that Google isn't pushing his product to every searcher who hits their site. So he talks about the "undemocracy" of Google, but when it comes down to it, his main issue is that Google isn't helping his business, or rather, that Google's ranking algorithm isn't compatible with his business plan.
Too often, when people say something is undemocratic, it's just because they aren't getting there own way.
link to article, a quote, and my response (Score:5, Interesting)
http://www.salon.com/tech/feature/2002/
(might be a space inserted in the URL by the browser submission, apologies)
Second, a quote from the article:
"Brandt sees this as Google's major flaw. "I'm not saying there aren't some sites that are more important that others, bu t in Google the sites that do well are the spammy sites, sites which have Google psyched out, and a lot of big sites, corporate headquarters' sites -- they show up before sites that criticize those companies.
In other words, Brandt recognizes that ther e has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site crit i cal of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorabl e Donald Rumsfeld."
I must disagree with the ideal expressed here as Mr. Bran dt's. If I was searching for material on the Web about Donald Rumsfeld, I would rarely search for information critical of him *first*. If I was ego surfing on myself, I'd want to see my own material about me returned by Google, ahead of negative reviews and sites. I don't think that's an unfair way for Google to operate. While some of the issues Mr. Brandt raises might be valid, I do not feel that Google is required to promote or support Mr. Brandt's agenda over the agenda of the people and organizations Mr. Brandt chooses to focus on. M
k00k? (Score:2, Interesting)
Is it just me, or does this guy sound like yet another internet kook? Get "untied.com" ranked first when searching for "united airlines"? That makes no sense.
Google is a system -- a system that works a certain way. His complaints about PageRank are like complaining about an automobile for the way its wheels go 'round and 'round.
I'm surprised salon dedicated any article space to this.
Google can be hijacked (Score:2, Interesting)
Self defeating point... (Score:4, Insightful)
"Having a thousand links from sites that are performing poorly does no good!"
Here's how I interpret it:
"Only sites that have really marketed themselves are viable. Pay me to make your site more popular."
Anybody get an impression that he was saying something different? I don't think the guy understands that his comment makes no sense if Google's popularity's based on people finding what they're looking for. It seems more to me he's just mad that he can't buy search rankings.
Namebase vs. Google (Score:4, Interesting)
NameBase...is...designed...to find books and newspaper articles that mention a specific person. For example, if you're trying to find out who this character "Donald Rumsfeld" you keep hearing about on the news is, you'd type it into NameBase and find about 50 books and articles that mention the man.
No, I'd type it into Google, where I'd find about 130,000 pages that mention him, and the top hit is
Biography of Donald H. Rumsfeld
THE HONORABLE DONALD RUMSFELD. Secretary of Defense. Photo: Secretary
Donald H. Rumsfeld. Until being sworn in as the 21st Secretary
www.defenselink.mil/bios/rumsfeld.html - 11k - Cached - Similar pages
Well, what do you know, he's Secretary of Defense.
The really relevant part of the Salon article (Score:4, Insightful)
So Mr. Brandt wants his internal pages to be the most important page about the person rather than, say, the person's own home page. Personally, I take leave to doubt that Mr. Brandt's opinion of a person is the ultimate reference in most cases (if any).
To me, it's the old, common story of, "You're not doing what I want. You're mean and unfair!" One expects this in the kindergarden sand box, but not in adults. Alas, the latter expectation is all too often unmet.
World is full of whineboxes.
Looking at the results (Score:2)
Google is undemocratic ... (Score:2)
Searches before Google was operational... (Score:2, Funny)
Result: Teen Catholic barely-legal sluts from Holland!
Search: Mali Timbuktu empire
Result: Malian Cum-Slurping Sluts! Timbuktu Kama Sutra Style Mature Singles Waiting For You!
Thank goodness Google is here, even if it's not 100% perfect.
He want's google to be non-objective (Score:4, Insightful)
In other words, Brandt recognizes that there has to be some order to Google's results, and that some sites might deserve to come up before others. He just disagrees with the way Google does it. In Brandt's ideal world, if you searched for "United Airlines," you would see untied.com -- a site critical of United -- before you see United's page. And if you searched for Rumsfeld, you'd see NameBase's dossier on him before the Defense Department's site on the "The Honorable Donald Rumsfeld."
He wants google to be a political action site that favors his views. He's a whiny little baby.
Sites that critisize corporations should appear before the corporations main site? Why? Did you search for the company or for criticism? If the company/group in question was something he agreed with, perhaps some environmental organization or the democratic national commitee, would he want criticism of them to come up first too?
A quick stop at google shows that if you search for "United Airlines" you get their site first, and the site he thinks should be first shortly thereafter. If you search for "United Airlines criticism" you get the site he reccommends first. Looks like google is doing it's job correctly to me.
Why is salon publishing the crap?
I see a great need... (Score:5, Insightful)
Cache of site (Score:3, Informative)
I know the guy will be gutted about that
Who remembers Altavista? (Score:3, Insightful)
Brandt's complaint appears to be that he has a database of citations, but when you search for Donald Rumsfeld his site is more than 10 pages down, where nobody ever looks. And that's fine with me. That's what I expect from Google. He obviously expects something else (like united.com appearing higher than United Airlines real site), and being the kind of person he apparently is, he expects Google to change to become how he expects them to be, rather than realigning his expectations with reality.
Undemocratic? Who cares? (Score:3, Insightful)
Google's page rank isn't democratic, and thank God for that. Otherwise I'd have to wade through a bunch of crap that I generally don't want to wade through.
Different search engines are better at searching for different things, but Google is my first choice almost every time. It is, by far, the most effective search engine I've seen. If it wasn't, I don't think it would be the most popular.
Someone explain to me why anyone pays attention to this guy.
Here's my essay (Score:4, Interesting)
A few points missed in the Salon piece:
I specifically pointed out to the author of the piece when he interviewed me, that I felt my site did okay in Google, and that I was speaking for the public interest. The so-called "royal we" that Mr. Manjoo, the interviewer and author, refers to sarcastically, is used because I'm speaking for a tax-exempt, nonprofit public charity, Public Information Research, Inc. We do not sell widgets. Some of the comments in Slashdot have me mixed up with another person who is selling ads based on PageRank. But then, who expects Slashdotters to actually read the article?
My main site in Google is www.pir.org and it has a PageRank of 7. The www.namebase.org, with a PR of 6, is a streamlined CGI version of the main site, without all the essays and cartoons. NameBase began in the early 1980s and has been on the Internet since early 1995.
The other problem I have with the author's spin is that a good half of the interview was about Google's cookie. Most of the work I put into www.google-watch.org has to do with the cookie. In the article, the cookie is briefly mentioned, and most of the article is about how selfish and silly I am to think that Google should rank me higher.
My complaint about Google is not that PIR got the short end of the stick from Google, but that Google's stick should be longer.
My essay about PageRank is below.
_____________________
PageRank: Google's Original Sin
by Daniel Brandt
By 1998, the dot-com gold rush was in full swing. Web search engines had been around since 1995, and had been immediately touted by high-tech pundits (and Forbes magazine) as one more element in the magical mix that would make us all rich. Such innovations meant nothing less than the end of the business cycle.
But the truth of the matter, as these same pundits conceded after the crash, was that the false promise of easy riches put bottom-line pressures on companies that should have known better. One of the most successful of the earliest search engines was AltaVista, then owned by Digital Equipment Corporation. By 1998 it began to lose its way. All the pundits were talking "portals," so AltaVista tried to become a portal, and forgot to work on improving their search ranking algorithms.
Even by 1998, it was clear that too many results were being returned by the average search engine for the one or two keywords that were entered by the searcher. AltaVista offered numerous ways to zero in on specific combinations of keywords, but paid much less attention to the "ranking" problem. Ranking, or the ordering of returned results according to some criteria, was where the action should have been. Users don't want to figure out Boolean logic, and they will not be looking at more than the first twenty matches out of the thousands that might be produced by a search engine. What really matters is how useful the first page of results appears on search engine A, as opposed to the results produced by the same terms entered into engine B. AltaVista was too busy trying to be a portal to notice that this was important.
Enter Google
By early 1998, Stanford University grad students Larry Page and Sergey Brin had been playing around with a particular ranking algorithm. They presented a paper titled "The Anatomy of a Large-Scale Hypertextual Web Search Engine" at a World Wide Web conference. With Stanford as the assignee and Larry Page as the inventor, a patent was filed on January 9, 1998. By the time it was finally granted on September 4, 2001 (Patent No. 6,285,999), the algorithm was known as "PageRank," and Google was handling 150 million search queries per day. AltaVista continued to fade; even two changes of ownership didn't make a difference.
Google hyped PageRank, because it was a convenient buzzword that satisfied those who wondered why Google's engine did, in fact, provide better results. Even today, Google is proud of their advantage. The hype approaches the point where bloggers sometimes have to specify what they mean by "PR" -- do they mean PageRank, the algorithm, or do they mean the Public Relations that Google does so well:
PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."
Google goes on to admit that other variables are also used, in addition to PageRank, in determining the relevance of a page. While the broad outlines of these additional variables are easily discerned by webmasters who study how to improve the ranking of their websites, the actual details of all algorithms are considered trade secrets by Google, Inc. It's in Google's interest to make it as difficult as possible for webmasters to cheat on their rankings.
It's all in the ranking
Beyond any doubt, search engines have become increasingly important on the web. E-commerce is very attuned to the ranking issue, because higher ranking translates directly into more sales. Various methods have been designed by various engines to monetize the ranking situation, such as paid placement, pay per click, and pay for inclusion. On June 27, 2002, the U.S. Federal Trade Commission issued guidelines that recommended that any ranking results influenced by payment, rather than by impartial and objective relevance criteria, ought to be clearly labeled as such in the interests of consumer protection. It appears, then, that any algorithm such as PageRank, that can reasonably pretend to be objective, will remain an important aspect of web searching for the foreseeable future.
Not only have engines improved their ranking methods, but the web has grown so huge that most surfers use search engines several times a day. All portals have built-in search functions, and most of them have to rely on one of a handful of established search engines to provide results. That's because only a few engines have the capacity to "crawl" or "spider" more than two billion web pages frequently enough to keep their database current. Google is perhaps the only engine that is known for consistent, predictable crawling, and that's only been true for less than two years. It takes almost a week to cover the available web, and another week to calculate PageRank for every page. Google's main update cycle is about 28 days, which is a bit too slow for news-hungry surfers. In August, 2001 they also began a second "mini-crawl" for news sites, which are now checked every day. Results from each crawl are mingled together, giving the searcher an impression of freshness.
For the average webmaster, the mechanics of running a successful site have changed dramatically from 1996 to 2002. This is due almost entirely to the increased importance of search engines. Even though much of the dot-com hype collapsed in 2000 and 2001 (a welcome relief to noncommercial webmasters who remembered the pre-hype days), the fact remains that by now, search engines are the fundamental consideration for almost every aspect of web design and linking. It's close to a wag-the-dog situation. That's why the algorithms that search engines consider to be consistent with the FTC's idea of impartial and objective ranking criteria deserve closer scrutiny.
What objective criteria are available?
Ranking criteria fall into three broad categories. The first is link popularity, which is used by a number of search engines to some extent. Google's PageRank is the original form of "link pop," and remains its purest expression. The next category is on-page characteristics. These include font size, title, headings, anchor text, word frequency, word proximity, file name, directory name, and domain name. The last is content analysis. This generally takes the form of on-the-fly clustering of produced results into two or more categories, which allows the searcher to "drill down" into the data in a more specific manner. Each method has its place. Search engines use some combination of the first two, or they use on-page characteristics alone, or perhaps even all three methods.
Content analysis is very difficult, but also very enticing. When it works, it allows for the sort of graphical visualization of results that can give a search engine an overnight reputation for innovation and excellence. But many times it doesn't work well, because computers are not very good at natural language processing. They cannot understand the nuances within a large stack of prose from disparate sources. Also, most top engines work with dozens of languages, which makes content analysis more difficult, since each language has its own nuances. There are several search engines that have made interesting advances in content analysis and even visualization, but Google is not one of them. The most promising aspect of content analysis is that it can be used in conjunction with link pop, to rank sites within their own areas of specialization. This provides an extra dimension that addresses some of the problems of pure link popularity.
Link popularity, which is "PageRank" to Google, is by far the most significant portion of Google's ranking cocktail. While in some cases the on-page characteristics of one page can trump the superior PageRank of a competing page, it's much more common for a low PageRank to completely bury a page that has perfect on-page relevance by every conceivable measure. To put it another way, it's frequently the case that a page with both search terms in the title, and in a heading, and in numerous internal anchors, will get buried in the rankings because the sponsoring site isn't sufficiently popular, and is unable to pass sufficient PageRank to this otherwise perfectly relevant page. In December 2000, Google came out with a downloadable toolbar attachment that made it possible to see the relative PageRank of any page on the web. Even the dumbed-down resolution of this toolbar, in conjunction with studying the ranking of a page against its competition, allows for considerable insight into the role of PageRank.
Moreover, PageRank drives Google's monthly crawl, such that sites with higher PageRank get crawled earlier, faster, and deeper than sites with low PageRank. For a large site with an average-to-low PageRank, this is a major obstacle. If your pages don't get crawled, they won't get indexed. If they don't get indexed in Google, people won't know about them. If people don't know about them, then there's no point in maintaining a website. Google starts over again on every site for every 28-day cycle, so the missing pages stand an excellent chance of getting missed on the next cycle also. In short, PageRank is the soul and essence of Google, on both the all-important crawl and the all-important rankings. By 2002 Google was universally recognized as the world's most popular search engine.
How does PageRank measure up?
In the first place, Google's claim that "PageRank relies on the uniquely democratic nature of the web" must be seen for what it is, which is pure hype. In a democracy, every person has one vote. In PageRank, rich people get more votes than poor people, or, in web terms, pages with higher PageRank have their votes weighted more than the votes from lower pages. As Google explains, "Votes cast by pages that are themselves 'important' weigh more heavily and help to make other pages 'important.'" In other words, the rich get richer, and the poor hardly count at all. This is not "uniquely democratic," but rather it's uniquely tyrannical. It's corporate America's dream machine, a search engine where big business can crush the little guy. This alone makes PageRank more closely related to the "pay for placement" schemes frowned on by the Federal Trade Commission, than it is related to those "impartial and objective ranking criteria" that the FTC exempts from labeling.
Secondly, only big guys can have big databases. If your site has an average PageRank, don't even bother making your database available to Google's crawlers, because they most likely won't crawl all of it. This is important for any site that has more than a few thousand pages, and a home page of about five or less on the toolbar's crude scale.
Thirdly, in order for Google to access the links to crawl a deep site of thousands of pages, a hierarchical system of doorway pages is needed so that crawler can start at the top and work its way down. A single site with thousands of pages typically has all external links coming into the home page, and few or none coming into deep pages. The home page PageRank therefore gets distributed to the deep pages by virtue of the hierarchical internal linking structure. But by the time the crawler gets to the real "meat" at the bottom of the tree, these pages frequently end up with a PageRank of zero. This zero is devastating for the ranking of that page, even assuming that Google's crawler gets to it, and it ends up in the index, and it has excellent on-page characteristics. The bottom line is that only big, popular sites can put their databases on the web and expect Google to cover their data adequately. And that's true even for websites that had their data on the web long before Google started up in 1999.
What about non-database sites?
There are other areas where PageRank has a negative effect, even for sites without a lot of data. The nature of PageRank is so discriminatory, that it's rather like the exact opposite of affirmative action. While many see affirmative action as reverse discrimination, no one would claim (apart from economists who advocate more tax cuts for the rich) that the opposite, which would be deliberate discrimination in favor of the already-privileged, is a solution for anything. Yet this is essentially what Google claims.
Those who launch new websites in 2002 have a much more difficult time getting traffic to their sites than they did before Google became dominant. The first step for a new site is to get listed in the Open Directory Project. This is used by Google to seed the crawl every month. But even after a year of trying to coax links to your new site from other established sites, the new webmaster can expect fewer than 30 visitors per day. Sites with a respectable PageRank, on the other hand, get tens of thousands of visitors per day. That's the scale of things on the web -- a scale that is best expressed by the fact that Google's zero-to-ten toolbar is a logarithmic scale, perhaps with a base of six. To go from an old PageRank of four to a new rank of five requires several times more incoming links. This is not easy to achieve. The cure for cancer might already be on the web somewhere, but if it's on a new site, you won't find it.
PageRank also encourages webmasters to change their linking patterns. On search engine optimization forums, webmasters even discuss charging for little ads with links, according to the PageRank they've achieved for their site. This would benefit those sites with a lower PageRank that pay for such ads. Sometimes these PageRank achievements are the result of link farms or other shady practices, which Google tries to detect and then penalizes with a PageRank of zero. At other times professional optimizers get away with spammy techniques. Mirror sites and duplicate pages on other domains are now forbidden by Google and swiftly punished, even when there are good reasons for maintaining such sites. Overall, linking patterns have changed significantly because of Google. Many webmasters are stingy about giving out links (which can dilute your transference of PageRank to a given site), at the same time that they're desperate for more links from others.
What should Google do?
We feel that PageRank has run its course. Google doesn't have to abandon it entirely, but they should de-emphasize it. The first step is to stop reporting PageRank on the toolbar. This would mute the awareness of PageRank among optimizers and webmasters, and remove some of the bizarre effects that such awareness has engendered. The next step would be to replace all mention of PageRank in their own public relations documentation, in favor of general phrases about how link popularity is one factor among many in their ranking algorithms. And Google should adjust the balance between their various algorithms so that excellent on-page characteristics are not completely cancelled by low link popularity.
PageRank must be streamlined so that the "tyranny of the rich" characteristics are scaled down in favor of a more egalitarian approach to link popularity. This would greatly simplify the complex and recursive calculations that are now required to rank two billion web pages, which must be very expensive for Google. The crawl must not be PageRank driven. There should be a way for Google to arrange the crawl so that if a site cannot be fully covered in one cycle, Google's crawlers can pick up where they left off on the next cycle.
Google is so important to the web these days, that it probably ought to be a public utility. Regulatory interest from agencies such as the FTC is entirely appropriate, but we feel that the FTC addressed only the most blatant abuses among search engines. Google, which only recently began using sponsored links and ad boxes, was not even an object of concern to the Ralph Nader group, Commercial Alert, that complained to the FTC.
This was a mistake, because Commercial Alert failed to look closely enough at PageRank. Some aspects of PageRank, as presently implemented by Google, are nearly as pernicious as pay for placement. There is no question that the FTC should regulate advertising agencies that parade as search engines, in the interests of protecting consumers. Google is still a search engine, but not by much. They can remain a search engine only by fixing PageRank's worst features.
*
[Daniel Brandt is founder and president of Public Information Research, Inc., a tax-exempt public charity that sponsors NameBase. He began compiling NameBase in 1982, from material that he started collecting in 1974, and is now the programmer and webmaster for PIR's several sites. He participates in various forums where webmasters share observations about the often-secretive algorithms, bugs, and behavior of various search engines. Brandt has been watching Google's interaction with NameBase ever since Google, in October, 2000, became the first search engine to go "deep" on PIR's main site by crawling thousands of dynamic pages.]
Re:Here's my essay - here's my comment. (Score:4, Insightful)
Google became what it is because it saw an unfilled niche, and filled it. They "built a better mousetrap", and the world did indeed beat a path to their door. There is nothing stopping you from doing the same. If you're half as smart as you seem to think you are, you should have no problem implementing a search engine, and becoming as successful as Google is now.
Google is NOT a public utility, nor is it any form of monopoly. It needs to be regulated just as much as YOUR site does.
Unlike so many other companies, Google got where it is today solely on the merits of it's technology. It didn't succeed by pumping millions of dollars into marketing, it didn't succeed by using underhanded business tactics to squash its' competitors. All it did was make the best product.
Contrary to your essay, I (and I think many
NameBase sucks (Score:5, Insightful)
-
It's a search engine. You find info by typing names into a form. There are no obvious links to the content. How's that supposed to get spidered?
-
His search engine is overloaded right now and just returns error messages. Maybe that's what Google sees.
-
The good data is by subscription only:
"And ask your library or student government to subscribe to NameBase ($200 for two years of unrestricted access from any campus computer) so that we can continue to add names, and you can continue to find them."
-
<meta NAME="GOOGLEBOT" CONTENT="NOARCHIVE"> can't be helping.
-
This guy is very picky about who gets to spider him. Here's his "robots.txt" file:
-
He uses one-pixel GIFs to trap spiders. [namebase.org] He also uses cookies and web bugs, providing a long-winded explanation of why what he does is OK, but what Google does is evil.
In conclusion, this guy created his own problem.I run three web sites. Each is at the top of the Google rankings for its obvious keywords, and I've done nothing whatsoever to make that happen. I just have useful content that people like.
Re:Cookie? What cookie? (Score:2, Interesting)
I think the guys claims are probably bullshit, or at least gross exagerations. Google do seem to be one of the good guys, so it's inevitable that someone would come along and try and dig up stuff, real or imagined.
Makes you wonder who might be bankrolling the guy...
Cookies? (Score:3, Informative)
In Mozilla -> Tools -> Cookie Manager -> Block Cookies from this site...
Pointless complaints (cookies, article) (Score:5, Insightful)
I never understand why people make such a big deal about cookies. If you don't want to be tracked (like me, like most of us here at slashdot) there are countless ways of protecting yourself via browser settings, CookieCop, Proximotron, etc. Anyone who really cares about privacy probably already knows how to disable cookies. And anyone who doesn't know probably doesn't care about privacy (my grandmother, etc.). It seems like people just enjoy complaining about a standard web technique even though it is easily circumventable.
Second, why the hell is slashdot even posting this article? I've skimmed plenty of the below comments and they all seem to agree that this anti-google guy is a goofball. Just because Salon ran an article on him doesn't mean that this fruitcake's complaints have any merit. Considering how many stories get rejected from slashdot on a daily basis, why was this chosen? Is it just me, or did anyone enjoy/learn from that article?
GMD
Re:Pointless complaints (cookies, article) (Score:5, Insightful)
Re:Pointless complaints (cookies, article) (Score:4, Informative)
sheesh, whats next, perfect grammer and spelling?
GMD
Re:Sad news ... Stephen King dead at 54 (Score:2, Offtopic)
But wait a minute, didn't he die last week...and the week before...and the week before that...and the week before that...etc, etc.
This is probally gonna cost me some karma but screw it. I have to ask the question.
Why do you exist? Have you nothing better to do than to post the same bogus piece of news over and over again. Hardly an article on
Goddess preserve me from these random pieces of wasted, self-replicating genetic material
Re:Sad news ... Stephen King dead at 54 (Score:3, Funny)
Where do these people come from? Is there an agency out there that reads the Net and says "Oops, not enough morons on this group," and then assigns some slack jawed, inbred, grit-eatin' stooge to gum up the works?
- Jim Cowling
Re:Google, and WAP? (Score:4, Informative)
Short answer: It's a bug/quirk/feature of IE that, somehow, the page came across screwed up and got cached that way and, despite anything and everything you may have told IE about "check for a new page every time I visit...", it still checks this screwed up cache version first. The solution is to delete your temporary Internet files (Tools->Internet Options->"Delete Files" in "Temporary Internet Files")
Long answer:I had this problem with a site [bemoedee.com] I frequent quite a bit. Since I know the author personally I told her about it. When I would actually save and view the page as prompted I would see all the HTML like I was supposed to but tons and tons of gibberish right before it. I told her to republish her blog but that didn't do it. No one else on her forum was having these problems and I figured since I told IE to check for a new version of the page every time that that couldn't be it. However, after clearing my cache out that did it.
Slightly More Elegant Solution: Instead of setting your homepage to Google, get the Google Toolbar [google.com]. This way you can set your homepage to whatever and use the Toolbar to do whatever Google searching you want. With all the options its got it's easily the most useful thing I've ever used. Be sure to check the experimental options [google.com] as well.