Putting Google to the Test

Big Nothing writes "Google has built its reputation on being the fastest and most accurate way to find information. But is the internet really the quickest way to access facts - and get them right? The Guardian puts Google to the test against more old-fashioned methods."

Re:Time to get to the Library?

[mopslik]

...and even then, some of their numbers are questionable themselves:

Question 1: List the titles of all the books written by Piers Morgan, editor of the Daily Mirror - Library Stephen Moss, 20sec (1st)

So you're saying that once I'm at the library, it takes me 20 seconds to look up the call number/location of Who's Who, turn to the appropriate page, and list out all of the man's books? Right. More than likely, this is an example of "you are in the library, with the book in hand, opened directly to the page you want."

Re:I wonder...

[Spansh]

Had you even bothered to read the article (Duh, this is slashdot, what am I thinking).

You'd notice this was the whole test.

We asked various "pub quiz" type questions and then comapres the speed of response of various methods of finding the answers, such as telephone, library and of course google.

Then again I wouldn't have needed to write this post if you'd bothered to read the article.

Re:Google Answers

[Peter Cooper]

I'm a Google Researcher. Generally most Researchers use the Internet or their own knowledge as their primary resource when answering questions. This is because most questions have a low fee and can be given answers which are merely pointers to further resources. Researchers are also encouraged to give the methods they used in their answer, to help the person who asked the question.

However, there have been quite a number of extraordinary cases where people have been so interested in answering the question that they've made phone calls, chased people, and dug out answers to incredibly complex questions. In one case, a Researcher managed to track down someone's obscure pre-20th C. German heritage.

Do remember that Google Answers is primarily for people who don't have excellent research skills of their own. While easy to use, finding certain things with Google (and other engines) requires skill, time, intelligence, and abstract thinking faculties that many people lack. Also bear in mind that most Google Researchers don't do it for the money. You will inevitably get a far higher quality (and longer) answer than you could possibly expect for the money. This is why tips are given to Researchers so often on the system.

The third question

[Peter Cooper]

Who is the vice chairman of the all-party parliamentary group on back care?

Google came last in their test, with a time of 6 minutes and 27 seconds. I decided to recreate their test (before knowing what the answer was). I entered.. "vice chairman" "parliamentary group" "back care"

First response, scrolled down a few pages till I saw 'back care' highlighted.. found the name, Janet Dean. Less than a minute! These people are not very good at their Google ;-)

Google is not some magic research machine. The person is the magic research machine, who uses Google as a tool. Just like "Do It Yourselfers" at home use the same hammers and saws that carpenters do.. but make a crappier job of it.

Re:Is google really that accurate?

[JimDabell]

I'd have to say that google deliberately alters certain rankings.

They have done in the past for legal reasons. They do not do so for editorial reasons. For instance:

Our search results are generated completely objectively and are independent of the beliefs and preferences of those who work at Google. Some people concerned about this issue have created online petitions to encourage us to remove particular links or otherwise adjust search results. Because of our objective and automated ranking system, Google cannot be influenced by these petitions. The only sites we omit are those we are legally compelled to remove or those maliciously attempting to manipulate our results.

(From google.com/explanation [google.com]).

By almost any measure of page rank google would have to be listed first, but it is not. This means that google deliberately lowered their rankings.

No, that is just one possibility. A far likelier reason would be that you don't know the Google ranking algorithm and so haven't taken into account important details.

Re:I wonder...

[Spansh]

Since it was a question (and remember we are newspaper people, this article actually came from the main paper) about a major british newspaper person, he would have known it would probably be in this years who's who, and since the person also knows his library well, then he would know where that was located (I wasn't there, yes he may have had it in front of him).

It's then a mere matter of locating the section on Piers Morgan.

Yes I'm actually a little slightly sceptical about the 20 seconds too, but then again I work in the development department and I'd have used google anyway.

Janet Dean is Vice Chair of Back Care group

[Lew Pitcher]

Question 3: Who is the vice chairman of the all-party parliamentary group on back care?

Google search criteria: 'UK +"vice chair" +parliament + "back care"'

Results 1 - 9 of about 10 for UK + "vice chair" +parliament +"back care". (0.24 seconds)

First page presented was http://www.parliament.the-stationery-office.co.uk/ pa/cm/cmparty/memi135.htm which takes you directly to the Back Care Group, where we find that Janet Dean (Labour) is listed as Vice Chair.

Perhaps the testers don't know how to use Google?

Re:god google

[los furtive]

Google knows nothing, except where words are placed.

Wrong! Google knows more than you think [google.ca].

For those too lazy to follow the link, type something like 4*5 in google and it will give you the result, or type 100 miles and it will show you how many kilometres that is.

It still is faster... if you know how to use it.

[Black Perl]

Obviously, the researcher was not an experienced googler. I saw one question that they claim took 6 minutes 27 seconds:

Question 3: Who is the vice chairman of the all-party parliamentary group on back care?

A google search for:

"vice chairman" all-party parliamentary group "back care"
resulted in *exactly one* hit, a pdf document listing all parliamentary groups. A click on View As HTML, a find on "back care" and Voila, the answer took about 30 seconds to get.

An experienced googler can find things faster than they did. This particular case was just a matter of knowing the difference between words and phrases and putting quotes in the right place. But there are many other tricks (such as negation and using 'site:') that their google searches could have benefited from.

the library has PEOPLE

[dangermouse]

What I find disconcerting is that so many people don't even realize that the library is not just a big stack of books, it's a service. Libraries have trained staff-- many of them degreed in library & information science-- who spend all day finding information for people.

Who cares if you don't know where to look for a piece of information? The reference librarian does. In larger libraries, there are usually librarians who specialize in particular fields of research. My university's library, for instance, has at least one research librarian assigned to each college or school within the university-- all degreed, and many dual-degreed in library science and their respective specialty fields. And they don't care in the least who is asking them for help-- it's not like the CS librarian will only talk to CS students.

Google is convenient, and fast for most searches, but there's a lot of information that just isn't available to it. Libraries buy access to that information, both in print and in databases, and they hire people to help you find the stuff you need.

The most important library skill, and the one that is most often overlooked, is recognizing the reference desk and asking for help.

Re:I know the answer

[schon]

It was Grace Slick, right?

Nope.

during We Should Be Together on Dick Cavett?

No, that was August 18, 1969. We're talking almost three months before that. :o)

Re:I wonder...

[trentblase]

It's more of a comedy of self-referential absurdity. Like looking up the Encyclopedia Britannica in the Encyclopedia Britannica. I bet they have an entry, after all it's the benchmark of encyclopedias. Maybe IHBT, but I think it's funny in a bizarre way that they might come up with a study about the effectiveness of internet research, realize it might have been done before, google for such a study, and find the results of that prior study. This would answer the original question in what I deem to be an ironic fashion. Yeah, I'm DEFINITELY not using a karma bonus on this monstrosity of a post.

Re:It still is faster... if you know how to use it

[TNS_the-bebop]

Google is like any other tool; one has to know how to use it in order to get the desired results. The thing I find interesting about this article, however, is that the tester obviously didn't know how to use Google effectively and was still able to find the answers pretty quickly on a fairly regular basis.

I know that if I was in a library, I would be hard pressed to find what "Sophie and Edward Wessex did on Tuesday." This tells me that Google is useful and accessible to just about everyone, whereas libraries are much less user-friendly.

That, in my mind is more than enough qualification for the title "fastest and most accurate way to find information."

Here's an example of the tester's inexperience with Google:

What was unusual about the British gold medal victory in the 400m in the 1908 Olympics in London?

It took the tester 1min 45sec. I thought that it seemed a little unreasonable for the search to take that long, and I was quite right. I searched for
British gold 400m 1908 Olympics and found the answer in about 11 seconds (on dial up). Who would have thought that the answer would be in the second paragraph of the first result? Also, I could have cut down my search time by viewing the cached page.