Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
The Internet Education

Putting Google to the Test 441

Posted by michael
from the time-is-of-the-essence dept.
Big Nothing writes "Google has built its reputation on being the fastest and most accurate way to find information. But is the internet really the quickest way to access facts - and get them right? The Guardian puts Google to the test against more old-fashioned methods."
This discussion has been archived. No new comments can be posted.

Putting Google to the Test

Comments Filter:
  • Not versus, with (Score:5, Insightful)

    by Angry Black Man (533969) <vverysmartman@nosPAm.hotmail.com> on Thursday May 06, 2004 @09:25AM (#9073055) Homepage
    Instead of comparing them against each other, its more important to use both internet based as well as "old fashioned" resources together. Its important to realize that hard backed enyyclopedias are better than google and wikipedia for some things, and not for others. The younger generation needs to learn how to recognize what source to use, instead of automatically going to google. The internet should not replace old fashioned resources but merely embrace them.
  • by dunedan (529179) <antilles@@@byu...edu> on Thursday May 06, 2004 @09:25AM (#9073061) Homepage
    That none of the questions included something likey what is the maximum sustainable speed in Mb/s of the alcatel 8100 series router

    Thats the stuff where Google with kick everyones trash, not complete list of authorships
  • Library (Score:5, Insightful)

    by artemis67 (93453) on Thursday May 06, 2004 @09:26AM (#9073062)
    He didn't count the time it took for him to leave his office and drive to the library. So add another 20 minutes to all of the library times.
  • Re:Library (Score:5, Insightful)

    by siliconwafer (446697) on Thursday May 06, 2004 @09:27AM (#9073086)
    Exactly. Google (and the entire internet) are accessible from the comfort of our homes. Going to the library to research requires getting up and actually going there. While traditional methods of research certainly have their merits, nothing beats the convenience of Google.
  • by lukewarmfusion (726141) on Thursday May 06, 2004 @09:30AM (#9073113) Homepage Journal
    Their library lookups don't appear to have included:

    Looking it up in a card catalog (electronic or not)
    Finding the book/periodical on the shelf
    Accounting for missing resources (like a real life 404!)

    Yeah, I'm a Google fan. Sometimes the library is better - but not for factoid lookups or finding out what the Royal Wessex couple did on Tuesday.
  • by kbsingh (138659) on Thursday May 06, 2004 @09:30AM (#9073116) Homepage
    The comparison dosent seem to be so much as google / other means - its more of an Online V/s Offline means to search for specific stuff.

    I think its wrong to brand Google as the only means to look for information online.

    Secondly, the issues that the reviewer raises are also adhoc - they cant be used to generalise the entire deal / spectrum of infomation that people need / want / desire.

    Try looking for a code sample that shows you how the GTK# can be used from Mono to display a Multi level Outline filelist. What are the options that you have for this in the Non - Online world ?

    The guy already knew who to ask / who to talk to - what if you dont know that - what then ? how do you go about finding the best non-online resource to speak with / enquire from ? My guess is that you are going to be heading right back online.

    What about the fact that the online resources / google are avilable to you when you want it - how you want it and where you want it. Ever looked up what a word from the bible meant in the middle of sunday mass at the local church using a Wap phone over gprs at wml.google.com ? Me neither....

  • by Dareth (47614) on Thursday May 06, 2004 @09:30AM (#9073125)
    These are just times for the homeless guy living behind the library who hangs out there all day harrassing patrons and looking up porn on the library computers.

    For people who have computers and access at home, the internet has many sources. The web is not the whole internet, nor is google the whole web.
  • Re:Library (Score:3, Insightful)

    by tolan-b (230077) on Thursday May 06, 2004 @09:31AM (#9073126)
    Also, I suspect being a journalist you would have a better idea of who to ring for various information, as research is a fairly important part of journalism.
  • by beh (4759) * on Thursday May 06, 2004 @09:31AM (#9073129)

    After reading the article, I feel there is a slight bias in favour of the libraries when looking at the questions. Of course a library has a master index of books of one author. Or - to find out about some very specific question about an event you immediately know what kind of journals to look in.

    The only question really geared for search engines was the Thatcher quote (as that would be a full text search).

    Would this be the time to create a true categorisation of questions to be used in comparisons? (Note - not the ACTUAL questions, so that search engines could optimise for them, but only specify the general direction of questions).

    I admit, it would be pretty hard to do, but I guess it could be worth the effort...
  • by log0n (18224) on Thursday May 06, 2004 @09:34AM (#9073169)
    This could be because no-where in the top page source does Google ever have the word 'engine'.
  • by millahtime (710421) on Thursday May 06, 2004 @09:35AM (#9073180) Homepage Journal
    Like you said, you didn't know the other places to search for things. My kid sister has the same problem. She doesn't know other places to find information other than the web. This is a shortcoming we now have because we rely to much on one source and grew up doing it.

    As far as looking for the information in places other than the net, I found my mom knows all those places and where to find things quickly without the web or google.
  • Bias? Proof: (Score:2, Insightful)

    by Anonymous Coward on Thursday May 06, 2004 @09:35AM (#9073186)
    Google

    1min 17sec (1st)

    1,201 km (499 km of which is electrified). I type "percentage" as well as "Slovenian railway system" and "electrified". Google isn't playing with that combination at all, so I take out "percentage" and separate "Slovenia". Scanning the results, I choose a site I've visited before: the CIA World Factbook, Washington's greatest gift to the web. I am prepared to trust the CIA on Slovenia. For the time being, anyway.

    Verdict: The higher figure attained over the phone may be more up to date

    Phone

    1hr 4min 5sec (2nd, after disqualification of Stephen Moss)

    It's 5pm in Slovenia by the time I begin and according to Bo, at the embassy in London, Slovenians go home at 5pm. Sure enough, when I call Bo's number for Slovenian Rail, the phone rings unanswered. So I call him again. He puts in a few calls. I wait. Then he calls back: it's 60-65%, equivalent to 1,200km of track. He stresses that this information is provisional, but I owe Bo a beer.

    Verdict: Slow, but perhaps likely to be the latest and most accurate information


    Google took 1 minute 17 seconds, with an answer of 1,201km. The verdict is the LARGER number produced by phone is more accurate. Phone's answer: 1,200 roughly (60-65%) and took 1h 5m. It's a smaller number, a rough guess, and took over an hour! How is the phone more accurate again?
  • Searching skills (Score:5, Insightful)

    by OblongPlatypus (233746) on Thursday May 06, 2004 @09:35AM (#9073189)
    Question 3: Who is the vice chairman of the all-party parliamentary group on back care?

    Google - 6min 27sec (3rd)

    Quote: "Unfortunately, "back" is rather a common word, and is turning up in all sorts of irrelevant documents..."

    Entering "back care" in quotation marks got me the answer in 25 seconds, much less than either of the "offline" sources. If they're going to have an accurate test, at least make sure the person performing it knows how to use a search engine.

    Or maybe I'm wrong; maybe most people don't have these basic searching skills, in which case the test is accurate after all?
  • by thebra (707939) * on Thursday May 06, 2004 @09:37AM (#9073200) Homepage Journal
    I would think it would take 5 mins just to find the phone number to the library, get some one to answer the phone and then explain the question, PLUS have them search for answer. This also would all depend on the library you call. Google is google no matter where you live, but not every library is staffed when the same people.
  • by GreyPoopon (411036) <gpoopon @ g m a i l . com> on Thursday May 06, 2004 @09:39AM (#9073215)
    Although this article is entertaining, the search results are meaningless. The power of Google is in the fact that I can find something I'm looking for in just a few minutes without having to leave my desk or engage in social pleasantries. To really make the results accurate, you must at least include all the factors that impact the time it takes to arrive at an answer. For calling someone on the phone, you need to add in the time it takes to figure out who to call (maybe they did that). For the Library, you need to add: drive time, time spent waiting for the online (or paper) card catalog to be available, and the time it takes to find an appropriate reference in the catalog and then go get the book or periodical you need. You can't count on people knowing exactly what source of information to use for the facts they want to know.

    I believe in using the right tool for the job. If you are in the middle of something at work or at school and need to check on a fact real quick, use Google. If you are doing in-depth research on a topic, you are probably better off first going to the library because it's easier to determine the quality of your source material there. Afterwards, you can supplement with a bit of Googling and you'll probably know whether your search results are useful or pure hogwash. The phone call method? Use that if you're lonely.

  • by Albanach (527650) on Thursday May 06, 2004 @09:39AM (#9073218) Homepage
    From the article: Question 3: Who is the vice chairman of the all-party parliamentary group on back care? Google - 6min 27sec (3rd)

    But if you search google for "vice chairman" "all-party parliamentary group" "back care" you only get two results which are actually for the same document - an alphabetical list of all-party groups. Scroll down to back care and there's your answer. Why would that would take six and a half minutes?

  • by Anonymous Coward on Thursday May 06, 2004 @09:39AM (#9073223)
    Go to google and search for:

    Best Search Engine in the World

    Hit I'm feeling lucky.
  • by pubjames (468013) on Thursday May 06, 2004 @09:41AM (#9073247)
    That's what I thought when I read the article.

    I bet a normal person would do considerably worse trying to find those answers on the phone than a professional journalist.
  • Re:I wonder... (Score:4, Insightful)

    by Anonymous Coward on Thursday May 06, 2004 @09:41AM (#9073252)
    it's interesting that your 'library searches' seemed to take only a minute or two to perform. You didn't bother counting the time it took to actually go to the library and find the relevant book, then?
  • by Anonymous Coward on Thursday May 06, 2004 @09:46AM (#9073302)
    Usually leaving out quotes on your first search attempt makes sense, so that "care for your back" would match too.

  • by Tanami (601011) on Thursday May 06, 2004 @09:47AM (#9073318)
    I found the answer to the question 3 in about 30 seconds, well under the 6m 27s quoted by their researcher. It's clear from their comments about irrelevant pages that they hadn't enclosed 'back' in quotes to form "back pain", as '"back care" parliamentary group' puts the result on the third link (from google.co.uk). Also, it doesn't seem very fair to compare a researcher who doesn't think to use quotes round that expression with a librarian who knows to look "on page 242 of the excellent Vacher's Quarterly", a publication with which I (and most of the public, I would imagine) have no familiarity whatsoever.
  • Re:god google (Score:5, Insightful)

    by lphuberdeau (774176) on Thursday May 06, 2004 @09:49AM (#9073336) Homepage
    Google knows nothing, except where words are placed.

    But really, that test does not consider the fact that it takes a while to go to the library and that you actually need to get out of your house. Plus, library isn't available at night, neither is most people you can try to call.

    Google sure wins any convenience test.
  • by NitroWolf (72977) on Thursday May 06, 2004 @09:50AM (#9073349)
    This isn't a comparison of anything...

    The library searches don't include travel time. They also appear to only count the time it takes you to read the text in the book... not to:

    a) Find WHAT book you want (Card catalog?)
    b) Locate the book on the shelf
    c) Find the correct page

    All those things take the MOST amount of time, not reading the actual text. This is assuming that you KNOW what book you're looking for to begin with. I had no idea Who's Who would be a good place to look for the answer to the author's books. Google would have given me the answer pretty quickly without the need to know that information. How much more time would it have taken to find out Who's Who is the book you wanted?

    Add on top of the fact that I'd have had to drive ot the library, and the time increases dramatically.

    Calling a friend? Maybe faster, but I don't have many friends that would know answers like that... nor do I have the number to railway stations on speed dial... especially those in other countries.

    Google is simply the fastest AND most convenient method to find the information. Or at least, if not Google, SOME search engine. If you're already at the library and KNOW what book you want, it might be a better choice, but seriously, how often does that happen? How often do you sit at the library and think of things you want to know?

    I don't... I'm usually sitting at home reading, or surfing the web and come across something I want to know more about. Driving to the library to find that information would be ludicrous... and calling my friends regularly with mundane questions would cause me to lose what little outside life I already have.

    Bleh... this isn't even an aritcle worth reading... jeez.
  • by Anonymous Coward on Thursday May 06, 2004 @09:53AM (#9073375)
    Google is not accurate for phrase searches. A good example is "to be or not to be". Typically, this comes up with from 1 to 3 of the top 10 results being totally wrong (not being the phrase). Right now, it brings up one that contains "tobeornottobe", which I did not ask for. Other times, it brings up misspelled variations. This search is just one example: I get bogus results in long phrases all the time.

    Note:

    Do not mention meta-tags of pages linked to the result pages. According to Google, these are not used to return results, only to rank them. Pages containing the phrase linking to pages returned in the results have nothing to do with the actual results being returned (according to Google)

    "tobeornottobe" is an erroneous return for "To Be Or Not To Be". "go ogle" is not google. "Now Here" is not nowhere.

    I did use quotes around the search. Typically, someone says "use quotes and it works" and gets modded insightful. Neither the person nor the moderators bother to try the search to see that it produces error results with quotes around it.

  • by whiskeypete (305461) on Thursday May 06, 2004 @09:53AM (#9073384)
    Yes they were able to get the required information from the library, but the only way to confirm that the data was accurate was to compare it to a known "up-to-date" source.

    If I used the encyclopedia that was available in my High School library (in 1983) I would have learned that because of the recent Sputnik launch that man would someday walk on the moon.
  • Re:Library (Score:3, Insightful)

    by surprise_audit (575743) on Thursday May 06, 2004 @09:55AM (#9073406)
    That definitely should be taken into account. Not everyone has extensive contacts they can call at any time for random information. Not every library is close-by, and they're certainly not usually open 24x7x365.
  • Re:Library (Score:3, Insightful)

    by nautical9 (469723) on Thursday May 06, 2004 @10:00AM (#9073478) Homepage
    ... as research is a fairly important part of journalism
    That should be moderated Funny, not Insightful, at least by today's journalistic "standards".
  • Re:I wonder... (Score:5, Insightful)

    by TheViciousOverWind (649139) <martin@siteloom.dk> on Thursday May 06, 2004 @10:02AM (#9073502) Homepage
    So you're telling me that the first library test which took 20 seconds, involved looking in a card index, fetching the book and looking it up? Or did he have the relevant book in front of him already? - That strikes me a bit as cheating, otherwise I'd say it would take longer than 20 seconds just getting the book and opening it.
  • by mopslik (688435) on Thursday May 06, 2004 @10:08AM (#9073559)

    In the library at the firm where I work, I know exactly where Who's Who is. East wall, middle set of shelves, on the second shelf from the top. Can't miss it.

    This is where the "Google vs. library book" analogy isn't quite accurate. Google doesn't contain any info itself -- Google is like the (card/electronic) catalogue system. It points to sites that contain the information. Who's Who, on the other hand, is a specific book with information. Granted it's pretty generalized, but still...

    By knowing exactly where WW is already located, it's like saying "I already know the URL of the website that should give me the answer". That URL is equivalent to the book itself. In this scenario, using Google is a pointless exercise, and the "Internet" seek time would be much lower.

  • Anecdotal at best. (Score:3, Insightful)

    by steve buttgereit (644315) on Thursday May 06, 2004 @10:12AM (#9073606) Homepage
    Well, this isn't a very good test. Really what we learned is how effective one individual can be using google compared to other methods. Even then the tests didn't try to see how this applied to different kinds of information or how these results may have different from a group perspective. Worse still, the results for the author's first question makes me question if the author knew the answers ahead of time and had no way to call a result 'correct' or otherwise.

    Also, more comprehensive searches at a library could involve actually having to visit the library... with it's associated drive time.

    A good test would have had more questions, more participants and questions selected for a vareity of information types. The premise of the article I think is interesting: what kinds of research is the net really good for? Other than porn, of course, which is a given (try not finding it).

    The problem with Google (et al) isn't finding information: it's finding reliable information (for most subjects). There's a hell of a lot of noise out there.

    Cheers!
    SCB
  • by wintermute740 (450084) <wintermute@@@nitemarecafe...com> on Thursday May 06, 2004 @10:13AM (#9073616) Homepage
    "Besides, half the fun of researching in the library is the irrelevant but interesting information you stumble across as you browse!"

    I remember a time when that was half the fun of using the internet for research as well.
  • Advantage: Google! (Score:5, Insightful)

    by Don Tworry (739153) on Thursday May 06, 2004 @10:16AM (#9073650) Homepage
    Google has some other advantages that phoning and the Library don't have:

    1. Google is pretty much 'always on'. I can do a Google search any time of day where as I can't use the phone or the library at 3 am.

    2. The ability to Find a keyword. Usually when I use a google search I use the google cache. This highlights the terms I am looking for so I can find them easily on the page. This is an inherent advantage of the computer over people or your eyes - scanning through text looking for what you really want.
  • by Retired Replicant (668463) on Thursday May 06, 2004 @10:17AM (#9073662)
    The times they list for the library searches are bogus because it doesn't include the time needed to walk or drive to the library. With Google or the phone you can start your search immediately right from your desk at work.

    Libraries are expensive dinosaurs. All information in book or journal form should be digitized and put on the internet.

  • by Smidge204 (605297) on Thursday May 06, 2004 @10:17AM (#9073669) Journal
    It also doesn't account for the fact that Google is not responsible for the accuracy of the content, only the relevance of the results compared to your search keywords.

    Books in the library tend to be checked and reviewed for accuracy of thier content. Websites generally are not. Even then, books might be wrong, so it is still up to the person doing the research to determine if the information presented is good or not.

    Also, it seems the author's googling skills are somewhat lacking. It took me less than a minute to look up who the vice chairman of the parliamentary group on back care was... all I did was search for vice chairman parliamentary group on "back care" (Note the quotes!). First hit for me. Use Google's PDF view-as-html link, and scan for the magenta highlighted text ("back care"). Presto, "VCh. Janet Dean MP (Lab);"

    Here's the query results [google.com]. First link, about 1/4 way down...

    I suppose it would help if the author decided to give a little more information about his searching methods instead of just saying "it took me xyz minutes to find it". I suppose it would also help to learn the tools, because I openly admit it would probably take me much longer than him to find certain kinds of information in a library than on the internet!
    =Smidge=
  • by Anonymous Coward on Thursday May 06, 2004 @10:21AM (#9073706)

    There is absolutely no comparison when it comes to low priority information - things you're curious about but aren't willing to spend any significant amount of effort to locate. This is where internet resources really shine; you can quickly obtain little bits of useless information with minimal effort. Of course, I think all of the examples in that article fall into this category, and the times listed were a bit skewed by the author's inability to use a search engine, endless lists of telephone contacts, complete knowledge of the contents and locations of almost everything in the library, and ability to travel to the library and the appropriate volume at the speed of light, but that's already been discussed...

    An hour or so ago I overheard some people talking about a joke involving the three phases of sex and something about hallways. I suppose I could have walked over and asked them to tell the joke, but instead I was able to find it in a few seconds using Google. Imagine asking a librarian for that piece of information...

  • Lexis-Nexis (Score:5, Insightful)

    by squashed (664265) on Thursday May 06, 2004 @10:21AM (#9073711)
    How about a meaningful comparison of electronic data retrieval services?

    Compare Google to Lexis-Nexis.

    Lexis-Nexis has boolean logic driven search (not natural language), and lacks "PageRank", but it includes all sorts of major periodicals not offered and certainly not archived on the web.

    Lexis-Nexis would win hands down in all sorts of categories of questions.

    It's an object lesson in the impact of intellectual property laws on access to information in our societies.
  • by royalblue_tom (557302) on Thursday May 06, 2004 @10:25AM (#9073749)
    But they have got an ordinary Joe Shmoe doing the google. I did the parlimentary search using "vice chairman"+"back care"+"parliamentary group", and I get two identical hits, a PDF that has the right answer. Why this took the guy 6 minutes I don't know, unless he was searching on "back" ... not a highly competent google user.
  • by Saint Stephen (19450) on Thursday May 06, 2004 @10:29AM (#9073818) Homepage Journal
    I have a feeling this whole thing on Slashdot is part of an atroturfing campaign by google to hype up the stock prive.
  • by JayJay.br (206867) <100jayto@gma[ ]com ['il.' in gap]> on Thursday May 06, 2004 @10:32AM (#9073862)
    I find it slightly disconcerting that we may be producing a generation that has no research skills bar Google.

    Heck, I find it even MORE disconcerting that most of those people you are talking about are people who cannot effectively use Google itself!

    For instance, most of the people I know feel disappointed by Google because it always gives too many results for their searches. I always try teaching them to RTFAdvanced Search, using double quotes, or using keywords like inurl, intitle, filetype, and so on, to narrow it down. But they never put that in practice, as it feels better to keep on bitching than learning how to use the tool. Makes me wonder if they could find their own heads if not attached the their necks.

    Maybe the Internet is dumbing every non-techie down.
  • by haystor (102186) on Thursday May 06, 2004 @10:34AM (#9073903)
    Your assignment is to compare two authors. The lesson is on how to do the research. While in this particular case the internet was faster, you have passed up the learning experiences the library has to offer. Basically, the professor was asking you to not use a calculator.

    I'm convinced that my ability to find what I want on the web was greatly honed by my time spent in the stacks. I also miss a bit of the serendipity of where the card catalog could lead but that is just nostalgia that doesn't recall all the dead ends.

  • by Anonymous Coward on Thursday May 06, 2004 @10:47AM (#9074080)
    Probably because they didn't think someone would be so cheap as to cheat in a pub quiz for $20. They probably didn't put a rule in, on the grounds that they hadn't needed to, because the people there to enjoy themselves were not competing *just* for the money. Now, the spirit of the event is ruined for everyone. You are indeed a sad individual.

    Admit it, you acted like a child. Grow up.
  • Travel time (Score:1, Insightful)

    by Anonymous Coward on Thursday May 06, 2004 @10:47AM (#9074090)
    Interesting to see all these complaints that the travel time was not included in the library time. This automatically assumes that every time we want an answer to a question we are sitting at a computer (stupid question on Slashdot).

    What if I'm out at lunch in town, 30 minutes from my office, an hour from home and two minutes from the library? What about adding in the time to travel to a computer to access Google?

    Of course, of the three options only a phone can be with you at all times so it automatically is the quickest in most situations, whether hiking in the mountains, sitting on a train, in a pub etc.
  • by Michael_Burton (608237) <michaelburton@brainrow.com> on Thursday May 06, 2004 @11:44AM (#9074801) Homepage

    But is the internet really the quickest way to access facts - and get them right?

    It's not a fair test. The "get them right" requirement skews results against the internet.

  • by henryhbk (645948) on Thursday May 06, 2004 @11:47AM (#9074843) Homepage
    This is similar to medline, where professional researchers and librarians have pre-indexed the material with known keywords (on medline called MESH headings). The key is that unlimited free-text searching is usually much poorer versus use of predefined keywords.


    Since there is a committee that predefines the keyword, and a modern search engine (on medline for instance Ovid), will map your free text to the MESH heading to which all articles have been mapped by a review committee. This is simply shifting the time to lookup to someone else. These articles are essentially "pre-looked-up". However, it makes the search much better, as someone who actually knows how to search, has pre-classified the articles with all the relevant search terms. Free text searches like google, on these massive databases typically return thousands of articles with marginal relevance.


    And like most users of these specialized DB's, there are professional librarians available at most sites (or available via phone/email) to assist in searches, since these are mostly for business purposes (where time is money).

  • by hak1du (761835) on Thursday May 06, 2004 @12:04PM (#9075016) Journal
    It also doesn't account for the fact that Google is not responsible for the accuracy of the content,

    Neither is your library.

    Books in the library tend to be checked and reviewed for accuracy of thier content. Websites generally are not.

    Many websites are. And many materials at the library aren't. Either way, you have to figure out who can be trusted yourself.
  • by mossmann (25539) <mike@ossmann.com> on Thursday May 06, 2004 @12:22PM (#9075199) Homepage
    Even better:
    all-party "back care"

    Feel lucky and you can't miss it. The problem was that the tester didn't use google at all for this one. He thought the parliament web site's built-in search would be better. They say they're testing google, but they're really just testing the surfing habits of one guy (who uses google when he feels like it).

    It would be interesting to see a more scientific study along these lines: more information targets, more users, and some kind of standardized way to measure time (including travel time, etc). Except we all know how it would turn out. . .
  • preknowledge (Score:2, Insightful)

    by claurent (680933) on Thursday May 06, 2004 @01:26PM (#9075819)
    It would see that some preknowledge of the answers was going on.

    For example, on the tuesday wessex couple question, one of the keywords used for their google search was "engagement"...the searcher already had preknowledge of the answer.

    That kind of skews the results.
    Maybe this happened on the others too. They didn't list their search pattern.

It's hard to think of you as the end result of millions of years of evolution.

Working...