AllTheWeb Claims Bigger Index Than Google 302
An anonymous readers writes: "Hoping to attract more mass appeal for an online search engine with a cult following, Norwegian search engine AlltheWeb on Monday declared that it indexes more Internet information than longtime pacesetter Google. Boston.com has the story." Of course, pages indexed is not the only measure of a search engine and probably isn't even the most important.
AllTheWeb (Score:2, Informative)
When you click? (Score:2)
Also, lots of people prefer opening new sites in new windows. Myself included.
Re:When you click? (Score:2)
But alot of us don't. Which is why you should let the user choose. Or even better, do what Google does and give you an option to have the windows open in a new window (if you get sick of right clicking each link).
ALL YOUR WEB PAGES ARE BELONG TO US! (Score:4, Funny)
Great, you have a huge index. I know a haystack that has more than one needle, but the stack is about the size of Texas.
Re:Something else to consider... (Score:2)
Being a Norwegian company, would they be under the same mandate to hand over all 'suspect' search queries for abuse by the US's new CIAFBINSASSSASD (known in PRSpeak as the Information Awareness Office)?
Re:Something else to consider... (Score:2)
I would hope not, but perhaps there might be a profit angle involved. [montypython.net]
More seriously, do you have any knowlege that this "mandate" exists? Is it public law? Executive order? Secret executive order? A directive from "high levels"? Or is this more of an "intelligent concern" of yours? There's nothing wrong with that - everyone with half a brain should be concerned about these possibilities.
Re:ALL YOUR WEB PAGES ARE BELONG TO US! (Score:3, Funny)
*ahem*
Simplicity not quantity. (Score:5, Insightful)
Do I love google because it's so simple and easy to use with very quick download times and simple graphic interfaces, and good search algorithms that more often then not give me the sites that I am looking for in one page.
or Do I love google because it has a ton of useful sites logged in its database including all copies, half sites, under construction sites, etc.?
I am willing to say that's it's likely the first one, and I think that it might be that for most other people.
But either way, it'll be neat to see what AllTheWeb.com does well.
Re:Simplicity not quantity. (Score:3, Insightful)
The only problem was it found many real estate sites within the US, and other places. And these places have absolutely nothing to with the Alsace
What I also love with google is that it will translate my english searches into the appropriate language search. So Real Estate becomes "immoblier", which brings up another ton of results.
It is the little touches that keeps me coming back to google.
Hmm... (Score:2, Funny)
Re:Hmm... (Score:2, Funny)
Less webpages (Score:5, Funny)
Google: 185
AllTheWeb: 57
I'll stick with google. It indexes more interesting stuff.
Re:Less webpages (Score:3, Funny)
Re:Less webpages (Score:2)
Re:Less webpages (Score:2)
Google 7,760. Search took 0.39 seconds
AllTheWeb 1,761
However if I search on only my surname (which is very rare) I get
Google about 11,000. All the Web 11,374
This basically corresponds to cases where I am cited by first initial only.
This indicates to me that Google has much more accurate discrimination functions on cross matches.
Re:Less webpages (Score:2)
AOL. Actually, I've been using alltheweb from time to time. It has happened that it has been able to dig up some really obscure things that Google hasn't found, but generally, I find that if there are at least twenty pages likely to come up with the search words, Google does a whole lot better job ranking them. That really counts a lot.
Obvious Ads--bleah (Score:4, Informative)
However the first two returns for Scientology are the Scientology homepage and Operation Clambake. I wonder how long it will be before AllTheWeb is threatened.
Aside I'll need more proof that this thing is more accurate than google before I would consider switching.
Re:Obvious Ads--bleah (Score:3, Interesting)
Clambake (Score:2)
Could be interesting, since AllTheWeb is based in Norway, the same country where Operation Clambake is. They might say "DMCA, what?"
But then, they might not, since the index itself is probably in the US, and besides, our Big Sister Sunde thinks DMCA is Norwegian law anyway, so she'll be banging on the doors once she gets $cientology on the phone.
yeah but.. (Score:4, Funny)
Re:yeah but.. (Score:2)
Nobody I know...
Remember, the only reason Dilbert is funny is because we all get to see our pathetic, cubic lives reflected in his mirror. And when we see fun-house images of ourselves, we laugh. Scott Adams has a gift for bending the mirrors just the right way.
But when he "tries" to be original, he kind of falls flat IMHO. Sure he sells books & stuff, but I think it's only on the Dilbert name.
Of course, he brings money home by the truckload and I don't, so what the hell do I know?
Re:yeah but.. (Score:2)
then came he end of the week and I was majorly disapointed.
good not to have google monoculture (Score:5, Insightful)
In any case, it would be terrific to have a viable alternative to Google...despite Google's almost unnerving ability to do *so* many things Right, it is good to have somewhere to turn just in case something went wrong there. Not having a monoculture (which is what we're almost on the verge of with Google) is generally a good thing.
Re:good not to have google monoculture (Score:2)
Funny, my name didn't even get a hit, whereas it's the first match on Google. But, like you say, it's a tiny datapoint. I'll stick with Google though.
American Tech Companies (Score:5, Funny)
Re:American Tech Companies (Score:2, Interesting)
No. I'm English - and you're a colonial who has a drinking song [lib.ak.us] for a national anthem.
Re:American Tech Companies (Score:2, Insightful)
Well Last time I checked, the Scandinavian countries were market economies and parliamentary democracies..
Just cuz our chicks are hot and our health care is free doesn't mean you have to get your panties in a bunch and start throw the socialist label around!
3 cheers for scando chicks!
Nothing to write home about... (Score:3, Insightful)
This may be a case of a company picking a poor benchmark as their performance measurment. Google's draw is their great ranking logic, not index size.
-Pete
In other news ... (Score:5, Interesting)
Gnome declares itself better than KDE
Emacs declares itself better than VI
PHP declares itself better than Perl
Let the flames fly
How does this work? (Score:4, Funny)
Re:How does this work? (Score:2)
One thing I will give google a hand for is there ability/choice to not have to have large graphical ads anywhere. AllTheWeb has a banner ad at the top of each page, and another one that appears beside your search results. It is mildly annoying, and if it proves better, could be tolerable. Google's slim environment is very appealing though.
I'll have to try AllTheWeb for a little bit first and see if it suits my needs before I can give a better evaluation than this.
Re:How does this work? (Score:3, Informative)
This search engine can't even find monkeys, then forget about it.
They ignore Google's cache (Score:2)
This is far more important to me as a user than some extra pages that alltheweb may have (presumably because they ignored a few 'nobots' tags? that Google's crawlers respected?)
Google advantages (Score:2)
Banner ads (Score:2)
Where's TEOMA? (Score:2, Insightful)
They should be featured on one of those shows - Where Are They Now?
Re:Where's TEOMA? (Score:3, Funny)
Slashdot.org not among 10 first matches when searching for "slashdot"...
Needless to say, I never wasted more time by checking the next page...
Re:Where's TEOMA? (Score:2)
Danny.
Re:Where's TEOMA? (Score:2)
Enough with the Google worship, already. (Score:4, Interesting)
I just used Alltheweb for some common searches I do, and you know what? It found a lot more useful hits than Google did. Yea, imagine that.
But Alltheweb didn't seem to have a cache, which I thought was very useful in Google.
So, come on, folks, give it a chance, and don't jump to conclusions without an objective analysis. The tendency to blindly worship things like google/linux/linus/transmeta is far too common on this site.
Re:Enough with the Google worship, already. (Score:2)
Well, look at it this way... if you want to look at a fan site for something else, go there. Don't expect everyone to be completely objective. Fact of the matter is that most people who are subjective, are vocal about said topics. Folks that don't care, or don't want to inadvertently press anyone into anything generally don't peep.
It's not worship when... (Score:2)
And, Google forbid, should google start to suck, or something else start to be better, then I think most of us would find another search engine to "worship", like I (and I assume many others) did when Yahoo went down the toilet.
For me, the one mention of pop-ups and heavy graphic ads is more than enough to make it not worth my while to check out (and yes, I know, at home, I can filter out all the banner ad and pop-up garbage, but here at work I don't have the luxury of arbitrarily installing proxies and browsers to do that sort of thing. besides, web sites that use pop-ups piss me off).
Re:Enough with the Google worship, already. (Score:2)
Yeah! *looks around* Fuck Transmeta!
Re:Enough with the Google worship, already. (Score:4, Interesting)
Other than that you just say that alltheweb gave more hits for you on a couple of searches. That's a pretty useless measure of search tool quality to anyone else.
For my part I'm not sure what would make me switch from Google. I really value their interface, and none of the recent challengers are preferable to me in that regard. I switched TO Google because IMO it was a big leap in result quality over the earlier generation, but until we start using personalized intelligent agents, I'm hard pressed to imagine how a new engine could produce as big a leap. Basically, as search engine quality improves, for most people the sample of searches that they'd have to do in order for a comparison to produce clearly discriminable results also grows. Will very many Google users really sit down and do like 50 searches with both it and AllTheWeb, on the chance that ATW proves slightly better? I doubt it, and I'd hate to have to be in the planning room of a Google competitor, trying to think of the killer app that could get people to switch with just a few searches.
Though I'd certainly switch if I could have the Librarian from Snow Crash.
Re:wasn't a snip (Score:2)
I don't know if you're a Google-worshiper, but you certainly ran to its defense when faced with a strong claim from a competing search engine.
Re:wasn't a snip (Score:2)
I think you're right, except for the "in favor of Google" part. Timothy said, "pages indexed is not the only measure of a search engine and probably isn't even the most important." AllTheWeb claims that their page index is big, and Timothy is reality-checking that claim.
I think Slashdot editors get too snippy too often in their story posts. But this isn't one of those occasions.
Re:wasn't a snip (Score:2)
If the article had been about Google indexing it's N-billionth page, do you think Timothy would have quipped something about it's irrelevance? I doubt it. More likely there'd be the usual drooling.
Re:wasn't a snip (Score:2)
Nothing special this way comes (Score:2, Informative)
"php regular expression" AND "tutorial"
on AllTheWeb gives me 131 results, with more than half being a reference to a PHP website manual (and even a dislaimer footer because it had the words "PHP" and "and" in it ???). Moreover, it took my "and" literally as a search criteria, though my advance searching techniques could probably use a bit of help
In comparison, Google gives me 73 links (without omitted results showing) with many results displaying ALL my keywords in bold and not ONE of them using "and" as a keyword.
Dunno, I'm probably a bit biased anyway since "Google" types out so much easier for me (repetition i guess) than "alltheweb".
pblt....
RTFM (Score:2, Informative)
There is actually a help link. "php regular expression" + "tutorial" would have given you what you wanted. If you want to compare two tools you should at least use enough time to see if you have to use the two differently, and then see what is best at getting the job done.
Higher for a reason... (Score:2, Informative)
Google returns one accurate site for the company "DataHive", one domain name (not the proper one, but how would it know =)
This site returns 3 different domains, and tries to present them as different pages, though they all have the same content.
I can imagine its easy to claim more than google when you multiply the number of real hits.
I must say though, the results I found were pretty good for a number of queries. Definetly a google competitor. It does not seem to find all of the newsgroup/mailing-list stuff that google returns, good or bad depending on what you are searching for.
Its nice to have another competant option
Re:Higher for a reason... (Score:2)
I have a feeling this could be similar to the recent Hotmail changes, where a friend of mine with the last name Hancock was told he had to change his last name because it violated their rules. Same went for someone with the last name Hacker. When filtering like that you have to be very careful, because just because something is a naughty word in one context, doesn't mean it is in every context.
Re:Higher for a reason... (Score:2)
Targetted searching is needed, never mind size (Score:3, Insightful)
Google is my favourite search engine, even now, its ads are unobtrusive and don't pollute the search results. They've been good net citizens and they've done substantial research into how to better search. There results are typically the best as well.
In this case their search results were very broken however, at least for the purposes of my search. What I'd like to see is google, or an engine as effective as google, add in the ability to constrain your search to subject areas. In this instance I'd constrain my search to historical sites and would have received mostly uncorrupted hits. This is different than a web directory. Web directories don't classify sites based on there quality. Google does in a round about fashion, it lists sites with more people linking to it higher than sites with less links.
I'm not sure how the details of this would work, self-nomination would not necessarily work. Porn companies would gladly pollute the keywords on the off chance that somebody looking for history would buy a membership to their site. Letting individuals vote a site into or out of a keyword might work, though you'd be in danger of concerted efforts to say vote out anti-Scientologist information and vote in pro-Scientologist information when both actually could be under a religious keyword.
Anyway, linking to more sites isn't necessarily helpful in my opinion. What I'd prefer is the ability to narrow the focus of my searches.
Re:Targetted searching is needed, never mind size (Score:2)
The true test (Score:5, Funny)
Google: 63,500,000
AllTheWeb: 25,435,205
I think I'll stick with Google :o)
Re:The true test (Score:2)
I searched for "Juliette Binoche tits" on the image search of both sites (with filtering turned off in both cases) and neither returned satisfactory results.
Re:The true test (Score:2)
Damn. Spotted it just after I'd clicked submit.
Re:The true test (Score:2)
More pages = more crap (Score:2)
But can it withstand /.? (Score:2)
No cache. No images. No thanks.
:Peter
Cached here (Score:5, Funny)
Re:Cached here (Score:2)
Slashdot's names for comment modes are confusing. "Plain Old Text" is really a "do what I mean" mode: line breaks become paragraphs, but HTML tags are interpreted as HTML tags. If you don't want any tags interpreted, you have to use "Extrans" mode. Kuro5hin's names for the modes make more sense.
Major, Major Flaw (Score:4, Interesting)
"Redirection limit for this URL exceeded. Unable to load the requested page."
Which, as near as I can tell, is their way of throttling commercial hits. Wonderful. Moving the mouse over the link doesn't reveal the address in the bottom bar, either, so the only way I can think of to obtain the address of the item it matches is by right-clicking and selecting 'copy link address', opening a new window and pasting it it (and having a browser that is capable of doing this), then editing the URL so only the target link text remains.
You can't even right-lick and open in a new window to do this. If you try, you get "about:blank" which, afaik, means they're using javascript.
These people sure go through a lot pains to render a result and then not let you anywhere near it. Saying they're bigger than Google is a bit like someone bragging about how their PDP-11 is bigger than my Athlon. Cripes.
Re:Major, Major Flaw (Score:2)
The link does the same thing to me when using 1.1alpha on Win2k SP2+ (My primary browser and platform).
Internet Explorer 6 follows the link with no problems.
So I'd say it's a browser issue and not a web page issue.
Re:Major, Major Flaw (Score:2)
The link does the same thing to me when using 1.1alpha on Win2k SP2+ (My primary browser and platform).
Internet Explorer 6 follows the link with no problems.
So I'd say it's a browser issue and not a web page issue.
I got the same error he got when using mozilla so I looked at the source and it's not javascript coming from that page. I followed the link and got the same error so I decided to telnet to port 80 and check out the source on the page and this is what I got:
telnet www.alltheweb.com 80
Trying 66.77.74.20...
Connected to www.alltheweb.com.
Escape character is '^]'.
GET
HTTP/1.1 302 Found
Date: Mon, 17 Jun 2002 16:27:31 GMT
Server: Apache/1.3.24 (Unix) PHP/4.2.0-atw
X-Powered-By: PHP/4.2.0-atw
Location: http://www.kaosinc.com/jen.shtml
Connection: close
Content-Type: text/html
Your browser does not support HTTP redirects.
click here [kaosinc.com] to reach http://www.kaosinc.com/jen.shtml [kaosinc.com].
Connection closed by foreign host.
Looks fine to me.... the 302 (redirect) should be working fine. I wanted to try with IE but my laptop seems to be having somenetwork problems at the moment.
Very, very strange...
It's your webserver! (Score:2)
telnet www.kaosinc.com 80
Trying 192.203.175.245...
Connected to www.kaosinc.com.
Escape character is '^]'.
GET
HTTP/1.1 302 Found
Date: Mon, 17 Jun 2002 16:51:47 GMT
Server: Apache/1.3.23 (Unix) Debian GNU/Linux PHP/4.1.2 ApacheJServ/1.1.2
Location: http://www.kaosinc.com/index.shtml
Connection: close
Content-Type: text/html; charset=iso-8859-1
302 Found
Found
The document has moved here [kaosinc.com].
Connection closed by foreign host.
Your web server is broken (Score:3, Informative)
That is a Mozilla error message (source [mozilla.org]) and does not come from alltheweb. Your web server is broken. http://www.kaosinc.com/jen.shtml redirects [mozilla.org] to http://www.kaosinc.com/index.shtml, which then redirects [mozilla.org] to itself. This happens regardless of where I find the link to http://www.kaosinc.com/jen.shtml, or what browser I use to load it. IE appears to just sit there, Opera bounces between various stages of trying to connect, and Netscape 4 gives up after a few redirects and displays a raw 302-found page ("The document has moved _here_") without redirecting.
Moving the mouse over the link doesn't reveal the address in the bottom bar, either, so the only way I can think of to obtain the address of the item it matches is by right-clicking and selecting 'copy link address', opening a new window and pasting it it (and having a browser that is capable of doing this), then editing the URL so only the target link text remains.
An easier way to see the URL of the link is to hold the mouse down over the link, and then move off of the link before you lift the mouse button. But I still get the infinite-redirect error message if I type your URL directly.
You can't even right-lick and open in a new window to do this. If you try, you get "about:blank" which, afaik, means they're using javascript.
If I right-click on a link from the alltheweb search results and select "open link in new window", I see http://www.alltheweb.com/go/1/H/web/http/www.kaos
AllTheWeb _has_ one advantage (Score:3, Informative)
It was hidden as ftpsearch.lycos.com for some time, but now it seems to have come "home".
BTW: the last time their OS was visible through the firewall, it was FreeBSD...
Anyone remember archie ?
still very important (Score:2)
How about the image search then? (Score:3, Funny)
Google manage to get a graph of the slashdot effect [google.com] among the first 20 hits, while AllTheWeb just manage to get Cliff showing a Think Unix book [alltheweb.com] (in weirdo hawaiian clothes).
I don't know about you, but Google give me more relevant matches as usual.
Old pages, not grouped by site - less useful (Score:2)
Worse than the outdated and useless search results is the way they are presented - there is no grouping by site to put similar pages under one entry. Of the 167 results, almost all of them are from two distinct sites, but you have to wade through all of them to find any different ones. With a more common search string, it will be almost impossible to find what you are looking for, and it is still difficult with a narrow focus search. Google ain't going down that easy...
My index... (Score:2)
..is bigger than your index.
Computer scientists - pfft...
News? (Score:2)
Today the New York Times claimed that it had published "All the News That's Fit to Print."
One question remains unanswered: Will they be able to do it again tomorrow?
Note to moderators: This is sarcasm. It isn't off-topic. I'm implying that some marketing ploy by alltheweb.com isn't exactly newsworthy. Thank you.
Depends what you are searching for (Score:2)
For general searching google still rocks.
The old saying holds true: (Score:3, Funny)
It's not how big it is, it's how you use it.
Google is still way more useful in my opinion.
Real Life Example (Score:2)
Alltheweb's claims are not unfounded, and I find it always worth checking when google fails.
Here [sourceforge.net] is one of several real life cases where it found software for me that google didn't.
(It still does [alltheweb.com], and google still doesn't [google.com].)
hmm passes the pr0n test (Score:2, Redundant)
Google Pr0n Search [google.com] finds 46,200 results.
Searching for pr0n via alltheweb.com leads to 2318 more potential pieces of pr0n to be seen.
Vote Google (Score:5, Interesting)
I'd rather support a company that uses subtle advertisements like Google does than a company that uses in your face banner ads, etc. (Then again I'm posting on Slashdot!) Also I make a point to check out the ads evey now and then on Google and visit the company's site. I may be getting hosting from an advertiser on Google soon.
If people who advertise on Google make more money than they do with banner ads, pop-ups, etc. then we'll see the idea spread. I don't like in-my-face ads, so I do what I can to tell companies that. It's called being a responsible consumer.
Plus more valid hits come up when I search for myself on Google
Hrm... (Score:2)
I can't belive how many people have my "Subtle mind control? why do all the HTML buttons say 'submit'" quote on their sites.
Re:Hrm... (Score:2)
Have you seen this? Submit [randomfoo.net]
Not entirely fair (Score:2, Interesting)
Of course, as has been mentioned a few times above, competition is a Good Thing (TM).
- Ardenstone
THE (Score:2, Interesting)
Then how come the word with the most search results (FYI: the) on Google [google.com], returns less results on alltheweb [alltheweb.com]?
Main Pages Found (Score:2, Interesting)
slashvertisement? (Score:2)
'I've got it Herbert! Let's make some inflammatory claim about Google that has nothing to do with the actual quality of either sites results and sit back and watch the hits roll in!'
yeah, so it's an obvious troll, but i guarantee you it's true.
bigger index doesn't always help (Score:2)
It's not hard to find our site, either. Our company's name is "foo bars"* and our URL is "foobars.com." Google nailed it, while AllTheWeb bombed.
Doing a more complex search with lots of words from our home page did, finally, get AllTheWeb to cough up our site. So I know it's in there.
So in my opinion it has little to do with how big their index is. It has to do with how good they are at finding what I'm looking for. For me, Google almost always finds what I'm looking for. I've even started using the "I Feel Lucky" button to skip the search results altogether and just take me straight to the first listed site.
*Incidentally, I've always wanted to open a pub called the Foo Bar, but I don't think many people would get it.
comments on AllTheWeb (Score:2, Interesting)
The ordering of pages seems less helpful. In many cases, the page I'm looking for is farther down the page.
The sponsored links and advertising are way more noticeable, and get in the way of the search results, although they're probably easy enough to ignore.
Google seems to be better at rating by search term proximity, under the useful assumption that if the search terms occur close to each other, it is less likely to be a random hit. One irritation with AllTheWeb is that for many results, it doesn't show you the context of the search terms in the summary.
Obviously AllTheWeb lacks the excellent USENET archive. The video and MP3 search festures might be pretty useful, I haven't had a chance to try them.
I realize I'm coming across as entirely pro-Google, but these are the only observations I have right now. I'll give AllTheWeb a chance, and let internet darwinism settle the issue.
alltheweb claims (Score:2, Interesting)
Our old friend ftpsearch.ntnu.no reincarnated (Score:2)
Glad to see it's back, after a sojourn as a non working component of Lycos.
Catchy name (Score:2)
As a bonus, alltheweb (when properly separtaed with spaces) is proper English.
size matters? (Score:2)
:)
A small test (Score:2, Informative)
Fnord: Google: 104000 AllTheWeb: 46439
Cheese: Google: 3690000 AllTheWeb: 7718252
Linux: Google: 48000000 AllTheWeb: 26670311
Windows: Google: 44600000 AllTheWeb: 66545303
Extropian: Google: 4460 AllTheWeb: 3999
Kumquat: Google: 32600 AllTheWeb: 42889
Question Authority and the authorities will question you.: Google: 90 AllTheWe b: 74
Hot man meat: Google: 229 AllTheWeb: 1661
Hot pussy: Google: 104000 AllTheWeb: 770057
"undefined reference to" error: Google: 31700 AllTheWeb: 8548
"Antimatter-Catalyzed MicroFission / Fusion": Google: 6 AllTheWeb: 1
Surprisingly alltheweb does return more hits in some areas, most notably for che ese, windows, and pr0n. With the cheese test, AllTheWeb helpfully cluttered my s creen with a banner for food products. Google, thankfully, is still bannerless, and returns more linux hits, fnords, and Voltaire quotes. Alltheweb also stalled several times and I had to resubmit a search. Conclusion: If you're a linux gee k or you want to know about fnords, futuristic philosophies, compilation errors, or advanced space propulsion concepts, google is better. If you're a horny wind ows user and want to find gay or straight pr0n, and if you for some reason like kumquats and want to learn more about cheese, use alltheweb.
Seriously, I'll probably stick with google, better numbers or no. The only thing AllTheWeb has going for it is the ftp search. The original is owned by lycos no w and broken.
The REAL Measure of a Search Engine... (Score:2)
Right?
Google creating a new information economy? (Score:2)
Washingon Post article (Score:2)
Re:google wins! (Score:2)
Re:google wins! (Score:3, Interesting)
Re:google wins! (Score:2)
Re:Kuro5hin is dying!!!! (Score:2)
Re:I have seen the Fast technology. (Score:2)
I'm interested.