Gap Between Google and Competition Widening

eldavojohn writes "Business Week has up an article trying to explain why it is getting harder and harder to 'catch' Google in the search engine game. We've heard of many different kinds of search engines and many different companies entering the market but: '... Google keeps gaining share in the face of newly launched capabilities on other engines. In August, Google sites gained 6.8 percentage points of search share from a year earlier, according to researcher comScore Media Metrix. Meantime, Yahoo lost 1 percentage point, Microsoft's sites lost 3.3 percentage points, and Ask.com lost one-half of a percentage point.' All of this on the heels of recent news that A9 scaled back its features. Is it possible to think of a number better than a one with a hundred zeros behind it?"

Odd

[pdbaby]

It's odd that people should say Google are widening the gap... Google's certainly the best, but lately I've been noticing a lot more search results that lead to pages that don't load, or result in 404s (in fact, a domain I used to run 3 years ago is still in Google's index).

Is google not removing ages from their index to try and seem impressive, or getting lax with recrawling sites? Or am I the only one noticing this?

Ganz Scheiß

[DerGeist]

This is all very ironic nonsense; Google showed us just how easy it really is to catch up with "the big guy." Back when Yahoo and Altavista were king, Google overthrew those powerhouses like it was Superbowl III. The key was Google knew what people wanted, and gave it to them. Now, certainly, it doesn't seem like Google is going to forget that anytime soon, and no, it's also not likely at this point that a little guy could wipe out Google, but who knows?

Look at the way great ideas have grown quickly: YouTube, digg, and so on.

And you can bet that if someone came up with a radically new search algorithm that provided noticeably better search results than Google (which is actually falling a bit behind, which is a dangerous mistake...) you can believe that most people would quickly migrate to their new engine of choice. (Of course, if it had little to no ads, speedy and reliable service, etc...)

Re:Odd

[Soul-Burn666]

Noticed that too. Many 404s, outdated pages (which are USUALLY in the cache tho) or pages that have not a single reference to the search terms.
Another truly annoying set of results are links to other SEARCH sites indexing some pages which may or not have anything to do with the search terms.

Being a HUGE Fan of Google...

[eno2001]

...I have to say this is great news but not wholly unexpected. If you provide a very useful service for free, then you deserve to be at the top of the heap. There's no competing with free unless you're, oh how should we put this... BETTER.

Re:Um..

[waa]

Where did my 5 moderator points go?
The parent should have been modded "Informative."
Why the -1? Sheesh.
Information is not a bad thing" (tm)
Sadly, in the US, a disproportionate number of people can not identify The United States on a map!
What makes one think that they know that "google" is not "googol" as the parent tried to differentiate?
Hell, I'd be willing to guess that a full 80% or more don't know that a 1 with 100 zeroes is a googol.

it's the number of sites.

[krell]

I like Altavista's actual search mechanism better (it is more precise, and makes sure that the returned pages contain the information I look for without having to place Google's annoying required) "inanchor" tag. However, sites like Altavista and others don't have anywhere near as many pages as Google. Also, the UI/interface/look on all of the other sites really sucks compared to Google.

Better search options

[iangoldby]

I find that Google is becoming less and less useful.

The web now has become so large that a simple keyword search just doesn't cut it anymore. Try searching for information about a popular digital camera from someone who isn't trying to sell them. It is next to impossible. (Yes I know about http://www.givemebackmygoogle.com/ [givemebackmygoogle.com] - a good try, but not really addressing the fundamental problem.)

The best way that search could get better in my opinion is to introduce some kind of filtering on the type of organisation that produces the pages you are searching for. Google already does a bit of this with Google Scholar [google.com]. But we need something far more general, and more to the point, a facility for excluding results of particular types, e.g. blogs, sites trying to sell something, ...?

I know that some people will complain that it may be a very subjective judgement whether site X is commercial or not. But search results are never going to be perfect anyway. Let's have the improvements where they are available, and worry about the corner cases later.

I see no significant difference between Google

[Krotos]

and, say, Clusty.com, except that the latter doesn't collaborate with the Chinese Communist Party.

Is it a case of "tall poppy syndrome"?

[russ1337]

It seems that if you cant catch them, you take shots at them to change peoples view, or you try to change the rules to make it harder for them to succeed. We are seeing this with the threat of a non-neutral Internet where most proponents nearly always use Google as the example.

I think the shots at Google are a little bit of 'tall poppy syndrome' [wikipedia.org] kicking in. The only thing keeping Google from being resented is their 'humility' - that they aren't flaunting their position and their committed to 'not be evil' - like not handing the info to the NSA without a warrant like the 'others' did.

Firefox

[managementboy]

I bet there is a correlation between the switching rate from IE to Firefox. It having google as prime search engine makes up for a lot of searches.

Re:Time for some anti-trust!

[Concerned Onlooker]

No, successs and being the best isn't wrong. Using your success and money to oppress others through unethical business practices is. There is a huge difference. Staying on top because you have a great product is one thing. Staying on top because you can quash others unfairly is another.

They missed the boat.

[Jartan]

In general the main cause is the bigger search engines are still not even trying to copy the big selling points of google.

Their front pages are still a big abortion of pictures and junk. Google is simple "box + logo".

Their results are trying to coppy google but the no.1 thing the google results page sells is TRUST. Most people trust google that all adds are going to be labeled clearly and they will not be inserted into the results!

MSN/Yahoo/etc already missed the boat on this issue. If anyone is going to compete with Google it's going to have to be someone new at this point probably. Unless of course someone thinks up a new must have feature.

Surprising Google hasn't spawned a new rennaisance

[erroneus]

I think among the tech crowd, it's a given that companies who cut back on R&D are simply shooting themselves in the foot. Google is all about R&D and trying out new ideas. The amoebic growth and success of everything "Google" should be more than just noticed by various companies... it should be mimicked.

Instead, we still see a whole lot of "heads in the sand" and people wondering why their previously successful business models are failing. But then again I can see where people are trying to demonstrate that they learned something from the dot-com failures too... but perhaps they didn't learn what they should have since a great deal of the mentality from the dot-com boom is present in Google's "just try it" ideology.

If you want to understand why Google is winning...

[Mark Programmer]

Compare http://www.google.com/ [google.com] to http://www.lycos.com./ [www.lycos.com] Google realized early on that to win in the searching business, all you need to do is search really well. As long as I still have to scroll my browser page to see everything on a search site's front page, that search site is too complicated. Having a simple main page lets users set it to their home page with negligible impact to their browser's startup time; that really matters more than some people think.

AltaVista got the message, but they're still playing catch-up.

Re:Firefox

[kamochan]

Google is the default also in Safari, the Mac OS X default browser. I guess we could generalize that "I bet there is a correlation between the switcing rate from IE to *[xX]" :-)

They appear to be addressing this piecemeal

[fizbin]

For example, if you search google for the name of any common drug (even if you use the generic chemical name instead of the brand name), you'll get options immediately to narrow your results to a bunch of useful subcategories; for example I just did a search and got this before the rest of the results: (I've inserted brackets to show where the links are, or just do the search yourself)

Refine results for Clonazepam:
[Drug uses] [Interactions] [For patients] [From medical authorities]
[Side effects] [Warnings/recalls] [For health professionals]

Now, if they could extend this kind of categorization to consumer electronic devices, I think that would address your main concern.

how much they've won

[SethJohnson]

In the past six months, I've noticed two computer newb friends of mine doing the same exact thing-- When provided a URL for a website, they don't know they can type it into the browser's URL field. Instead, they use their bookmark for google (it's also set as their home page) and then type the URL into the google search field. In most instances, Google returns a link to the URL they have just typed.

In the most recent instance, it didn't because it was a website I had just created for my friend. He told me on the phone that he couldn't find the website I had sent him the URL for. I knew the domain was propagated in DNS, so this sounded odd to me. Then when I visited him at his house, I saw him typing it into google instead of the browser's URL field and I had to explain that google didn't yet know about the website and that he needed to request it directly.

The other guy opens his browser, which has google set as his home page, then he types "www.hotmail.com" into the search field so he can check his email.

So, yeah, Google has established itself as a fundamental component of the internet for many, many people.

Seth

Re:Google is better than we think

[m0llusk]

That sounds like a kind of Silicon Valley syndrome. All of these things you say about cool stuff and the brightest minds have been true of the old Apple that nearly died, of Silicon Graphics which has finally died, of Sun which is floundering, and the list goes on. Data General anyone? WANG? On top of that they are now victims of their own success. Having acheived so much they are now in the position of being essentialy monopolistic thug capitalists. Other search engines and portals and ad services are not keeping up, so even without intention they become the New Microsoft. Google is a very strong company indeed, but in that market strength can ironically be a huge weakness and even the very strongest tend to last only so long before the entire market changes.

Re:Better search options

[kestasjk]

'digital camera -sale -site:.com'; just keep excluding results until you're left with what you want.

The only way to get accurate, personalized results from a flexible search engine is to learn at least some basic query syntax. I don't want digitalcameras.google.com, laptops.google.com, anythingyoucanthinkof.google.com to walk me through making common searches, because it's more effective, flexible and easier to just add on a few extra clauses yourself.

Re:Better search options

[kthejoker]

The ideal search engine would be just like Google, but have one more aspect: real-time fuzzy sliding algorithms.

These sliders could control all kinds of variables: whether you're shopping or not, whether you want local results or not, are you looking for more academic sites or more personal sites, more historical data or more current data, etc. And they need to work seamlessly and on the fly.

Google used to have something like this in their Labs, but I don't think they do anymore.

In any case, the grandparent's point is simple: query syntax is for chumps. Give me a search engine where I can do the same thing as query syntax, but without having to learn query syntax. Give me a search engine that can take what I search for, then ask me a quick question or two about what I want, then throw out the things that don't apply.

How hard would it be for Google to just keep a list of things people buy, and when you search for those, have a link at the top that says, "I am interested in buying digital cameras" or "I am not interested in buying digital cameras" and you click one and the results re-filter on the fly?

The answer: not hard. But Google is just a big ubiquitous monster of a search engine. They just don't try to help you out. Period. Someone else will, someday, and that'll be great. Google is great, too, but they're not the answer. Not yet.

Re:how much they've won

[martin-boundary]

If you think about your example, it's quite revealing for google's own vulnerability. Take Microsoft as an example. Right now they're still being eyed carefully because of the lingering antitrust issues, but once this passes they can aggressively make inroads by simply offering their own search box prominently enough that users think that's the box they have to type in their urls. Similarly, firefox is giving google an unfair advantage by having the search box prominently available, and even though the search engine can be changed, it's still google by default.

I would expect that those two issues - default google search bars in browsers, and Microsoft's as of yet refusal to make their own search bars extremely prominent - probably account for around half the search traffic/marketshare that google gets.