People Trust Yahoo! and Google For the Brands

amigoro writes "Here's an interesting experiment: Copy Google results pages from four different e-commerce queries. Tell 32 test subjects who are going to evaluate the results that the results were from four different search engines: Google, MSN Live Search, Yahoo! and an in-house engine created for the study. Then see which ones they rate as the best. As it turns out Google and Yahoo! win hands down, proving that even on the Internet it's all about branding."

obvious

Well, google and yahoo are much like Cocacola and Pepsi... MSN is always giong to be the live.com though. Just doesnt taste right... kind of bitter.

Re:obvious

I'm sure this is completely unrelated, but Google did recently hire the hypnotoad as head of development in its search engine division.

Branding, or reputation?

One could easily give credit to the results of past performance. Yahoo was the #1 search engine for years because it delivered good results. Google became first because it delivered beter results, and generally still does. MS search got a bad reputation from stories about manipulating search results. And the "in house" engine has no past performance - who would trust that at all?

Confused experiment

The experiment is confused. Are the evaluators supposed to do their own research on the products, or respond with their trusted search engine? If the former, they shouldn't care what the raw data is from. If the latter, then it's not just branding, since each company is different and the point of trusting a company is that you accept their results without scrutiny (at least, with less); after all, what's the purpose of using their services if you feel you have to verify everything yourself anyway?

Control?

"Despite the results pages being identical in content and presentation, participants indicated that Yahoo! and Google outperformed MSN Live Search and the in-house search engine."

I don't understand, did people see the identical results and rate them differently? Or did they show Group A the google results of one query and the MSN results of another, and Group B vise versa?

Study Disproves Marketing, Proves Reputaion.

The overall results of this study are that performance and reputation trump marketing money. M$ spends close to a billion dollars a month on advertising, dwarfing the combined spending of the rest, but people clearly think their stuff is second rate. This is how a the market should work and it's encouraging. People are not nearly as dumb as M$ thinks they are.

Reputation is a legitimate decision factor in information services. It's right for people to think M$'s search engine is goofey when M$ is such a dishonest company, their results have been poor in the past and they admit to selling placement. It's also right for people to have a neutral or favorable view of Google and Yahoo given the performance record of both companies in search. The neat thing about search is that things that don't look like useful results often are, at least when you use a good engine.

There were a few problems of sample size. The group size is to small and the responses were too poor to mean much. Only 36% of the results were judged relevant, which means the results from all the engines were poor. A larger study may show a real relationship between performance and trust that goes beyond marketing.

I'd rather see

I'd rather see people search for several queries of their choosing from the same four engines and be delivered result pages free from branding and see which results they preferred. It'd be much more useful.

Re:I'd rather see

That'd just be a study to see which search engine gives the best results, which is a pretty normal boring done a million times thing.

They wanted to study what affect branding had on user perceptions - a completely different thing, also normal and studied slightly fewer times.

Re:I'd rather see

Useful to whom?

If the intended audience of the study are search engine power users who need to know about slight differences in algorithmic performance, then yes, your study would be more "useful."

However if the intended audience is the marketing and research divisions of major search engine players, then this study goes a lot farther in saying what gets market results than yours would. It answers the question: "do we spend $1 million improving our tech or do we spend it selling our brand?"

Of course it's hardly a secret that customer satisfaction has very little to do with the freely made up mind of the customer. Its why car companies go to such lengths to provide little perks to purchasers and leasers of new cars. A few tens of thousands of miles of free oil changes tends to make the average customer a lot happier with their car than actually buying a really good car does. What makes the average geek happy with one gadget he buys, uses for a week and then leaves in the drawer, and unhappy with another? Even with smart consumers its very rarely a hard honest brass tacks look at features, performance, and price.

Consumer trust

Branding is about consumer trust. If they trust your company to do something consistently well, they will place a good deal of faith in that (hopefully without becoming fanboys). As a result, whenever a product buying decision comes up, they are more likely to select the branded product.

A short introduction to branding [slideshare.net]

Branding can also work for open source. When people come to trust a "product," or piece of software like FireFox, they will keep using it until given a reason to do otherwise.

Was it the "brand" or past experience.

If I was told a tidbit of medical advice came from Dr. Dean Edell I'd have weighted that higher than if I'd been told it came from Dr. Laura. This would be based on having heard each of them dispense advice in the past.

Now if they asked people who had never used a search engine and they rated them based on the name, then I'd call it "branding". But if the people polled had past experience with one or more of the search engines, that direct experience is liable to have a much greater influence than name recognition only.

Isn't there a difference between branding and reputation?

A little overkill?

This is in line with the finding last year by German researchers who showed using MRI scans that well-known brands activate positive emotional responses in people's brains.

Surely the fact that people's preconceptions color their perceptions has been known for more than a year, and doesn't require looking directly into brains...?

Lies All Lies

Lies, all lies.

People use logic for decisions, NOT emotions. Geez everyone knows that

Seriously though, branding may seem a bit shallow and unscrupulous, but it certainly taps into how the brain works. We associate and generalize. Google good and MS bad, that type of thing. Then you throw in colors, images of happy people, etc, and you get a positive reaction. It may not be enough to surpress logical thinking, but these associations are powerful.

Too small of a pool

So let me get this straight. 1 search result, and brand it 4 different ways. Then you take 32 people, and show each all of the results. There's no way that's enough to overcome statistical errors, biases, vague worded questions, morons, etc.

And most importantly, you taint the results by showing each person all four!

"Do you like Result 1?"
"Sure"
"Do you like Result 2?"
(thinking) "That looks familiar" (said) "I guess."
"Do you like Result 3?"
(thinking) "Okay, now that has to be the same thing" (said) "I guess so, sure."
"Do you like Result 4?"
(thinking) "wtf?" (said) "Whatever."

Especially if you are measuring not by written or oral results, but by "brainwaves". Of course there's going to be different activity in the brain. The brain is seeing patterns and getting confused. Is that skew trust, or taint?

Wait A Second...

Hey, wait a second... im getting confused. If they wanted to see what brand people trust the most can they just look at usage logs (google being used more than other search engines). To me a more interesting experiment would be to do this all without showing and the logos to see what engine gives back the most useful results. I don't understand why they did the experiment....

I COMPLETELY disagree

I remember when google first came out. Yahoo, msn, altavista, hotbot and several other sites were stronger brands than google. Google's search page was clean and wasn't cluttered with tons of stuff. It returned lots of results and it had the least amount of spam compared to the others. Nobody knew what google was but once I found it I started telling people about it.

I would say the factors for internet site success are: results, interface & word of mouth.

The tricky part about the internet is that if any of those three things change or is bested by another competitor, the flow of traffic will change.

Internet customers are not loyal to brands. They go with the flow. If all of a sudden google became obsessively cluttered or slow response or cluttered with spam, then we'd be ready for something else. And which ever service seemed to step up to the plate then the flow would change to the new place. But since we're all comfortable with 'google' right now, if there's a competitor that is offering a similar service, we have no reason to move the flow of traffic. But if google started being annoying, there are NO loyalties.

Re:I COMPLETELY disagree

I remember when google first came out. Yahoo, msn, altavista, hotbot and several other sites were stronger brands than google. Google's search page was clean and wasn't cluttered with tons of stuff.

I heard about Google long before I used it. I stayed with AltaVista because I had learned their search syntax, so I knew how to do very targetted searches. What killed AltaVista for me was they somehow let their spidering get very out of date, and eventually, AltaVista just returned pages of broken links. Probably somewhere, some PHB, decided that money could be saved if they did less of this pesky "spidering the Internet" stuff which seemed unrelated to delivering adverts.

In other news...

Slashdot only puts the Google and Yahoo! icons by the summary, proving... oh I give up.

yahoo paid inclusion

This is very interesting. I too would generally rank Google and Yahoo as my top search engines in terms of "quality.

A few months ago, though, I became aware of Yahoo's "paid inclusion" program. Basically you pay Yahoo money, and your company is more likely to show up in the "organic" results area of the search. Note i'm not talking about the sponsored results at the top of the page or the ones on the right side, the "paid inclusion" results are indistinguishable from normal search results. Apparently this is a well known feature in the industry but personally i was surprised to find out about it.

Technically, the ranking algorithm doesn't weigh these sites higher. PI just assures that your site is crawled often and that it is "crawled" according to the page definition feed that you provide.

here's a link to the program:
http://searchmarketing.yahoo.com/srchsb/ssp.php [yahoo.com]

I ran into this whole concept because i was surprised that the company i work for came up at the top of certain searches even though we have a lot of competitors. Even though we ARE better and more popular than many of our competitors, i was still surprised that we were at the top on some pretty specific terms. It showed a keener understanding of our site than i would have expected a crawler to have. At first i was really psyched about Yahoo's technology, but then i found out that we use paid inclusion. :-)

I'm still undecided how i feel about this program. In my mind the main results are supposed to be purely based on site content/popularity , unlike the sponsored links. When you get an advantage by paying for access to their crawler... that's no longer the case. On the other hand, this isn't THAT different from other SEO techniques which are by definition mechanical ways that you can improve your standing. Only difference is that yahoo is directly involved in it.

Well

When the term 'googling' is used in place of 'searching' for the average PC user, I'de say that they have some market penetration.

nice experiment

crap conclusion.

I hate this type of research. It's pointless infinite recursion. It's anthropology with numbers, but at least real anthropologists are willing to accept that their field is purely subjective.

define: human
see human.

Not a valid double blind test

They said the trial subjects were only shown the test results, but they could have easily picked out Google and Yahoo anyway. They're the one's not mentioning the Tianamen Square Killings.

Hint for aspiring web search companies: The Wayback Machine is not your "Cache"

Beat This!

http://www.gahooyoogle.com/ [gahooyoogle.com]

Brand and logo

I like the Google brand, but I can't seem to get over the ugliness of the logo. Yahoo, on the other hand, has an appealing and very identifiable logo, but I can't stand using the search engine/directory/whatever the heck it is.

I wonder, how much does the asthetics of the logo affect the brand strength? Any thoughts?

The original research article (and why it's bogus)

Here's a link to the original article at the author's site that unlike the ACM digital library version is not locked up behind a subscription requirement: The Effect of Brand Awareness on the Evaluation of Search Engine Results [PDF] [psu.edu]

Here's a more understandable summary that in TFA: They got Google search results on four e-commerce topics, stripped identifying HTML, and created 16 fake results pages with branded header and footers for four search engines The query results for a topic had identical content and presentation. They presented each of 32 test subjects four results pages, one for each topic. Which of the four brands each was shown and in what order the topics were presented were randomized. Subjects looked at each result one at a time. They were asked to evaluate the results, following links as they chose to and commenting out loud. Note that all results sections were identical is both content and format, with only the branded page header and footers being different.

The authors claim that Yahoo got the highest ratings (averaged over all four topic queries), 15.3% more than the overall average, compared to Google's 0.7% over average. But their table and graphs show results all over the map. Google scored 52.2% over average on the home improvement query. MSN had the highest score on the camping mexico query, Yahoo was highest on techno music, and the made up unknown search engine got the highest score on the laser removal query. That says to me that when you have four search engines and four queries to mix together in random order you need a lot more than 32 subjects to get statistically meaningful results. The paper contains no analysis of statistical significance of the results.

32 test subjects, this is a joke

"proving that even on the Internet it's all about branding."

I would be more like to think that it proves that on the internet, any two-bit study of a few dozen people is given significance it doesn't deserve.

Slashdotted

Tsk. If only they had hosted with Google instead of whatever inferior brand they're using, they wouldn't have been slashdotted!

- RG>

Re:search all three

like dave said, it shows the results from the three big search engines, side by side... altho keep in mind it was a 48 hour side project of mine just, and the level of bugginess can very... windows live search for example has an ugly tendency to just not work at all with a lot of queries... :P... i've been meaning to take a serious stab at the project shortly tho when i get some free time :)

Re:Unexpectedly mindbogling results

Don't you mean mindBoogling®?