Browser Stats For The BBC Homepage

Lord_Scrumptious writes "An interesting article titled 'The software used to access the BBC homepage' has recently been published on a blog by a BBC employee. It's all about the different browsers and operating systems accessing the BBC's homepage. The analysis is from a week of page requests in September 2005. Not surprisingly, Internet Explorer accounted for 85% of site visits, but Firefox had a very respectable 9.7% share. Even requests from Sony's handheld PSP device were recorded, but interestingly there's no mention of mobile phone devices."

Finally....

Finally some reliable website records which arn't off some obscure coding page. :)

Re:Finally....

You make a good point, that cache config can affect the amount of traffic directly hitting your website, and therefore affects your logs.

However, given the headers returned by the BBC site, caches should NOT cache the HTML, as the headers say the content expires immediately:

Expires: Mon, 24 Oct 2005 11:57:59 GMT
Date: Mon, 24 Oct 2005 11:57:59 GMT
Content-Type: text/html
Server: Zeus/4.2
Cache-Control: max-age=0

So, the BBC figures may be more accurate than you think.

Mobile devices

There are specific editions [bbc.co.uk] for mobile devices. It's no wonder that they don't access the the front page directly.

Many people go to BBC, CNN and other major sites through their mobile service provider's front pages. These would naturally point to the dedicated mobile editions too.

errr

Linux (various distributions) 0.41%

Windows Vista 0.15%

  MSFT's unreleased os has nearly the same market share as linux?

We've got a long way to go.

BBC news, typically read at work

So it's probably about right for UK business desktop stats.

 

Re:errr

So what you are saying is that 1 in 11 people I walk up to on the street will be using Linux?

I think not.

Re:errr

Most linux people use a browser string to look like windows so sites wont reject them.

Errr... no. Most Linux users will use the default setting for their browser, which for most people will not identify them at using Windows or IE. Yes, a very small number of people will do this, but to claim that it's "most" is just laughable.

If we all set up some bots...

...with a shiny firefox user agent string - we could easy get that figure up to 30%!

Representative of Overall Market Share

As of September 2005 [e-janco.com], Internet Explorer has an 85% market share, while Firefox has a 9.5% market share.

The BBC's numbers are simply representative of this, as any large web site would be.

mobile devices

The BBC provide specific pages for mobile devices. The front page is way too big/rich for a limited handset.

Opera

My install of Opera is set to identify itself as IE to websites as I am sure many others set theirs the same way on install. So in that light, are those figures trustworthy?

Re:Opera

My install of Opera is set to identify itself as IE... are those figures trustworthy?

Yes, they are.
Old versions of Opera that identify themselves as IE by default use a user agent string like this:

Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686; en) Opera 8.02

So the "Opera" string is here and easily identifiable.

New versions should simply use the proper Opera UA string by default [slashdot.org].

If you use Opera I suggest to check that it sends the "correct" Opera UA string: the sky will (mostly) not fall down.

Re:Opera

To the nearest few percent they are trustworthy, even with your Opera install skewing the figures.

We need to remember that people who do unusual things with unusual browsers are an incredibly small fraction of all internet users. The message of the article is that there's very rougly a 8/1/1 split between IE, firefox and 'other'. That message is not affected in the slightest by Opera, lynx or any other niche browser.

Variability by site

Interesting. I wonder how much variation there is of browser use by other sites... I imagine BBC is higher than most in the Mozilla-bred catagory, as the BBC News site has posted lots of articles about Firefox over the years. I wonder how different it would be for msn.com, foxnews.com etc.

On a related note, I hosted some pictures on my website last week that were posted into a fark.com forum, 47.6% of fark readers seem to use Firefox (from some 14,000 hits in two days) - I bet slashdot beats this though!

Re:Variability by site

I imagine BBC is higher than most in the Mozilla-bred catagory, as the BBC News site has posted lots of articles about Firefox over the years.

I doubt it makes much difference. The BBC news site is read by a lot of Normal People who either couldn't care less about what browser they're using, or have no power to change it because it's a work computer.

I'm really surprised that firefox has such a high share. Of course there have been similar stats released by sites like i-am-a-1337-linux-doodz.com and windoxxors-is-teh-suxxors.com, but to get them from a mainstream site like the BBC must be very encouraging for the developers

Fatally Flawed

I visit the BBC web site multiple times a day, but I haven't been to the "main" page in months. I expect most regular Firefox visitors will have bookmarks or just type a URL that goes past the main page.

The author does point this out:
And I must stress again, these figures don't represent the breakdown of visitors to the BBC site as a whole, they are based on requests to the homepage alone, over the course of one week in September. Nevertheless I think they provide an interesting snapshot of web activity.

but it should have been avoided

Slashdot stats?`

Would be interesting too. Browser stats, OS stats ...

/.'d

Already.

Ah well, at least I had time to read the first page of the article, shame really as I was looking forward to that second page :(

Mirror

Well, I for one couldn't access that blog. Here's a mirror [mirrordot.org]...

How about Slashdot generating a mirror link via a neat little "mirror" icon next to the links?

Re:Mirror

Damn, disregard that post... As usual they've split it into pages...

Only nyud.net links [nyud.net] may help then, although my experiences with those aren't the best and why I tried to avoid it in the first case.

Slashdotted....

Hmmmmm.... Slashdotted already.

I have a hunch this guy's web stats are going to show a MASSIVE influx of FireFox users, then a long period of downtime...

And his second report...

Will be on the visits to his webpage, when the server recovers :o)

How about stats on slashdot readers

Does slashdot publish browser/system stats about slashdot readers ? That would make interesting reading.

It depends upon your site content

First here is mirrordot link [mirrordot.org], if you cannot open page (slashdoteffect).

My site and blog mostly related to Linux and Open source stuff, and here is my exprince so far:
OS
Most of the corporate users, uses Windows XP/2000 desktop
Individual user uses Linux/BSD/Mac OS desktop


Browser
Firefox rules
IE (6.x/5.x)

So it depend upon your site content, if you wanna see this stats they are here [cyberciti.biz]

No MSI build for Firefox - no mass deployment

at companies that run Windows clients. I wouldn't bother to install Firefox more of less by hand on hundreds of desktops myself. The Firefox guys should really get a MSI build ready for easy deployment _and_ update. Firefox is just not 100% enterprise ready like IE is with it's managabilty by group policies. I wonder how many people check bbc.co.uk from their workplace. They might even have Firefox installed on their home computer.

Re:No MSI build for Firefox - no mass deployment

There you go..
http://www.frontmotion.com/Firefox/ [frontmotion.com]
MSI installers for Mozilla Firefox! Useful for installing Firefox on a single computer for the home user or deploy across thousands of computers automatically with Microsoft's Active Directory. Use Firefox on your corporate computers to decrease virus incidents and increase overall security. Save time and frustration with our installer that is targeted toward the corporate IT administrator with manageability and upgradeability in mind. This is not just a wrapper around the exe installer nor is it another half baked 'captured' install.

Coral cache

http://www.currybet.net.nyud.net:8090/articles/use r_agents/index.php [nyud.net], but that seems to be /.'d also.

Super Respectable

"but Firefox had a very respectable 9.7% share."

I use firefox and even I can't keep a strait face reading that line. I mean have some self-worth, man. There's nothing respectable about that. Can't we aim just a tad higher here? Especially if we're gonna tag on the word "very"?

As always, defaults play a role

Remember, with every English-US installation of Firefox comes a preloaded RSS feed on the bookmark toolbar that points to the BBC for news (I say this as an avid Firefox user)

Re:As always, defaults play a role

Firefox comes a preloaded RSS feed ... that points to the BBC for news

Maybe so, but that's not the homepage, which is from where the stats were taken :-)

Firefox comes with a "Live Bookmark" to the BBC

One thing I noticed when I installed Firefox, is that it comes with just one live bookmark. It is called: "Latest Headlines", and pulls the feed from http://fxfeeds.mozilla.org/rss20.xml/ [mozilla.org] But, this feed is the same as the main stories feed at BBC. I would figure people would click on these and get some more exposure to the BBC site, more than usual. This has actually made myself more aware of those stories, and made me more likely to visit again.

I run a website with 20% Firefox usage as of now!

I run a website, a webmagazine in Sweden. It started out as a music/lifestyle webmag, and is now more of a collection of blogs, mostly about music and related things (sports, debate, feminism, lifestyle, TV). In other words, the visitors are not at all tech type guys, but it's definately an inner city, trendy type of crowd. I would not hesitate to call them early adopters. Nonetheless, I was amazed when I checked the browser stats for October after reading this article. WE HAVE 20% FIREFOX VISITORS! Please see below the figures for the top 5 browsers (not counting the mysterious "Unknown" browser, which is mostly RSS aggregators and its ilk):

(Sorry for the bad formatting, why can't Slashcode support the pre-tag?)

Month IE Firefox Safari Mozilla Netscape
May 64.2 16.7 5.3 0* 1.3
June 64.7 17.9 5.4 0* 1
July 65.9 15.6 4.6 3.8 2.1
Aug 66.8 16.6 5 2.5 1.6
Sep 62.9 19.2 5.9 3.2 1.7
Oct 59 20 5.8 3.9 2.4

* Before this date, Firefox & other Mozilla were lumped together.

Thecounter.com has stats as well.

Thecounter.com gets like 600 million hits a month since they make software for many different web sites, here's a link to their browser stats for sept. 2005:

http://www.thecounter.com/stats/2005/Sept ember/browser.php

1. MSIE 6.x    62223734 (83%)
2. FireFox       5806423 (8%)
3. MSIE 5.x      3170911 (4%)
4. Safari        1311540 (2%)
...

Here's a link to their OS stats for the same month:

http://www.thecounter.com/stats/2005/Sep tember/os.php

1. Windows XP    54945908 (73%)
2. Win 2000       8468339 (11%)
3. Win 98          6806316 (9%)
4. Mac             2317188 (3%)
5. Unknown         1132090 (1%)
6. Win NT           473897 (0%)
7. Linux            322362 (0%)
...

Probably more accurate since they count more hits on a wide variety of different types of web sites.

Obvious solution

The obvious solution is to make the BBC homepage the default homepage for Firefox!

Let IE and Firefox battle it out

I'm all for a diverse browser ecology, it only makes exploit writers' lifes more difficult. I, for one, am a most pleased Opera customer/user and while I hope that Opera the company will stay in business and refine their browser for a long time to come, I also kind of hope that Opera the browser remains in its 5% niche where it attracts no major attention from mentioned exploit writers.

Small AuPair website in the UK

I'm the webdesigner for a small AuPair company in the UK, our demographic is entirely UK Families and young foreign nationals.

For this month, this is the breakdown of browser access

5250 Views this month:
* 77.5% Internet Explorer (inc. Maxthon & AOL) = 4070 Views
    v5 (57 views) v5.5 (27 views) v6 (3703 views)
* 10.9% Mozilla Firefox (inc. Netscape & SeaMonkey) = 574 Views
* 02.3% Apple Safari (inc. Linux Konqueror) = 122 Views
* 00.4% Opera Browser = 22 Views
* 08.8% Other (Unknown, bots and rare browsers) = 462 Views

Even with this incredibly Windows/IE centric demographic (almost all being "regular" people), I'm very pleased to see a 10% Firefox Usage. The site only counts 1 view per IP per 24 Hours and ignores views from my IP and the Company's business address IP.

Microsofts true competitor...

The market share of Win98 is bigger then Apple + Linux. That is a ten year old OS pretending to be a seven year old OS. And they say Windows is not stable?

Most visited site in the UK

I've read the bbc news website is the most visited website in the UK, so it's probably the best indication of what UK people use to browse the web. I wonder how many of the IE stats are Opera however.

User Agent Switcher

Thanks to the awesomeness of the Firefox extension model, you can dynamically switch your user agent to whatever the hell you like.

Sploooooooooooo!

Re:User Agent Switcher

Yeah, switch to "googlebot" and have free access to all these pay-for-registration sites.

Default?

Remember when you first install firefox, there was this shing RSS bookmark with 'Latest Headlines' and pointing to the BBC news pages?

Anyone considered that, maybe, that might have influenced the results? Having a default bookmark as the page of the study? You wouldnt take browser results from MSN.com or whatever IE's default home page is.

Nevermind me though, I just suggested that a pro-Firefox poll might be biased. Karma be dammed!

I donno about this...

I run a web hosting business and here is the breakdown for webstats last month, and yes its linux hosting btw:

Windows 16400 70 %
Linux 5497 23.4 %

MS Internet Explorer No 14012 59.8 %
Firefox No 7579 32.3 %

Though windows is the dominator in this respect but it goes to show the website content does really depend on who visits the site and therefore produces the stats.

Lets face it BBC is a news network business people and general interest users are reading these articals; ofcourse nerds and geeks are but on the scale of things we really only make up that 9% :)

That seems about right

The BBC stats aren't that far off from the stats for my planning-related Web site. Considering how many people visit the site from their workplaces, I'm really surprised to see a somewhat higher percentage of alternative browsers than the Beeb.

MS IE 81.9 %
Firefox 10.4 %
Safari 2.2 %
Netscape 1.6 %
Konqueror 1.1 %
Mozilla 1 %
Unknown 0.8 %
Opera 0.6 %
Camino >0.1%
I-Mode phone >0.1%
Others >0.1%

older stats

I used to be about 60% Netscape 35% IE 5% misc like lynx etc, when I started in 1998.
Then it got to about 95% IE, so 85% is quick a marked drop in IE support.

Do remember that the BBC is hardly a generic site for your average Internet user, it attracts a significant quantity of beginners and is dull for anyone technical (there are a higher proportion of technical users on the Internet than you'd meet on a street). So these stats are quite good.

I know the way they are worked out should be quite fair.

Those figures aren't influenced by ff's rss

IMHO Those figures aren't influenced by Firefox's 'Latest Headlines' because that RSS feed points to article pages, not the homepage, these results were from the homepage therefore anyone that says that FF's 'Latest Headlines' unfairly influences the results should RTFA!

FWIW Westminster Abbey's stats are (taken 16-10-05 to 22-10-05):
IE Variants: 65.9%
Netscape variants (mostly Firefox and other Geckos): 24.9%

This is compared to 6months ago (taken Jan 2005 - Feb 2005)
IE vars: 69.4%
Gecko vars: 21.1%

And finally last year (taken May 2004 - June 2004)
IE vars: 80.9%
Gecko vars: 11%

Our results do tend to suggest that Firefox has gained a significant market share from Microsoft's browsers.

Ipod Studio

I'm the webmaster for http://ipastudio.com/ [ipastudio.com] and out of the 10,000 visitors we have a day, 50% of them or more use Mozilla/Firefox.

9.7% isn't that un-usual

I work for a restaurant search engine company (www.dine.to) in Toronto, Canada, which is aimed about as far from web-hacker as possible. Firefox gets about 9.5% of the visitors. Like the story says, not a massive amount, but still respectable.

Competition Commission Now!!

In 2000, a Competition Commission found that Interbrew's (now Inbev) acquisition of Bass Brewers was posing competition concers for the British beer market. Interbew would have a combined 32% of the British beer market!! ONLY 32% and they put up a competition commission!! Why not investigate M$ monopoly??

Wait for October for movile phones

"but interestingly there's no mention of mobile phone devices"

Wait for October's stats, I used my Nokia 6630 to access the BBC's News site only the other day and very impressive their 'mobile' version of their site is too. The Beeb should get an award for a great attempt at accessibility and browser compatability.

legacy OSs

My personal web site gets on average one or two visits a month from systems identifying themselves as running CP/M. I always wonder whether there really are people still running CP/M and surfing the web or whether these are just joking misidentifications.

Linux More Popular Than Windows 95

And it only took 10 years!

Spider analysis missing

I used to analyze corporate web metrics and my comment about this article is that a bit more information should have been provided about the methods that he used to filter spiders and other automated retrieval systems, especially if you are going to make your own claims about browser/operating system share.

I have seen as much as 30% of activity on a single page be due to some some sort of automated retrieval.

It looks like he paid quite a bit of attention of the wide variety of user agent strings (kudos!) and I'm guessing that much or all of his filtering was based on his analysis of these strings.

Unfortunately, this is not enough. Many spiders identify themselves as some Windows/Internet Explorer variation without any indicator that they are a spider, business intelligence application, or other non-eyeball-on-the-page type browser.

One lo-fi suggestion for folks who want to check their current spider filter setup is to create a temporary honeypot link on one of their pages. An invisible hyperlink to a non-production page using a 1x1px transparent gif is enough. Then run a metrics report on that non-production page and include the IP addresses (watch for entire subnets of crawlers) in your production filters.

Also, I assume that whenever the author says "requests" he means "page views." It is good to make that specific distinction and focus solely on page views only for any kind of metrics reporting that is about eyeballs rather than server load. It provides for apples-apples comparison between browsers that might be more conservative at caching and re-using images, etc.

cheers

Other browser ID's

I'm using Opera, and have the default set to identify itself as I.E. Certainly this messes with the count. Granted, Opera probably doesn't get as much press as others, but I do know Konqueror can be set to identify itself as another browser, Mozilla among them. In other words, I find this statistic useless because it's not *really* able to identify what browser is being used, only what browser the user says is being used.

ugly

*comic book guy voice* ugliest website ever

Re:Sampling?

He had over 33,000 different user agents to sort through - why don't you email him and offer to trawl through the 22,000 UAs making up the 5% of traffic that he didn't generate stats for.

All I could get of the article (page 1 and 2)

It started with a casual enquiry from a colleague - "I wonder how many Firefox users visit the BBC homepage?" - and before I knew it I was involved in a lengthy statistical analysis of the browsers and operating systems that request the BBC homepage at http://www.bbc.co.uk/ [bbc.co.uk]

Our old stats reporting tool at the BBC gives a breakdown of requests from different user agent strings, which is where the browsers and operating systems people use to navigate around the web leave their digital fingerprints. It is about to be phased out in favour of a new solution, but I'm not sure that the new system gives the same granularity of data, so once I'd started, I thought I'd look at the figures in some detail before the old system gives up the ghost.

Now if you've never looked at user agent strings, they are rather dull and geeky, and full of lots of technical gubbins like these examples:

* Mozilla/5.0 (Windows; U; Windows NT 5.2; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6
* Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/85.7 (KHTML, like Gecko) Safari/85.5
* Mozilla/4.0 (compatible; MSIE 6.0; America Online Browser 1.1; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
* Mozilla/4.0 NETIKUS.NET GetHttp v1.0
* Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.5.1.0)
* Mozilla/4.0 (compatible; MSIE 6.0; Windows CE)
* Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 Firebird/0.7
* Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461; BT [build 60A])

There are of course some caveats around the figures I'm about to talk about.

User agent strings aren't an exact science. Or rather, they ought to be, but in the real world the come out a right mess. I've done my best to untangle them, but I still ended up with a significant number of user agents that I could not identify properly. And that is before we get started on the corporate networks that use the UA string to broadcast their corporate branding to the world whilst masking their operating system. Or requests claiming to come from both Internet Explorer 6 and Internet Explorer 5.5. Or that claim to be from a particular Linux distribution and Windows 98 at the same time. Or the plain weird like the inadvisably named KummClient from Hungary that proudly proclaims 'Linux rulez' to anyone like me dull enough to be delving through their logfiles.

User agent statistics on something as big as the BBC homepage could almost be the very definition of the long tail. The most popular user agent string - IE6 on Windows XP - clocked up nearly 6 million requests. I only counted user agents that had made more than 50 requests, but between 6 million and 50 requests there were nearly 11,000 different user agents to look at. Examining that number of requests accounted for 95% of the reported traffic, but only around 1/3 of the stats report. I initially suspected that counting the whole of the tail was likely to increase the market share I derived for the quirkier set-ups, but a random sample showed that a large proportion of the tail consisted of the most popular browsers and operating systems, but with different installed toolbars or corporate network messages that distinguished them as a unique string.

And I must stress again, these figures don't represent the breakdown of visitors to the BBC site as a whole, they are based on requests to the homepage alone, over the course of one week in September. Nevertheless I think they provide an interesting snapshot of web activity.

In total I've examined around 32 million requests to the BBC servers - although some of these have been discounted as 'unknowns' and some originate from crawlers and spiders.

The complete dominance of Windows XP and Internet Explor

... and this is page 3

Windows Operating System Share

Concentrating on just Windows alone we can see that Microsoft have done a very thorough job of converting their user base to the most recent iteration of the software. Windows XP accounts for just under 70.5% of the Windows requests, and Windows 2000 a further 17.4%. That means in total around 88% of users of Microsoft Operating System products are using the two most recent consumer releases.

Windows 98 features in 7% of requests made from a computer running a version of Windows, and after that the figures are very small in terms of market share. In fact the next largest figures is a clump of 'Windows other' including Windows CE, and various unspecific Windows NT user-agents that I couldn't pin down to a precise version.

Operating System Share of Windows Requests to the BBC Homepage
Windows XP - - - 70.5%
Windows 2000 - - 17.4%
Windows 98 - - - 6.99%
Windows (Various versions including CE, 3.1 and ambiguous UAs) - 2.23%
Windows NT - - - 1.90%
Windows NT5.2 -- 0.63%
Windows 95 - - - 0.21%
Windows Vista -- 0.16%

Chart illustrating the version share of visits to the BBC homepage using Windows software

Mac Operating System Share

I was frustrated in my attempts to similarly breakdown the different versions of the Mac OS that people were using to request the BBC homepage. I established that from the requests we saw I could identify Panther as supplying 31%, Tiger supplying 21%, with Jaguar lagging behind at 3%. However there were 41% of requests where I could identify that the computer was a Mac, but not the specific version. That is because Safari helpfully supplies in the user agent string the WebKit build, allowing the precise version of the OS to be identified, but most other browsers do not.

Linux Requests To The BBC Homepage

The number of Linux requests to the BBC homepage was very small, representing only 0.41% - less than 100,000 - of the 32 million requests included in this study. With such a comparatively low number I didn't take the time to delve into which different distributions were driving the requests.

The figures may, however, mask a slightly higher use of Linux. Since the user agents generated are more likely to be unique, they are more likely to have fallen into the statistical long tail. However I should add that my random samples of the tail did not show that it consisted entirely of Linux, in fact as I mentioned earlier, a lot of corporate-branded Windows networks show up in the tail.

Legacy OS Systems

We have some fairly strict standards for supporting legacy technology at the BBC on the client-side - but the long tail of older OS software visiting the BBC homepage is amazing. We still saw over 300 requests for the BBC homepage coming from machines claiming to be running Windows 3.1, and around 200 requests from machines claiming to be persevering with 0S/2 Warp.

Re:5% not a good penetration for windows, non-ie

Have you assumed that everyone not using windows is using Firefox? I don't think so.

Re:POINT OF THESE SURVEYS?

You could just, y'know, not read it. Instead of reading it, then taking the time to post a comment about it.

Re:LATE BREAKING NEWS!!!

Is the BBC homepage supposed to reflect some important or signifigant user base?

Yes. It is probably the broadest cross section of mostly British web users you are likely to find on a single site.

The fact that nearly 10% of those users use firefox is particularly relevant, and is a good weapon for those of us who do commercial web design to persuade our clients that the extra work to support alternative browsers properly *is* worth it.

Firewalls Don't Browse Much

I was surprised not to see BSD listed at all, and *somebody* had to make the "BSD is Dying" joke. But I suspect there are several things happening that account for its invisibility:

• Maybe it's lying about what OS it really is? Not an uncommon thing for browsers to do, as well as lying about what browser type they are.
• Lots of BSD use, especially OpenBSD, is for appliances like firewalls that aren't running browsers very often.