Economic Predictions Using Web Usage Data 149
Makarand writes "The Chicago Tribune has an article on the claims of ComScore Networks Inc., that it can predict major economic
trends by tracking the online activity of 1.5 Million people.
The company gains access to people's Internet travelogues by giving
them free security software and programs that speed up their connections. Economists say that
the company's models need to be tested over several years
before they can be considered accurate."
A cross section of society? (Score:4, Interesting)
Is it just me - or does that sound slightly worrying?
They claim to look at a cross section of society, but I'm willing to bet only the criminally insane would sign up knowing that they are perusing your credit card statements...
Spyware at its finest (Score:5, Interesting)
That's right: If you have their spyware installed on your computer, they are going to be looking through your credit card statements.
Why isn't this illegal yet?
Access to credit card stafements? (Score:3, Interesting)
It sends details from credit card statements!!? I wonder how many of the users of this thing are aware that it does this...
This sounds like spyware to me. 'Free security software and software to speed up their internet connection' sounds a bit vague about what this actual does apart from send confidential information to this company.
Is it just me? (Score:3, Interesting)
This actually sounds plausible... (Score:4, Interesting)
I'd have thought that if you could get a representative group of people of sufficient size, and allow for intrinsic skew in the data, then watching what they do online - what their ecommerce browse to conversion rates are, whether they're shopping at all, whether they're looking at holidays, cars, that kind of thing - could well provide a very good short-term predictor of where the economy is going next.
You could find out, for instance, that people were planning to buy new cars or go on a long-haul holiday weeks or months before that was converted to Real Money in the retailers' pockets, and upwards of three months before the quarterly reports from the companies themselves start to reflect the changes in the economic climate.
Sounds to me like this could be a really interesting toy to use as an adjunct to playing the markets
imagine what Google can do (Score:4, Interesting)
Danny.
Double Look. (Score:2, Interesting)
Directly from the article "As a second gauge of spending, ComScore also looks for trends in the credit card statements that about 30,000 of its panelists view online." and "ComScore gains access to people's Internet travels by giving them free security software and programs that speed up their Internet connections. With its capacity to download 18 billion Web page views annually, ComScore expects this year to capture 800 million Internet searches and 5 million online transactions." now after a quick serch of of the site ComScore Networks Inc [comscore.com] i couldn't find any reference to this free security software... So, is it just be or does it sound pretty fishy that this site looks at all your web queries, your online credit card statements, what you buy and dont, but isn't recognized by the company?....
Dare i scream invasion of privacy?
I'll forecast... (Score:2, Interesting)
I just got back from a week off and found 472 pieces of junk in my mailbox and web advertising as relentless as ever. Someone is paying, but perhaps fewer people, considering the attitudes of some friends, they can live without it all. I wonder which demographics then are more highly represented?
"Look ma, I got 18 more offers to make money at home and a penile enlargement and russian women are dying to meet me! Hyuk! Hyuk!"
only in theory (Score:3, Interesting)
Firstly, How can online activity ever be an active indicator of economic trends? Not everything done online is replicated at large in market in the real world. For example - I read news online, but dont buy any newspaper. I have browsed through catalogs of material online in amazon but havent bought much from them compared to what I would spend on totally different items in retail stores. The same applies for travel too.
Secondly, even if they do manage to get the software that tracks information on to peoples machine. How is this very different from online votes which almost always go with a disclaimer saying "The results represent only those who have been online on the site and is not scientifically valid" ?
They seem to have tons of predictions already [google.com]. Is it just me or does someone else see their common trend of predicting that online business is THE IN thing.
Very similar to Zipf's Law (Score:5, Interesting)
It's truly amazing. For example, it accurately predicts the populations of the 10 most populous cities, the number of appearances of the 10 most oft-used words on the entire Web, etc.
From a quick Google query: "Zipf's law, named after the Harvard linguistic professor George Kingsley Zipf (1902-1950), is the observation that frequency of occurrence of some event ( P ), as a function of the rank ( i) when the rank is determined by the above frequency of occurrence, is a power-law function Pi ~ 1/ia with the exponent a close to unity."
Here is some more information: http://linkage.rockefeller.edu/wli/zipf/ [rockefeller.edu]
Re:Sound like spyware to me. (Score:3, Interesting)
It would be scary if that ad was from ComScore, though.
I wonder what their software's license agreement will look like, and what "form" their software will be in.
Trojan, piggybacking, or something else? Will they actually plain-english tell us "This program is going to keep track of the websites you visit and purchases you make."?
Re:Spyware (Score:3, Interesting)
This capability is based on a massive cross-section of more than 1.5 million global Internet users who have given comScore explicit permission to confidentially capture their Web-wide browsing, buying and other transaction behavior, including offline purchasing.
Call me cynical but who wants to bet that the 'explicit permission' in question amounts to one of those Active-X popups we've all seen that say "Would you like to install and run Media Whore LLC Client (By Installing This You Agree to All Terms And Conditions of the License).exe?"
Actually, I've just done a bit of searching on the matter, and it turns out that this may not neccessarily be the case at all.. or not entirely at least. googling around indicates that Zone Alarm has (still does?) had comScore/MediaMetrix's 'value-added' software included with it in the past. No doubt plenty of additional dirt just waiting to be dug up by someone who's got more time to do so than I have right now..
Where's Free software when we need it? (Score:4, Interesting)
Speeding up an Internet connection is more-or-less a myth in the first place, you can't make software to cause a modem to go any faster than it goes physically. The only thing that really can be done is to make sure there's nothing stupid in the Windows registry slowing down the connection... and guess what, in older versions of Windows there is! Microsoft initially set the Maximum Transfer Unit (MTU) to something that made sense on a LAN connection, but caused an annoying number of retransmitted frames on a modem connection. Lower that number to something sane, and web pages will appear to the user to be faster. However, that didn't really speed up the modem, it's now just not wasting as many cycles on bad data. Changing the MTU number is a registry hack, the program needs to only be run once... no need for it to be there on every boot.
Another such program syncs your computer's clock with to official U.S. Government time. That's a cool and useful function, but it's really just using the Network Time Protocol (NTP) standard to contact government servers. Anybody who bothers to read the docs can write their own program to do that. Microsoft has even built NTP into Windows XP, although once-a-week updates isn't exactly enough for most users who care about their clock accuracy.
Another program hitches its ride offering the local thermometer reading from your local TV station's WeatherNet system in an icon in your system tray. Cool feature... but wait a second here. What if you don't live near a WeatherNet site? Oh, that's simple, it taps into the National Weather Service data to get you a report. But NWS's data is public, paid for by your tax dollars. The info is available on both FTP and HTTP servers that are absolutely free to access.
Open source projects could knock these "Download me!" programs out of existance. Why don't we?
They'll get blown away by Passport (Score:3, Interesting)
Consider marketing research firms that get paid boatloads of money to put people into demographic categories. Now, consider Microsoft's Passport initiative that tracks you online, where you surf, what you buy, where you live, work, and travel, and can infer all kinds of personal info like your domestic status. You are no longer part of a demographic group, you are a demographic; one of 200 million. How much would advertisers be willing to pay Microsoft for access to that database?
"How perfectly Goddam delightful it all is, to be sure." - Robert Crumb
Re:Spyware (Score:5, Interesting)
Except that in ComScore's case, #2 is "Provide that something for free if people agree to be tracked by spyware.". If this is the ComScore I'm familiar with (and it looks like it is), they're up-front about what they're doing. You get their package knowing that you're becoming the equivalent of a Nielsen family with every single page you view being tracked and recorded. I personally wouldn't agree to that, but I imagine there's a lot of people who wouldn't care (or who have another computer for the browsing they don't want recorded).
What they can do (Score:2, Interesting)
You can normalize for your sample by comparing your group against a known baseline. What is interesting to me is that the "normalized" groups like what Neilsen uses are tiny (3000 users) and thus have their own flaws (undercounting Goatse visits since their population of users is so tiny they're missing out on all sorts of behavior).
Here are some things you could do.
1) track e-trade, schwab, and yahoo finance to see what stocks people are checking - a good way to anticipate market changes/volatility
2) monitor site traffic for e-commerce sites to gauge how much business they are doing.
3) accurately measure how many click thoughs banner sites get (almost none)
etc.
--Pat / zippy@cs.brandeis.edu