Slashdot Log In
Economic Predictions Using Web Usage Data
Posted by
Hemos
on Mon Dec 02, 2002 08:00 AM
from the it's-the-super-nielsen-ratings dept.
from the it's-the-super-nielsen-ratings dept.
Makarand writes "The Chicago Tribune has an article on the claims of ComScore Networks Inc., that it can predict major economic
trends by tracking the online activity of 1.5 Million people.
The company gains access to people's Internet travelogues by giving
them free security software and programs that speed up their connections. Economists say that
the company's models need to be tested over several years
before they can be considered accurate."
This discussion has been archived.
No new comments can be posted.
Economic Predictions Using Web Usage Data
|
Log In/Create an Account
| Top
| 149 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Registration required Complimentry copy below (Score:5, Informative)
By Rob Kaiser
Tribune staff reporter
Published November 30, 2002
Investors are always scavenging for data that could indicate the market's direction. Changes in everything from cardboard box orders to hemline lengths have led to stock market bets.
Now upstart ComScore Networks Inc. is claiming it can predict major economic trends by tracking the online activity of 1.5 million people.
While using Internet data to gauge the entire economy remains unproven--and economists are skeptical--the possibility highlights the Internet's unique ability to capture how people spend their time and money.
"It's a heck of a lot easier to watch somebody's online behavior than to follow everyone around in their daily lives," said Brian Wesbury, chief economist with Griffin, Kubik, Stephens & Thompson Inc. in Chicago. "So the more things we do online, the easier it is to track our behavior."
ComScore gains access to people's Internet travels by giving them free security software and programs that speed up their Internet connections. With its capacity to download 18 billion Web page views annually, ComScore expects this year to capture 800 million Internet searches and 5 million online transactions.
The question facing the 3-year-old company is how to use all this data.
So far, ComScore has gone in several directions, publishing rankings of the most-visited Web sites, tracking the success of online marketing campaigns and predicting the results of e-commerce companies such as Amazon.com prior to the companies' earnings reports.
Now the company is launching its boldest initiative, betting it can extrapolate what is happening online to the offline world. ComScore says it can determine spending, employment, automobile sales and other economic measures by comparing prior government data to levels of Internet spending and traffic on certain sites during the same period.
"I've been in the research business for a long time, but this is blowing my mind," said Gian Fulgoni, ComScore's chairman. Fulgoni, formerly the chief executive of market research firm Information Resources Inc., is based in ComScore's Chicago office. The company is officially based in Reston, Va., where its president is located.
To estimate employment levels, ComScore looks at visits among the people it tracks to more than 1,000 sites with job listings. It estimates how many of those visitors are unemployed by looking at whether the searches are being conducted at home during normal work hours and how often they visit the sites.
Research tool
"Consumers use the Internet today more than any other medium to research important decisions," Fulgoni said.
ComScore tries to predict the government's overall retail spending figure by looking at online buying activity.
"It mirrors it enough that you can predict if spending is going to be strong or weak in a month," Fulgoni said. "I'm not saying it's a perfect correlation."
As a second gauge of spending, ComScore also looks for trends in the credit card statements that about 30,000 of its panelists view online.
ComScore charges $50,000 for an annual subscription to the economic data. So far, Fulgoni says, a few customers have signed on to receive the information.
The company is also selling its data to an upstart hedge fund for a reduced price in return for a percentage of the fund's gains.
David Nuelle, a founder of the Arcanum Fund, said he will use ComScore's data to make investment decisions in e-commerce companies and offline firms, such as Southwest Airlines, where customers often place orders via the Internet.
"You can get a strong sense of the revenues" of companies that do business on the Internet, Nuelle said. "It'll be the strongest data point we will look at."
ComScore, which has 200 employees, has enjoyed some success at predicting the results of e-commerce companies.
Last month, the company estimated that Amazon.com would report third-quarter sales of between $839 million and $851 million, exceeding analysts' consensus estimates of $807 million. Two weeks later Amazon.com announced third-quarter revenue of $851.3 million.
Still, predicting the results of individual e-commerce companies is a far cry from being able to provide a new window to the direction of the entire economy.
"To make money off this thing you have to be better than the Blue Chip consensus," said Anil Kashyap, an economics professor at the University of Chicago Graduate School of Business. "There's a long way between saying we can predict and making money."
Testing needed
Kashyap and other economists said the company's models need to be tested over several years before they can be considered accurate. Also, the company will have to learn how to account for a general increase in Internet use and sales as well as seasonal factors, they said.
A more proven area of ComScore's business is showing companies whether their online advertising is sparking offline sales. ComScore gathers grocery store scanning data for 60,000 of the people it is tracking to watch their buying habits.
Nestle Purina Pet Care has used this service to determine if its Web site and online ads are prompting additional sales.
"We don't have that closeness of data with any other medium," said Michael Moore, director of Purina Interactive in St. Louis.
Copyright © 2002, Chicago Tribune
Re:Registration required Complimentry copy below (Score:5, Insightful)
They could of course mine the net for data on past years and produce 'predictions' for those years to compare with actual data on those years and deduce the ratio from there.
Spyware (Score:4, Insightful)
I always thought that anyone who provides programs to "Speed up your internet connection" were crooked and could not be trusted. Spyware at its finest. As far as this company providing "security software"...I won't even go there.
Re:Spyware (Score:5, Funny)
-Mark
Re:Spyware (Score:5, Funny)
Sound like spyware to me. (Score:4, Insightful)
Tracking the behaviour of 1.5 million people. And all these people are aware they are being tracked? And they did agree?
I can't believe it...
PS. Watch out! You computer has an IP address...
Re:Sound like spyware to me. (Score:5, Funny)
Irony? (Score:5, Funny)
P.
security company? (Score:4, Insightful)
This would make the world a better place, even if it could not be used to forcast the next great depresion.
It begins. (Score:3, Insightful)
but that's just my 2 cents.
Income-bracket/age-range skewed? (Score:5, Insightful)
Given that the equipment you need to access the web is still fairly pricy, and also that the majority of people accessing it are still relatively young, I'd question the ability of this model to extrapolate to the wider world.
Cheers,
Ian
Re:Income-bracket/age-range skewed? (Score:5, Insightful)
Also, the sample will only include people stupid enough to download their spyware and let it report back on everything they do online.
representative group? (Score:4, Insightful)
Maybe the researchers should see the world is bigger than the US
In other news... (Score:4, Funny)
ComScore announces their predictions for this year based on web activity. You should take all of your money out of large cap stocks and invest heavily into p0rn!
A cross section of society? (Score:4, Interesting)
Is it just me - or does that sound slightly worrying?
They claim to look at a cross section of society, but I'm willing to bet only the criminally insane would sign up knowing that they are perusing your credit card statements...
Spyware at its finest (Score:5, Interesting)
That's right: If you have their spyware installed on your computer, they are going to be looking through your credit card statements.
Why isn't this illegal yet?
Re:Spyware at its finest (Score:4, Insightful)
Economic Predictions Using Slashdot +5 Rated Posts (Score:2, Funny)
Access to credit card stafements? (Score:3, Interesting)
It sends details from credit card statements!!? I wonder how many of the users of this thing are aware that it does this...
This sounds like spyware to me. 'Free security software and software to speed up their internet connection' sounds a bit vague about what this actual does apart from send confidential information to this company.
and how does... (Score:2, Funny)
I guess you could predicate higher economical activity followed by a brief period of laxed activity, cycling every 20 minutes between 10pm and 1 am...
So how is this useful again?
My economy prediction (Score:5, Funny)
Is it just me? (Score:3, Interesting)
This actually sounds plausible... (Score:4, Interesting)
I'd have thought that if you could get a representative group of people of sufficient size, and allow for intrinsic skew in the data, then watching what they do online - what their ecommerce browse to conversion rates are, whether they're shopping at all, whether they're looking at holidays, cars, that kind of thing - could well provide a very good short-term predictor of where the economy is going next.
You could find out, for instance, that people were planning to buy new cars or go on a long-haul holiday weeks or months before that was converted to Real Money in the retailers' pockets, and upwards of three months before the quarterly reports from the companies themselves start to reflect the changes in the economic climate.
Sounds to me like this could be a really interesting toy to use as an adjunct to playing the markets
Discretionary income (Score:2, Insightful)
It has also been my observation that most spending on-line is from discretionary funds, so this tends to skew the results as well.
Finally, it does take into account the type of information being accepted by their target audience. Those who get their information primarily from internet sources, deal with a different set of information than those who rely primarily on TV/newspapers, and will therefore make different buying decisions.
imagine what Google can do (Score:4, Interesting)
Danny.
What About Credit/Debit Card Co's Right Now (Score:5, Insightful)
Double Look. (Score:2, Interesting)
Directly from the article "As a second gauge of spending, ComScore also looks for trends in the credit card statements that about 30,000 of its panelists view online." and "ComScore gains access to people's Internet travels by giving them free security software and programs that speed up their Internet connections. With its capacity to download 18 billion Web page views annually, ComScore expects this year to capture 800 million Internet searches and 5 million online transactions." now after a quick serch of of the site ComScore Networks Inc [comscore.com] i couldn't find any reference to this free security software... So, is it just be or does it sound pretty fishy that this site looks at all your web queries, your online credit card statements, what you buy and dont, but isn't recognized by the company?....
Dare i scream invasion of privacy?
I'll forecast... (Score:2, Interesting)
I just got back from a week off and found 472 pieces of junk in my mailbox and web advertising as relentless as ever. Someone is paying, but perhaps fewer people, considering the attitudes of some friends, they can live without it all. I wonder which demographics then are more highly represented?
"Look ma, I got 18 more offers to make money at home and a penile enlargement and russian women are dying to meet me! Hyuk! Hyuk!"
analyzing porn viewing... (Score:5, Funny)
Teen porn would indicate a desire to return back to school for more education. This can be used to indicate a slowing in the job market or radical changes in job skills being required.
Lesbian porn indicates a desire for more social time, expanding ones horizons, and generally a good economy, since everyone is getting more then enough of the good stuff.
Hardcore porn would indicate a slowing economy, since you are just pounding away at the task at hand.
Gay porn would indicate a resession, since that is most likely when you are taking it in the ass at work, so why not see how the professionals do it.
Hope this helps with future economical models based off of the viewing habbits of porn.
only in theory (Score:3, Interesting)
Firstly, How can online activity ever be an active indicator of economic trends? Not everything done online is replicated at large in market in the real world. For example - I read news online, but dont buy any newspaper. I have browsed through catalogs of material online in amazon but havent bought much from them compared to what I would spend on totally different items in retail stores. The same applies for travel too.
Secondly, even if they do manage to get the software that tracks information on to peoples machine. How is this very different from online votes which almost always go with a disclaimer saying "The results represent only those who have been online on the site and is not scientifically valid" ?
They seem to have tons of predictions already [google.com]. Is it just me or does someone else see their common trend of predicting that online business is THE IN thing.
Very similar to Zipf's Law (Score:5, Interesting)
It's truly amazing. For example, it accurately predicts the populations of the 10 most populous cities, the number of appearances of the 10 most oft-used words on the entire Web, etc.
From a quick Google query: "Zipf's law, named after the Harvard linguistic professor George Kingsley Zipf (1902-1950), is the observation that frequency of occurrence of some event ( P ), as a function of the rank ( i) when the rank is determined by the above frequency of occurrence, is a power-law function Pi ~ 1/ia with the exponent a close to unity."
Here is some more information: http://linkage.rockefeller.edu/wli/zipf/ [rockefeller.edu]
psychohistory (Score:1)
Economics (Score:1)
Funny, they're making Economics out to be a bona fide science.
Reminds me of... (Score:2)
I guess ComScore makes use of some similar philosophy... :)
Efficient spyware (Score:5, Informative)
Reading their privacy statement [e-trends.net] it should be noted that they are an incredible security risk and this company should be treated with the contempt that they deserve, the information they take is everything from emaills to SSL traffic and should put a chill through anyone.
What information is collected?
During the initial registration process, we request certain information, such as name, address, e-mail address, and education, about you and other persons who live with you or have the same mailing address as you (collectively, we call this your "Household"). After you register, our Network then collects additional information about your Household's Internet behavior and that of any other computers used by members of your Household that you have configured to use the e-Trends service. This information is then combined with other e-Trends member data and other information to create an aggregate view of Internet e-commerce. e-Trends monitors your surfing, essentially logging information about the web pages that you visit and the actions that you take, such as the purchases and transactions you make. e-Trends can only monitor the Internet behavior and activity of your Household's registered and configured computers. As a member, you therefore control which computers the e-Trends service is available on. e-Trends monitors both the normal web browsing you perform, and also the activity you may have through secure sessions, such as when filling a shopping basket or filling out an application form. e-Trend's proprietary and patent pending technology allows us to see the details of secure pages while protecting such content from parties other than the site to which you are connected. We monitor these connections so we can accurately and anonymously model not only the browsing habits of Internet users, but also their shopping, registration, and other interactions as well. Although we generally monitor your Internet behavior as part of this service, e-Trends does not examine, use nor keep any instant messages or examine or use the contents of any of your e-mail messages, except to perform specific functions necessary to provide you the e-Trends service (such as scanning your e-mails to effectively search for viruses), and as a quality assurance check against and method for verifying information on the surfing and buying behavior of e-Trends members.
Quite simply they read all your internet traffic
These companies should be illegal and the quicker someone sues them to oblivion the better,
but i see that handily Comscore have this e-trends as a subsidiary company just in case someone does that it wont affect the parent company.
buyer beware
Re:Efficient spyware (Score:4, Informative)
"Although we generally monitor your Internet behavior as part of this service, e-Trends does not examine, use nor keep any instant messages or examine or use the contents of any of your e-mail messages, except to perform specific functions necessary to provide you the e-Trends service (such as scanning your e-mails to effectively search for viruses), and as a quality assurance check against and method for verifying information on the surfing and buying behavior of e-Trends members."
Somehow these marketers have pulled off the biggest scam of all, to finally get people to agree to simply hand over their lives, SSN numbers and all! I hope someone gets caught in an identity theft crisis and discovers it was because this company sold their private and financial lives to a third party. Really, some hood could simply round up 50K and have the info delivered to them. As usual I'm sure 99% of the 1.5 mill have never read this statement and they are probably the same type of user who will click on an email attachment every time - and thus believe it is very nice of this company to provide "security" for them. They even fooled the reporter!
redundant comment berating old news (Score:1)
Spyware? (Score:2)
This makes me laugh so much. They give you 'free security software' to prevent your computer and privacy from being compromised in exchange for compromising your privacy.... The paradox is incredible
Where's Free software when we need it? (Score:4, Interesting)
Speeding up an Internet connection is more-or-less a myth in the first place, you can't make software to cause a modem to go any faster than it goes physically. The only thing that really can be done is to make sure there's nothing stupid in the Windows registry slowing down the connection... and guess what, in older versions of Windows there is! Microsoft initially set the Maximum Transfer Unit (MTU) to something that made sense on a LAN connection, but caused an annoying number of retransmitted frames on a modem connection. Lower that number to something sane, and web pages will appear to the user to be faster. However, that didn't really speed up the modem, it's now just not wasting as many cycles on bad data. Changing the MTU number is a registry hack, the program needs to only be run once... no need for it to be there on every boot.
Another such program syncs your computer's clock with to official U.S. Government time. That's a cool and useful function, but it's really just using the Network Time Protocol (NTP) standard to contact government servers. Anybody who bothers to read the docs can write their own program to do that. Microsoft has even built NTP into Windows XP, although once-a-week updates isn't exactly enough for most users who care about their clock accuracy.
Another program hitches its ride offering the local thermometer reading from your local TV station's WeatherNet system in an icon in your system tray. Cool feature... but wait a second here. What if you don't live near a WeatherNet site? Oh, that's simple, it taps into the National Weather Service data to get you a report. But NWS's data is public, paid for by your tax dollars. The info is available on both FTP and HTTP servers that are absolutely free to access.
Open source projects could knock these "Download me!" programs out of existance. Why don't we?
Another way to manipulate the dog races... (Score:3, Insightful)
This kind of data if collected well could very well help in profiling online trends and giving subscribers to this data a "leg-up" on their competitors. That's True. BUT, I doubt that this data can be used for predicting meat-world trends. The only people you are dealing with are the ones who are both willing to buy online and are willing to allow spyware on their boxen. I'd guess that the fact that you're talking about a select group of unsophisticated users, who are yet sophisticated enough to research and/or purchase online, would mean that the data is self-censoring.
It's sort of like surveying people who hate telemarketers over the phone. You'll only talk to a very few people and likely have a useless data set. It would be like a survey on invasion of privacy issues only from people willing to report to the surveyor thier SSN.
Catch my drift?
The resultant data would influence an investor house to make an unwise decision and bet on the wrong dog in the grand dog-race called the Stock market. The data provider can dope up the right dogs to make itself some money. That's what I think of whenever I read about these "trend" predicting companies. That's just me though.
Spyware (Score:2)
2) Add some hidden spyware in the product.
3) Sell tracking information to the big corporations and make $$$.
Re:Spyware (Score:5, Interesting)
Except that in ComScore's case, #2 is "Provide that something for free if people agree to be tracked by spyware.". If this is the ComScore I'm familiar with (and it looks like it is), they're up-front about what they're doing. You get their package knowing that you're becoming the equivalent of a Nielsen family with every single page you view being tracked and recorded. I personally wouldn't agree to that, but I imagine there's a lot of people who wouldn't care (or who have another computer for the browsing they don't want recorded).
They'll get blown away by Passport (Score:3, Interesting)
Consider marketing research firms that get paid boatloads of money to put people into demographic categories. Now, consider Microsoft's Passport initiative that tracks you online, where you surf, what you buy, where you live, work, and travel, and can infer all kinds of personal info like your domestic status. You are no longer part of a demographic group, you are a demographic; one of 200 million. How much would advertisers be willing to pay Microsoft for access to that database?
"How perfectly Goddam delightful it all is, to be sure." - Robert Crumb
Amazon trends (Score:1)
What they can do (Score:2, Interesting)
You can normalize for your sample by comparing your group against a known baseline. What is interesting to me is that the "normalized" groups like what Neilsen uses are tiny (3000 users) and thus have their own flaws (undercounting Goatse visits since their population of users is so tiny they're missing out on all sorts of behavior).
Here are some things you could do.
1) track e-trade, schwab, and yahoo finance to see what stocks people are checking - a good way to anticipate market changes/volatility
2) monitor site traffic for e-commerce sites to gauge how much business they are doing.
3) accurately measure how many click thoughs banner sites get (almost none)
etc.
--Pat / zippy@cs.brandeis.edu
Consider the source of the data (Score:3, Funny)
Not exactly a random distribution of the populous, is it?
They are already restricting the data this software collects, as its source of data pertains to a specific sample base:
1, (PC) Computer owners
2, Home Users
3, The type of fool that downloads something willy nilly on their computer.
Given this conditions, the survey base could not really exceed 50% of the range of population. Therefore can really only be 50% accurate.
(yes I pulled the 50% out my head, based on my survey of my coworkers).
The best part of their theory (Score:3, Insightful)
By the time there is, they ought to be able to pump out quite a few subscriptions, books, and speaking tours.
On the bright side, at least now we know where the "Y2K" baloney purveors went. Even better, they are leaving us coders alone this time.
Let me get this straight (Score:2)
seen it, heard it (Score:1)
talk shit, and blow smoke up people's asses
while stodgy old men try to discredit their
competitors by pointing out their poor
business models.
basic problems with this idea (Score:1)
2). Unless these people are actually collecting details about financial transactions done over the web (that cannot be legal right ?), the figures they get will be page hits and sites visited which does not really correlate to actual sales figures.
3). How good are their statistical methodologies, you can find fairly strong corrilations between any two sets of data by chosing the right stat functions.
4). Don't we have enough trend indicators already that are far more accurate than spyware collected date ?
5). Of course if they actually collected credit card numbers and looked up transactions then they would have better results. Anybody want to tell them that ?
Lets skew thier data. (Score:1)
Let them think that we are all buying beanie babies or some other stupid thing.
The Next Economic Bubble (Score:1)
Last Post! (Score:1)
He who knows not and knows not that he knows not is a fool. Shun him.
He who knows and knows not that he knows is asleep. Wake him.
- this post brought to you by the Automated Last Post Generator...
Re:I'm sure that's all very accurate. (Score:2)
I bet that according to thier data, 100% of all internet users use Windows.
They're sampling the dumbest 10% of internet users, they're not going to get anything useful.
Inexperienced might be a better word than "dumb"
(don't confuse the user with their software).
It's still a shame that companies take advantage of inexperienced users.