Slashdot Log In
IDC Analyst Dan Kusnetzky Explains the Numbers
from the pulling-aside-the-veil dept.
First, Dan Kusnetzky says,
Now, on to the questions...I'm responsible for research on access-point device management software, operating environment software (embedded, client and server), clustering and high availability software, web-centric computing software, and some portions of storage software. You'll note that PCs, workstations, and servers were not listed anywhere. My team follows software, not hardware.Other groups within IDC conduct extensive research into hardware (PCs, handheld devices, appliance servers, servers, storage systems, etc.). My team and I are part of IDC's software research group.
1) I just have to ask...
by Ded Bob
Might he have the stats on the BSD's. People from the BSD community are curious. :)
Kusnetzky:
BSD revenue or paid copies are tracked as part of IDC's research on Unix. In 1999, just over 8,000 paid copies of BSD server software were shipped. This would give BSD about 0.9% share of worldwide Unix server operating environment shipments.
The 2000 figures have not yet made it through IDC's publication process and so I will not pre-release them here. IDC has a policy that subscribers see research first.
2) Funding
by ritlane
In studies of market share (or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.
My question is: Do those who fund a study influence how the study turns out (i.e. Microsoft studies show higher MS market share). Or is it that these corporations only decide to fund groups who they know will most likely return results in their favor.
Kusnetzky:
Over 95% of the research which IDC produces has not been sponsored. Therefore, our research studies are completely independent. In the few cases where IDC performs specific research for a client, IDC has strict guidelines and review processes in place to ensure the objectivity of our research. IDC never does market share studies on a sponsored basis.
3) More breakdown needed
by BillyGoatThree
When People magazine does an issue devoted to "what's hot" in fashion, do they interview Jane Doe from Des Moines, Iowa? No.
So why are OS numbers reported with equal rating? Not all users are equally suited to *choose* an OS, therefore not all users *choices* are equally interesting. I'd really like to see a breakdown of OS by user-type (levels of education, field of degree if applicable, occupation, etc). Keep in mind this applies just as much to business. A technology company presumably put more informed thought into their choice of server than an art supply house or whatever.
Kusnetzky:
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
The complete findings of IDC's research are supplied only to subscribers. So, the snippets of data that Slashdot community members may have seen in magazines only includes information IDC chose to make public. Quite often, IDC's intent is to interest companies in purchasing the entire study. 8^) Another point is that it is quite possible that the findings which are mentioned in the press were taken out of context or mentioned incorrectly.
4)Self-fulfilling analysis?
by dvk
Do you think that there exists a possibility (or can even provide examples of) self-fulfilling analysis, such as "analysis says X is losing market share=people get skeptical about X=X loses market share although it may not have done so otherwise"?
If it is possible or already happened, do analysts in general (and you in particular) find it a worrisom possibility, and if so, are there any attempts/ideas to deal with the issue?
Kusnetzky:
Having studied a great deal of physics in college, I understand the Heisenberg uncertainty principle. That theory indicates that it is impossible to observe a system without interacting with it in some way.
While I suspect that studies have been done to detail how analysts (and journalists for that matter) influence the creation of products and their success in the market, I have not read them and, thus, can not comment on what they may or may not say.
5)Data origins
by m2
Do you base your data mostly on marketing analysis or do you actually go a pay a consultor to scan machines on the net? If there are scans involved, how do you pick the IP blocks to be scanned and what's the uncertainty associated with such a method (and how is this uncertainty guessed)? If there are no scans involved, why not? If this is "maket analysis", can you defined that for me? Which factors are involved? And a different question: who's the target market for this kind of study? How much does such a thing cost?
Kusnetzky:
IDC conducts two types of research -- demand-side and supply-side. Demand-side (end-user) research is designed to shine light on who is buying and using information technology products, how these products are being used, and what plans decision-makers have for future purchases. A scan of the network is not involved in this process. Contacting people and working with them is the basis of IDC's demand-side research. Network scans, while interesting, are not part of IDC's current research methodology.
Another important point is that a network scan would only show systems which were up and available at the time of the scan. Any systems protected by a firewall wouldn't appear. This means that the study would not be able to shine light on what organizations were using inside of their firewalls.
The cost of demand-side studies is driven by how many respondents are needed, how many questions must be asked, and how many countries are included. A small study conducted in North America which asks a few questions can be conducted for tens of thousands of dollars. The data gathering phase of such studies might be completed in a few weeks. A large, multi-country study which must be segmented by company size, market type, etc. might cost many hundreds of thousands of dollars. The data gathering phase of this type of project might six or more months to complete.
Supply-side research shines light on revenues, shipments, technology trends, etc. IDC gathers information from the financial reports of public companies and from public statements made by executives of privately held companies. This data is then segmented into 102 different software markets, 9 different operating environment platforms, and 6 different geographical areas. The software markets, by the way, are defined by a software taxonomy which attempts to define a list of markets which is both exhaustive and mutually exclusive.
The company data segmented using this model is offered to representatives of each of the companies. After the company representatives have had a chance to review the models, an IDC analyst then contacts the representative to verify the segmentation.
Once IDC has completed this process, the company models are examined by internal and external reviewers. Any questions which arise during the review process are brought back to the company representatives and resolved to IDC's satisfaction.
IDC goes beyond determining the revenues to modeling revenue or paid shipments in some markets. In this case, the revenues produced by the first process are segmented using demand-side data which shows the average shipment value. The modeled shipment data is, as with revenue data, examined by both internal and external reviewers. As before, any questions are resolved before data is published.
IDC, by the way, is watching over 1,200 companies worldwide. Revenue data from these companies is collected on a quarterly basis, segmented into IDC's taxonomic software markets and then stored in the IDC software research group's software forecaster database for analysis.
6) *what* constitutes a Linux server?
by Anonymous Coward
At what point does a computer become a server? Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?
Kusnetzky:
IDC's software group is using a very simple definition. If it serves the needs of a single person, it is called a client. If it serves the needs of many people it is a server, regardless of the system configuration. IDC's enterprise server research team has detailed definitions of different server configurations, but I won't repeat them here.
Linux, as observed in a recent study (December 2000, N=1583 North American and Western European respondents), appears to have a different usage pattern now than seen in earlier studies. This recent survey showed that just over a third of Linux shipments were used as client operating environments, roughly a third of Linux shipments were used as server operating environments, and just under 30% of Linux shipments were used on a workstation which supported the workload of a single person and provided some service to a workgroup. We're calling that new category "Serverstations." For historical purposes, Serverstations are currently allocated across client and server operating environment shipment totals. IDC will be publishing data on each of these categories in the near future.
7) Polling questions
by cavemanf16
I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method? Now if the answer to that question is, no, how much can we rely on polls, since we have no way of verifying if the questions asked and the people interviewed were heavily biased to favor one outcome over another? (Such as in the recent large discrepancies of the 8% vs. 24% use of Linux as a server results that we've seen on Slashdot recently).
Kusnetzky:
I believe that I've already covered this area in previous answers. It is clear that IDC's server operating environments data is being compared to another research firm's server adoption data. This comparison really isn't valid for several reasons including the fact that supply-side research is being compared to demand-side research and the fact that software research is being compared to hardware research.
IDC's extensive hardware research shows that Linux only holds a small share of the market when one examines shipments of servers and then segments it by the operating system which was installed at the factory. IDC's software research is showing that Linux server software is being installed on both older and new systems and that the configurations being used as servers include PCs, workstations, appliance servers, and more traditional server configurations.
8) What about the so-called "third world"?
by Kareem Abdul-Lamarr
Do these analyses factor in the so-called third world? Most of these analyses are US-centric or some times do include the continent across the pond but what about Africa and Asia? Do these analyses *really* take inputs from these continents?
Kusnetzky:
At this point, IDC has offices in 43 countries. IDC's software research team is represented in North America, South America, Europe, Africa, and Asia. IDC's operating environments research included revenue and shipment data from over 140 different Linux distributions, and we have collaborative input from analysts in geographic areas around the world. I think that this means that the answer is yes, IDC is examining operating environment revenue and shipment data worldwide.
9) Who Keeps track of the Predictions
by dmccarty
I have a question on the area of predictions in general. For example, fellow IDC analyst Jill House has been severaly negative on Palm over the years, with regard to the Win CE operating system and devices. A sample quote from her in Feb. 2000 read, "If I was Palm, I would be beside myself with panic."
The issue is, that over the last 3 - 4 years she's been predicting the demise of Palm and the rise of Win CE, a claim that has never materialized. Who verifies the reliability of these predictions and keeps the analysts accountable. With the frequent sound bytes and one-liners that they give to the press, these analysts have significant influence over public perception of the issues. But how is policing done when the analysts don't analyze very well?
Kusnetzky:
I really can't speak to this issue directly. Jill is not part of the software research group.
I know that a prominent part of the software research group's annual forecast and analysis reports for each software market is reviewing last year's forecast, how close it came to this year's findings. If the findings differ from last year's forecast, the analyst(s) attempt to determine what changed and why. I'm told by subscribers that this analysis is often is very helpful.
10) Re:Question(ADDNUM)
by HeUnique
I just remembered an Idea that a friend of mine suggested:
What if IDC could work with the Linux distributions (RedHat, Mandrake, SuSE, TurboLinux, Debian) to add a small program which will run after the first internet connection has been succsessfull..
When this program runs - it will ask the user to "register" his copy of the distribution. If it has been purchased from one of the distributors then the user can add his serial number. Some other questions like will this distribution be used as a server, a workstation, or combination of the 2, or a development workstation. The survey SHOULD be anonymous (unless the user wants to give some details about himself)
By that way - the distributors can give the numbers back to IDC - and IDC can publish a report which will tell that the number of Linux installations - and that number is X. X is combined of Y free download version and Z purchased copies of Linux.
What do you think, Dan? (And what do Slashdot readers think about it?)
Kusnetzky:
I agree this would be interesting information. I don't believe that this would be a good business for IDC to be in. Our analysis is that many users of Linux don't want their organizations or their competitors knowing what they're running. They'd rather get kudos for a job well done than criticism for how they got the job done. Not only that, but some vendors that have added features to track activations have also faced a great deal of criticism. I doubt that Linux users would appreciate such tracking operations.
Why the emphasis on revenue? (Score:5)
Which is great for your supply-side surveys. But surely demand-side clients are far more interested in actual deployments rather than paid for shipments. When making a decision about an OS, I don't care how much revenue it's generated for a given company, but I am interested in how widely it's being used, and in what areas. For example, of our 25 or so servers running Linux, Solaris and OpenBSD, only 5 or 6 have paid-for copies of the OS, and I know that plenty of others are in the same situation. Going purely by paid for shipments, your figures are going to be wildly inaccurate.
Methodologies and Trust (Score:5)
Could you imagine a scientific journal that only contained abstracts? It would be laughed at. The methodology is absolutely essential to understanding the trustworthiness of the data. There needs to be full disclosure of these procedures before they release their snippets into the media.
Spreading unfounded information is as good as lying.
David E. Weekly [weekly.org]
Re:Linux on the Desktop (Score:3)
If 50% of the linux market is desktop then it has 10% of the total, not 0.3%. Your mistake came from assuming that 150:1 win32 client:server ratio even applied to linux(you said 1:1). In fact, this shows that windows servers have 0.6% of the server market, whereas Linux has 10% of the server market or 1500 times more servers.
Client side it's 10%(linux) vs 39.4%(windows). So windows has 4 times the market share. I do consider 10% cracking the market.
I believe your largest mistake was lumping both segments together and comparing them.
NB. True number are closer to
Win32 - 90%
Mac - 5%
Linux - 5%
for client
Re:The important bits (Score:4)
Er, I should have said "doing both an analysis based on actual usage surveys, and one on market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used, as well as how it's being sold, assuming you look at the right sets of their numbers."
-
The important bits (Score:5)
IDC IS doing an analysis based on actual usage surveys, not just market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used.
Dan knows his mouse from a hole in the ground.
Dan knows the difference between Linux, BSD, and Unix.
IDC seperates Linux from Unix and BSD because they think that division matters to their customers. They don't seperate BSD from Unix because they think that division doesn't matter to their customers.
IDC gives a shit whether it's predictions turn out correct, and tries to improve their accuracy by examining their results.
In all, it sounds like Dan's a geek, knows what Linux is, and is concerned first and foremost with producing accurate, truthful, scientifically-gathered information.
Now watch me get modded down for not following the party line.
-
Re:Would by zealots, for zealots be better? (Score:3)
There you go again, confusing "good" with "really unbelievably crappy".
Exchange is NOT a very good product. It happens to have a TON of *features* that PHBs really like, but nobody in their right mind would consider it stable, let alone good.
We have both sendmail AND exchange running. They both serve their purpose, and they are both stable *in their current state*.
The difference is that I *can* reconfigure the sendmail servers. I *can* perform upgrades. I *can* back it all off, and restore them, and expect them to come up just as they were. I *can* add different virus scanners. I *can* run other services on them.
The Exchange servers are black boxes. You don't run upgrades (of Exchange OR Win2k/NT) without testing them thouroughly on a control machine. You don't run other services on them. You don't expect a restore of the server to come up correctly.
Sure, they are stable, as long as you don't so much as look at them sideways.
Re:Would by zealots, for zealots be better? (Score:3)
Dude, I hate to tell you this, but where I work if you don't do upgrades without testing them thouroughly on a control machine, you'll probably find yourself out of a job. It doesn't matter if the platform is windows, linux, mainframe, unix, or the doorlock on the bathroom. If it isn't tested before going into production, you are going to find trouble.
I am as big a linux zealot as they come. All of my home machines are linux-only machines, and I quickly repartition any work hard disk so that I can have a copy of linux on it. I publically hate microsoft. BUT I don't think that it's good advocacy to start telling people that our software can be used without extensive testing. Sure it's a worse plan for M$ software, but it's a bad plan for any software.
I do understand your point that Exchange is crappy software, especially when compared to some of the opensource/free alternatives. But that doesn't justify ignoring good change control and change management policy.
$.02
--
Re:Heavy questions, Light answers (Score:5)
IDC's software research group has an editorial review board. All forecasts are reviewed before they are published. Since IDC's name and reputation are on the line, every attempt is made to make sure that the assumptions are reasonable and that they have been applied in a supportable way. They still, by the way, may prove to be wrong in interesting and highly public ways.
IDC analysts review previous year's forecasts each year when the report is published and discuss whether or not they were accurate. If they were not, the analyst tries to point out how market trends differed from those projected in the previous report. This is an attempt to help the subscriber gain some value from what the analyst learned from thet time the first forecast was published.
The bottom line, however, is the review provided by subscribers. If they renew their subscription, they found value in the program. If they don't, then they didn't.
Statistics & Methodology (Score:3)
As a former statistician, let me tell you that performing this type of analysis is not cookie-cutter stuff.
The methodology for gathering information, and the feedback mechanism he describes, seem prudent to me. I see little fault with his analysis, given the constraints on his access to the target data set. I think he did a good job explaining it to a group of non-statisticians. Nice interview.
Heavy questions, Light answers (Score:4)
So I was pretty surprised to read his interview. After thinking we were going to get some "real" answers, it was disheartening to read his corporate, fluffy answers to the questions. My question (question #9 [slashdot.org]) was glossed over with a "that's not in my group" answer, so the question of who keep the analysts honest is still a very open one. Someobody other than yourself has to keep you honest. Accountants have audits, programmers have code reviews and people in construction have inspectors. Reviewing your own data from last year isn't good enough without either a) an oversight organization to make sure that your mistakes won't happen again or b) a watchdog agency to make sure that your mistakes won't happen again. Apparently, IDC has neither.
Explanation (Score:3)
My analysis (Score:5)
IDC: By PHBs, For PHBs (Score:5)
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
To summarize: they make recommendations to the purchasers based on comments received from the purchasers, without regard to what the users think!
Now I know why I have to use Exchange.
Linux on the Desktop (Score:3)
Self-fullfilling? (Score:3)
It was announced this week that iPAQ will overtake the Palm very shortly (assuming you could Palm, inc seperate from Handspring, etc). Maybe she was right all along? Or did her predictions have some influence? Personally, I think the problem is that palms were great in 1997 but they haven't really gotten significantly better since then. But it is interesting to think about.
Thanks IDC! (Score:4)
1. The consulting company brought in by our PHSM uses IDC demand-side research (for PURCHASERS by PURCHASERS)?
2. We (the IT group) and our users are getting tired of our Novell file servers and *nix web servers never crashing?
3. Microsoft invested $600 million in our company a year ago when we really, really needed it?
And the bonus question: is it still called demand-side when it's being shoved down our throats?
(BTW, Deloitte Consulting believes in Microsoft software and IT outsourcing.)