jd's Journal: Who uses Freshmeat?

One thing that has often puzzled me, when working in places that use Open Source software, is how many people know of Slashdot (I'd say 75% or more read it daily) but how few were even aware Freshmeat existed. The same was true of an announcement service that tracked Open Source and shareware products. Yet those projects I track on Freshmeat (I own something like 150 records and am subscribed to something like twice that) show hundreds - sometimes thousands - of accesses after a new release. If the corporate sector is totally blind to Freshmeat, who is doing the accessing?

Looking at the numbers, I think I can hazard some guesses. Educational and Government places probably rank high in the user charts, as clustering and scientific software are often moderately or highly subscribed and show moderate to high activity after an update. The stats are also skewed towards servers and other administrative or maintenance software, so I'm guessing it's more used by admins than users, which is somewhat foolish as users should be the ones driving updates as they're the ones who know what functions they need and what bugs they experience.

The popularity of MPlayer is an odd one, as most users will get this from their distro and it's unlikely to be used for system maintenance. Nonetheless, it is more popular than any other package, including the Linux kernel. Even the Linux kernel is oddly placed, at second, as this is announced in so many different places, from LWN and Slashdot to the Linux Kernel Mailing List and LinuxHQ. Most software is only ever announced on its homepage and on Freshmeat only if someone has made a record for it and is keeping it up-to-date. The dilution of the Linux kernel announcements is so staggering that it is amazing that a single service would get so much attention.

I guess if we assume a heavy Government/Educational userbase, it's more understandable. Those are going to be places where heavy-duty mailing lists are not going to be an option, and where surfing websites on the off-chance of an announcement would be frowned upon.

If I'm correct, how do we interpret the numbers? The usage won't be a random sample of a complete cross-section of the population, it'll be a self-selecting group with relatively narrow interests that is largely built up from a relatively small segment of the Open Source userbase.

Well, why should we interpret the numbers? That's an easy one. Corporations resist software they consider "unpopular" or "unused", no matter how useful or productive it would be. They are staggeringly blind to reality. If you can produce meaningful usage estimates, and can defend them, it sometimes (not always) weakens resistance to vitally-needed updates and changes. If you can show that some project has been downloaded by tens of thousands of probable competitors, you can be damn sure that project will be on the server by the next morning, come hell or high water.

Some would argue that it doesn't matter - we get paid to do what we're told to do and to make the managers look good. That entire discussion could - and does - fill vast volumes, with no real answer. I've got my own thoughts, but that';s not really this discussion and I'd probably run Slashdot's servers out of disk space if I were to put them all down here.

Here, I am far more interested in knowing why the userbase for any announcement service should be self-limiting. I've seen places be utterly ignorant of what software exists or where it can be found. I've had people ask me how to search for programs or how I know about updates before the distros push the packages out. On the flip-side, as I've already pointed out, there are packages whose records show far greater levels of access than you would expect, given the availability of the same (or better) information elsewhere, sometimes much sooner.

Based on what I've seen, I am going to say that the records for "mission-critical" software and software of specific interest to one of the niches inhabiting Freshmeat will be relatively close to the actual levels of active interest. Passive interest (eg: users of a desktop Linux system are probably not actively interested in new kernel or glibc releases, but still use those updates) is probably a lot higher, but I don't think it's easily calculable. I'm going to guess that the number of people who actually download the source code is somewhere between two and five times the number who visit the site via Freshmeat.

For commercial and industrial software, I'm going to guess that Freshmeat numbers are way too low, that people discover packages by accident or media rumor, or outsource the updates to some group that use a commercial tracking/monitoring service. For this type of software, I'm guessing that the actual number of people impacted by announcements might be anywhere from five to fifty times the number given in the stats. There is no simple way of finding out who knows what, though, because there is nowhere to look.

However, when giving a presentation to managers on why product A is the one to go for, you can't be vague, you can't be hesitant and you absolutely can't be technical. That's why having a bit more certainty would be a good thing. Lacking any means of being certain, though, anyone in that position has to give some number that managers can use. I would take the URL access value from Freshmeat (the number who actually visited the site, not just the record) and scale it by the midpoint of the numbers I've suggested. It's not perfect, but it's almost certainly the best number you are going to be able to get as things stand.

Yeah, yeah, GIGO. But managers don't generally care about GIGO. They care that they have plausible and defendable numbers to work with. That is what they are getting. If you wait to give them something precise and accurate, you'll certainly be waiting until long after any decision has been made, and probably be waiting forever in many cases.

What if you're a home user? Plenty of those exist. Well, to home users, I would argue that updates from distros are typically slow in coming, that library version clashes are far too frequent, that permutations of configuration that may be interesting or useful usually won't be provided, and that even distros that build locally (Gentoo, for example) have massive problems with keeping current and avoiding unnecessary collisions.

If you're not specifically the sort of user served by the distro of your choice, you WILL find yourself building your own binaries, and you would be very strongly advised to be aware of all updates to those packages when they happen.