Slashdot Moderation : Exercising Agreement
Update (5/28/03): The information in this journal is outdated and no longer reflects the state of Slashcode; this journal is a historical record but is no longer accurate.
Slashdot Moderation : Exercising Agreement
A journey into the statistical methods employed to create Comments That Agree .
As I discussed in my previous journal entry, nearly 500 moderators were manually and permanently removed from the pool of potential moderators for giving positive moderation to this comment. While the manual banning of potential moderators by the Editorial staff is not documented, the source code used to accomplish this is readily available and the practice appears to be quite commonplace. The reasonable questions that arise from this practice are: how many moderators are being banned, what is the objective, and what are the actual results? The Slashdot Editorial staff has repeatedly stated that they don't have time to answer these questions (they're certainly not addressed in the FAQ), so it's up to us, the readers, to ponder them.
How many moderators are being permanently banned from the moderation pool? No one knows for certain except perhaps the Editors. Let's look at what we do know. Five hundred moderators were banned for moderating this comment. If we assume that one such comment is posted on a daily basis, that is approximately 185,000 potential moderators banned per year. That post was probably an exception, so we must revise our number. Jamie McCarthy made a post to a User-Created discussion which is deleted every two weeks, so I must reproduce his comment by cut & paste:
"Well, again, we have the logs, so we know what we're talking about. We log both how many times we give mod points, and how many times we would have given mod points but didn't. In the former category: 99.73%; in the latter, 0.27%." - Jamie McCarthy, Slashdot Editor
Additionally, Rob Malda made a related comment in a User-Created discussion:
There are a quarter of a million daily readers, but (and I just checked) only 1.2% of them post. So understand that those of you posting in the forums are already a minority of the community. - Rob Malda, Slashdot Founder & Editor
Now we have some numbers to run with. Of the 250,000 people who read Slashdot, 3,000 of them have an "Interest Level" high enough to post comments here. This "Interest Level" stands at 1.2%, or
What is the objective of the Editorial staff in manually removing User Moderators? That's a really tricky question to answer. Keep in mind that the Editors have gone to extensive lengths not to answer that question, so we must assume that it is an emotionally loaded subject. However, we can interpret some of their actions to arrive at an answer. The post mentioned in the beginning of this article was repeatedly moderated as Offtopic by many Editors. Rob Malda described it this way:
however only a smaller percentage of those moderators actually use their points... hundreds of users moderated it up [and I subsequently banned them all] I modded it [the comment] down a few times too. - Rob Malda, Slashdot Founder & Editor
Note that text in brackets was added by myself to reflect what we've learned. Reading that statement makes it pretty obvious that the purpose of removing Moderators from the pool is to ensure that the results of the Moderation system are consistent with what Rob Malda believes that the results of the moderation system should be. Moderators are chosen from a pool of Slashdot users who have not moderated contrary to the wishes of the Editors in the past. As the Slashdot FAQ states, "The Slashdot Editors have unlimited mod points
When surveying the impact of the Moderation system, there is one overridingly important statistic. That statistic is that over 99% of Slashdot readers do not post comments. Keep in mind that the default view of Slashdot has a threshold of one. This means that if you're moderated to zero or less, 99% of the people who would have read your comment... won't. Books could be written about whether or not blocking communication to 99% of the possible audience constitutes "Censorship", whatever that means today, but for the purposes of this essay, the point is moot. Suffice it to say that Moderation determines 99% of visibility, statistically. Now, consider that the Slashdot system, as a whole, is a constantly evolving system in which Moderators can transition from unbanned to banned, but not vice versa. Moderators who moderate differently than the "guide" Moderations of the Editors (which constitute 10% of the Moderation) are removed from the pool of influence. Therefore, Slashdot as a system moves in only one direction: towards promoting comments that are Moderated in agreement with the Editors. What is the role of Metamoderation? Let's look at the FAQ:
according to Meta Moderation, the fairness of these [editor Moderations] is statistically indistinguishable from the moderation of non admin users (92-93% of moderations are ruled 'Fair').
Keep in mind that the 5 to 18 percent of Slashdot Moderators who are banned from participating in Moderation are also banned from participating in MetaModeration. The purpose of MetaModeration, therefore, is that of reinforcing agreement. Those who have agreed with the Editors in the past are allowed to vote on whether the Editors moderate fairly. Not surprisingly, this subset of potential MetaModerators often agrees with the Editors. If they didn't, they'd probably have been removed already! MetaModeration, therefore, serves to reinforce agreement with the Editors among the selected Moderator pool, and Moderation serves to enact agreement with the Editors among the comments visible to 99% of Slashdot's readership. We only need one more quote from the FAQ to understand what this means:
Goals [of Moderation]: 1.Promote quality, discourage crap.
Since the Moderation system is a self-reinforcing system that promotes comments that agree with the Editors, we must assume that this comment means two things:
- What the editors believe is quality.
- What people who disagree with the Editors believe is crap.
That may be a strongly worded value judgement, but it stands as a conclusion easily & objectively reached. But aside from this value judgement, Slashdot stands as an example of a best-of-breed solution for those who wish to create a community that promotes agreement. Slash appears to have been specifically designed for this purpose, and it can be employed to create a community that agrees about.. just about anything! For instance, ask a Slashdot reader if Linux is a great Operating System. The answer will almost certainly be an emphatic yes! Rob Malda has created an excellent system for managing a userbase into agreement, and for the most part, the users agree with the system. Which was probably the whole point.
Slashdot: News for Nerds, Stuff that Matters. Comments that Agree.