CmdrTaco's Journal: Karma, Influence

(Standard mod system musings apply... I'm just ranting on thoughts about Slashdot's mod system. Nothing here is set in stone. These are just ideas that I'm mulling around.)

Karma doesn't work very well on Slashdot. The reason for that is really because it is tracking 2 seperate, and largely unrelated things: Posting & Moderating. The moderation system already seperates these things: moderation evaluates comments, and meta moderation evaluates the moderators. The former determines "Score" of comments, and the latter determines who gets more mod points in the future.

Unfortunately both of these systems use "Karma" as a key value in making their respective determinations. This kinda sucks.

It makes sense that we create a new variable, which right now I'm calling "Influence" to track a user's effectiveness as a moderator. Karma would continue to exist, but it would try instead to track your effectiveness as a poster.

Influence would instead track your moderation, more specifically your fairness rating in meta moderation. And the idea here would be to find the users who are the very best moderators, for more on *why* see my previous journal on rockstar moderators.

The new mod system will internally track comment scores as floats... so a user with a high influence could weigh more heavily then a user with low influence. An insightful from a user with 200 unanimous fair meta moderations under his belt could cause a comment to rise in score more than Joe Newbie who just got mod points for the first time today. But a single unfair moderation from our super user will cause him to lose much influence.

I also think there's something to be said about changing many of the numbers in slashcode from absolutes, to percentiles. This holds true in discussions (Score -1..5 is limited... we should instead be looking for percentiles... the best NN comments, the 10th percentile etc) as well as Karma... "Excellent" karma is not a static thing... it should instead refer to users being in the top NN% of all our actively posting users. The Karma Bonus should only be available to some "Best" percentile of users. Likewise, influence should be doled out in percentiles as well. But it's important to balance the system by increasing the penalties appropriately. If a user in the 90th percentile has the ability to move a comment score 10x more than a user in the 0th percentile, then his penalties should also be 10x as great. With great power comes great responsibiltiy!

The idea is that the very best moderators will have more power: perhaps gaining mod points faster, or else rate comments higher/faster than other users... but if they screw up, they should fall much faster.

Another interesting side effect of this is that we could give mod points to more users... but just have their "Influence" very low. We could use this to test their effectiveness. Here are some totally random numbers to try to explain what I'm getting at:

Joe has moderated 500 comments, and 499 of them are rated fairly. He is in the 99th percentile of Slashdot moderators. He gets mod points every 2-3 days. If he mods a comment 'Insightful' it goes from score 1 to 5 immediately. But if he gets modded unfairly, he will drop far... perhaps to the 50th percentile. From there his moderations only move comments up 1 point.

Bob has never moderated before. He gets his first mod points, and he actually only causes comments to move up .1. In other words, no meaningful effect at all (well, not exactly, since the new system is all floats, that could actually matter, but thats not what I'm getting at here). Anyway, he is meta moderated fairly, and with each passing 'Fair' his "Influence" increases a little bit. An unfair rating causes him to fall a bit, but since he has so little influence to begin with, its not that far... at least not the first time anyway... if a pattern is determined, he loses mucho influence. After a dozen fair mods, his moderations can move a comment by 1 or more.

The idea is as always, to keep damage control on high: if Joe goes outta wack, he's penalized hard and fast. But if he always mods well, he slowly rises in the rankings, and he can moderate more., and have his moderations weigh more in score calculation. Likewise, now we have a good way to determine if Bob will be a fair moderator. He can be "Ramped Up" as he earns it. This means we can give *more* users mod points. Encouraging more participation in the system, but weighing the newbies lighter than the old hands. That way the moderation system can move faster since trusted users will have more points, but newbies can participate. Sorta like a training period where mistakes aren't penalized as harshly.

Re: Karma, Influence

[BWJones]

which right now I'm calling "Influence" to track a user's effectiveness as a moderator. Karma would continue to exist, but it would try instead to track your effectiveness as a poster.

I like influence as a title and concept and one way to track it may be to evaluate how many responses a post gets. The problem with this is that inflammatory posts can certainly collect their fair share of replies and the parent post may not really be that "influential" or valuable. Perhaps this could be offset by some deg

Can be dangerous...

[singularity]

I moderate (*a lot*). I have not seen it lately, but every now and then I will get a message about a moderation being MM'd as "Unfair". I go back, review the comment and review the moderation. More often than not it is not too controversial of a moderation, and I end up wondering what the MM was smoking when he said it was "Unfair."

Since right now there is not a major drawback to getting MM'd every now and then (I would say that out of about 100 moderations, about 1-2 of mine get MM'd as "Unfair" for no ap

Experience Weight Multipler? - Flat or Curve

[Dark Coder]

This new system would effectively deal with bad meta-moderator and (assumingly) not keep their "Excellent Karma."

Looks like a trial and error time again with the moderation experience "weight" multipler value.

Dual-tracking should be easy to implement under the radar (for historical studies) while remaining on the old moderation.

These historical data can then tabulated to see whether my suspicious that this experience "weight" multipler value will be arrived to something like an inverse Bell Curve, rather