Forgot your password?
typodupeerror

Moderation Ideas 322

Posted by CmdrTaco
from the stuff-to-hammer-on dept.
So now with the IFRAME/ILAYER stuff mostly removed from Slashdot pages are rendering faster. I still have some SQL glitches, but I also have a few more ideas for the moderation system that I want to throw out to you guys for opinions (and my guess is you'll have a few).

First Some Comments on M2 and Karma

As a whole it seems to be working pretty well. Some people are really irritated, other people like it, and some people simply are fretting over their karma way to much. Let me just remind everyone that Karma isn't that important. You should expect to see some karma come and go since your activity is subject to both M1 and M2 moderation. As an aside, M2 moderation will not push you out of the -10...10 range (in other words, being a bad moderator will never cause you to get the Comment Penalty). The only thing karma affects is eligibility for moderation participation.

I really wish I didn't feel obligated to display peoples' karma on their user info page. It tends to be a distraction. It's not supposed to be an integer determining anyone's value as a human being, it merely functions as a sort of 'Risk Factor' for moderation related activities. High Karma means you are a low risk, negative karma means you shouldn't be given tons of moderator points and turned loose on the world ;)

"The Problem"

As I see it, the major problem on Slashdot is bad people gaining moderator points. The M2 system is one way to counteract that: An unfair moderator is (theoretically) weeded out. They are still free to post and participate as always, but they can no longer moderate if their karma drops below 0.

"The Assumption"

I'm an optimist. I assume that there are more good people than bad people in the discussion groups. I assume that the good people will participate, and help weed out the bad ones. I also assume that eventually there will be world peace and harmony amongst all races and creeds and religions and genders. I'm young and an idealist here, gimme a break ;) As a whole, I think this is true: I've Meta Moderated many times, and I rarely find that more than 20% of moderation is bad. Many moderations are questionable, but rarely do I read one and think "What a moron". But they are there, and I want to try to make it happen less.

"The Solution"

On an abstract level, the solution is to restrict the amount of power that any single user has within the system. Certain restrictions are in place already: moderators occasionally get 5 points, they and they lose them after 3 days, can't post and moderate the same discussion etc. Each of limits is designed to reduce the risk that any given person can screw with the system.

Each of these limits suck for a good moderator. This is given: the naughty people force rules to exist for the good people. If those 84 year old blue haired ladies who drive on the yellow line at 20MPH with their left turn signal on weren't around, maybe we wouldn't have speed limits either (Note: I also tend to drive 5-10 MPH under the speed limit and regularly forget to turn off my blinker, so I am part of the problem ;)

"The Idea"

The idea is inspired by M2 moderations removing of the ability to choose what you M2 moderate. (This is flawed of course because you can reload and get 10 new comments but I'll fix that eventually). It also is a pain because you have no context for many of the comments. By not choosing which comments you can moderate, the risk of doing something inappropriate is greatly reduced: eg, when Mr. Bad Moderator gets 5 points he waits until GNOME article appears and then moderates all the pro-KDE comments down.

The solution is to reduce the probability that Mr. Bad Moderator can moderate a comment that pushes his agenda. This is what the point limits and the time limits are designed to accomplish today, and what this idea is designed to accomplish should we decide to implement them.

So here are the ideas:

  1. Put 5x (Numbers are Variables here: Don't complain about them!) as many moderators into the system.
  2. When you have moderator access, only display moderate controls on say 20% of the comments.
  3. Increase the percentage of moderatable comments based on karma.

It makes sense: negative karma users would never moderate, and more karma would give you more choice in your moderation. It would cripple good moderators because they might see a bad comment but be helpless to moderate it, However since there are 5x more moderators, the odds are someone else will get it. Side benefits include more people given "a shot" at moderating and more people will be allowed to contribute. If the M2 stuff works, the bad moderators will lose karma and no longer be allowed to moderate, and the better moderators will be given a larger percentage of all comments to moderate.

Another Sort of unrelated idea:

  • Integrate M2 into the comments display tree: this solves the context problem that the existing metamod page has. Simply randomly select the occasional moderated comment for M2 moderation. Plus its much more "Real Time" and becomes less of a secondary chore. Plus it allows more people to participate since M2 moderation has much less restrictions on it. The downside is that this might be a pretty serious performance hit. I haven't really thought through it yet, but its an interesting thought. Comments on it?

And finally a note to the flamers...

Turn it off. Go to your user preferences and click 'Unwilling' and you won't be asked to moderate. Click 'No Scores' and you won't see scores again. Change your threshold to -1 and forget that they even exist. This system is designed to provide a solution to people who wish to use Slashdot like usenet. It's designed to provide a solution to people who want to use Slashdot more like a newspaper. It's designed for people who want something in the middle. If you don't like something, you should be able to change it. I've worked my ass off to make as much of this optional as possible. I don't care if you turn stuff off. It won't hurt my feelings at all!
This discussion has been archived. No new comments can be posted.

Moderation Ideas

Comments Filter:
  • by Anonymous Coward
    Ok,
    so I like reading comments, but after the first x odd, I get bored. So then I cull the boring comments by upping the threshold to 3 or 4. Fine.

    The problem is that moderators feel this way too. Any comment past the (say) first 50 is effectively lost.

    Unfortunately, the very people who have insightful comments are likely to also have better things to do all day than reloading /. so that they can make it in past the 50 comment limit. Firewalls and caches don't help either. Nor does the fact that it takes longer to write a well researched comment than a stupid one.

    I'm not proposing any fix, just pointing out a percieved problem.
  • I guess some k-r4d AC posted his five -1 articles from the POP I dialed into. Way to punish me for some 'l33t kiddie long since moved on to the next POP Rob.
  • by Anonymous Coward


    Just a question here (and not meant to be flamebait) but do you think that you're continuously losing karma because your moderation practices are at fault?

    It's possible, although I don't think so. (Obviously, I'm biased) I would say that at least half of the moderations I've done should be obvious (marking down obvious trolls and marking up obvious gems.) Perhaps I've been a little too aggressive, but I don't think it's been that bad.

    There are a couple of things to keep in mind, though. First, I have no way of finding out which moderations got me marked down. So if M2 is simply knocking me down for doing a crappy job, I have no way of going back and changing my ways. Secondly, I think the volume is excessive in any case. Even if I've been doing on balance a crappy job (say 60% unfair and 40% fair), losing 20 points in a week is ridiculous. I may be a special case since I get to moderate a lot more than most people, but it's still excessive. If I had to guess, I'd say I've doled out about 20 moderation points in the last couple of weeks, so that would mean that every moderation a moderator does (if they moderate similarly to me) they will lose one karma point.

    I also have talked to at least one other user who had a similar problem, so it's possible we are flukes, but it would be interesting to find out whether the average M2 is up or down, and how much the average moderator's karma has changed (perhaps karma/total moderation point doled out)
  • by Anonymous Coward
    >Turn it off. Go to your user preferences and click 'Unwilling' and you won't be
    >asked to moderate. Click 'No Scores' and you won't see scores again. Change
    >your threshold to -1 and forget that they even exist.

    Wouldn't it be easier for non-registered ACs if the defaults were set up this way? That way, if a troll doesn't see his post, then he won't be tempted to continue, unless he actually knows how to change the user preferences. If someone wants to read another's trolling, let them modify their user preferences.
  • by Anonymous Coward
    Please moderate this post down for being off topic.

    Oh wait! This discussion is about moderation.

    Mark it up then.

    In any case, use up those moderation points, because it makes you feel powerful.

    I moderated once ages ago before I quit using Linux and decided it was more fun to just cause trouble on Slashdot by adding less-than-dogmatic posts. I've thrown away three nicks so far out of boredom with the whole concept of meritocracy.

    Last week one of my AC posts got moderated up to 3, and I was trying to make fun of Linux with it (people thought it was funny... oh well) The post about putting black electric tape over your USB ports to make them Linux compatible.

    I guess I'm not a very good troll.
    • All systems are flawed ;)
    • I think the race condition is ok. I think moderation points should be assigned regardless of score. If I didn't think it would be obnoxious, I would prevent the display of the score for moderators.
    • Most of your people problems would be solved by having more moderators.
    • Individual preferences (personal kill files etc) are actually on my TODO list. I just can't come up with a clean way to code it, so I haven't bothered with that yet. It would also be cool to say "Show me what Bob liked". I may implement that at some point too.
  • Just one thing I've noticed, not saying it is good or bad, is that there is a lot more moderation being done now. The weeks before all these new moderating features took form, setting threshold to level 2 gave me a few selected good articles. Now at level 2 often there are a lot of articles which most of are good. Now I've set threshold to level 3. Great to see there is more work on the moderation field taking place. Lot of progress, this is really going to be a big sucess. Good job Rob!!!
  • Cfr subject. Sorry for not doing it myself. I would if I could.

    --

  • I emailed Rob about this, and he said that the top and bottom 20% of readers are thrown out (leaving the middle 60%). The theory here is that the most infrequent 20% of the readers don't really read enough to be able to make informed judgements. Throwing out the top 20% is intended to stop people in that category from writing reload scripts to get themselves into a "reads slashdot more often" category. It's probably still possible to cheat the system somehow, but it makes it more difficult I suppose.

    I dislike it as well, as i've been here for over a year and have high karma, but have never, not even once, been given moderator access, since I read slashdot two to three times a day and click the "read more" on nearly all the stories.
  • Over the last few weeks, watching how the moderation system has worked (including how it's affected my own comments), I have to wonder : How is the moderation value supposed to be interpreted?

    What I mean is, if a comment gets a 5 rating, should it be interpreted as being an exceptional comment, or merely a comment 5 people thought was a little better than the usual fare? Right now it seems like the second, but in my view it should be the first.

    After all, look at the disproportionate number of comments that manage to get a 5 rating because they're 'Funny'. Sure, they may be amusing, but are they really the best that Slashdot has to offer?

    The problem, as I see it, is that we are asked to moderate without knowing the current rating of a comment -- it takes a while to read through and moderate all comments to some articles, by which time the values the comments had when you loaded the page are no longer valid. So if you decide to moderate up some minorly amusing AC's comment as 'Funny', it suddenly ends up a 5, because 4 other people thought the same thing -- even though you might have thought it deserved no better than a 1 to 2.

    There's a nice solution to this problem, but it would be a bit more complicated than the current method. A similar problem exists in stock markets -- by the time a buy order you've submitted gets taken care of, the price of a stock could have moved way past what you wanted to spend. So along with your buy, you specify a price range -- if the stock is out of that range when the buy gets cleared, you're not left holding some overpriced stock.

    Anyway, here's the idea. Instead of moderating up or down by points, you should be able to specify exactly what rating you think a message should have. Your moderation would count as a vote in favor of any rating up to the one you specify -- past that, it doesn't count.

    Thus, if you think a comment deserves a rating of 2, but not better, you could moderate it to 2. If 10 other people do the same thing, the comment would still have a rating of 2 -- as opposed to the current situation, where it would have shot up to 5.

    (Mathematically, you pick the highest rating such that the number of votes for that rating exceeds the rating. To get a rating of 1, you just need 1 person to vote in favor of a 1 or better. To get a 2, you need 2 people to vote it a 2 or better. Etc.)

    Any thoughts?

    JRaven
  • the reason I think moderation is a chore is because 1) all of those select boxes make the page load and render slowly
    The reason that I find moderation to be a chore has nothing to do with the mechanism behind it. Normally I set my threshold around 2 to take advantage of the work that the current moderators have done. But when I have moderator points, I set my threshold lower so that I can do some of that work myself. If I could moderate at any time, I wouldn't bother to lower the threshold and very little of what I'd see would need further moderation.

    I think that the current system of assigning moderator points does a good job of distributing the work of moderating. The main idea for a change that I can think of would be to assign moderation points separately for each story (something like, there have been 10 comments, the next eligible moderator to enter the comments section gets a point). I think that this would be a better way of preventing the possible problem of not having any moderators read the comments for an article, but it might not be as good as the current system in avoiding moderator bias.

  • I disagree. To gain karma they need to write good comments and/or moderate appropriately. As long as they're doing that, I don't care if they're doing it to gain karma or for more altruistic reasons.

    At least as long as they don't get obnoxious about their high karma. And even that would probably be mostly self-correcting.

  • In image processing speak what I'm about to explain is basically "alpha trimmed mean".

    1. The moderators score an article on a scale of -1 to 5 (or what ever).

    2. Slashdot takes the list of scores for an article and sorts them.

    3. Slashdot discards the top and bottom 20% (or whatever, aka 'alpha') of the scores from the sorted list. (This is the trimming bit).

    4. Slashdot them averages the remaining list of scores and uses the result as the actual score for the article. (statistical mean bit here).

    Problem solved. Articles get moderated. Bad moderators have no effect. No meta-moderation is needed. No race conditions either. A few more moderators and moderator points would need to be handed around.

    As a bonus, a background process can hunt for bad moderators and eliminate them. It would just have to find moderators that constantly score articles way different from thier peers. (Insert statistically analysis here).

    Whatdoyathink?

    --
    Simon
  • And what of the lurkers? Just because a person doesn't post doesn't mean they wouldn't make a good moderator.

    Actually Rob, how many users are there who haven't posted, but still read regularily?

    Well, I've made a grand total of about 8 posts in the last year, and I've gotten moderator status t wice. (Karma is 6, I guess only making useful posts helps).

    Well, as long as I'm posting, I might as well make my own observations.

    • Metamoderation is good. Slashdot does have a bias towards linux and open source stories (and a bizarre fascination with that dead company Gateway bought out, but maybe Rob'll quit posting that... :). The moderation does need some sort of feedback mechanism. Though anyone who puts in too much negative feedback should be crosschecked, perhaps. As always, it comes down to a matter of trust.
    • I'd like to see some sort of personal scoring. I know it adds a whole new level of complexity to the database (Hey Rob, you do back this sucker up, right?) and it'll probably hammer the new machines right back into the performance pit they were in. Oh well. That's what you get for centralizing all the logic on one side. An NNTP interface would fix that, of course, but that would be a bear in and of itself. Perhaps I should dig my way into the slashdot code and do something about it myself. You know, submit some patches.... Though a posting mechanism would be harder. NNTP supports USERINFO AUTH though... but I digress.
    • Hmm. I had another point to make. What was it... Oh yeah. I'd like to see the unlimited moderation stuff. I think on the whole that would push positive comments up more, but more evenly. Some of us do silly things like advocate BeOS and OpenBSD, and being in the minority means we have fewer opportunities to moderate posts that make good points about our platforms that aren't otherwise moderated up. Yes, there's potential for abuse. Perhaps high-karma zero-complaint moderators (K1/K2?) could be handed this on an experimental basis....

    Just a few thoughts. Maybe we should hand out positive karma points for well formatted HTML (Check out my

    s!).

    -Dave, going back to sleep

  • Ugh. And let me say that I'd love the ability to fix that angle-P tag that got broken somehow. -dbt
  • When I am meta moderating, the intuitive thing to do is to glance at the comment, read the first line, and then check fair if the comment has been moderated up for 'interesting' or 'informative'. Sometimes when large amounts of text is being read on subjects that people aren't particularly interestd in, they try to skim. I don't curretly have an answer to this problem. I'm also not too sure this is a bad thing, as everything I've marked as one of the two I's has actually BEEN interesting when I read down further. My intuitions were right.

    Is the voting question comments included in meta moderation? I have yet to see one.

    The idea of the different reasons for marking up and marking down is interesting. Articles are posted for different reasons. For example, when Stanley Kubrick died [slashdot.org], there may have been a funny comment posted about him, and it may have been moderated up for such, but a funny comment is not necessarily the thing people would want to see moderated up to 5. However, it may deserve a 5.

    One solution to this problem would be drop down boxes next to all the moderation types for the poster of the news article. The poster would select a priority from 1 to the number of moderation reasons. When the user reads the messages, a '3' that is informative would be read before a '3' that is funny.

    This could be prioritized at the user level, but I think it would be more appropriate for the article poster to decide which the priorities should be.

    When The Autism Article [slashdot.org]'s comments were read by me, I was dismayed to find a humour post being moderated to the top of the posts, when this was a potentially sensitive and touching issue. (Although there is a place for humour, even there.)

  • I think that the tweaks done to the moderation system are GREAT!
    I personally have become much more sensitive to that urge I get to NOT post a quick smartass comment, because I want my karma to go up. I'm specifically being more careful and thoughtful about what I post. I think a lot of other people are too, and you can see that in the way the quality of comments has improved in the past few days (and the volume generally declined).

    I don't think this is the end-all, eventually, someone's going to figure out how to abuse this new system, but for now, it's working great.

    "The number of suckers born each minute doubles every 18 months."
  • I don't know - I think /. was getting really, really bad before the first iteration of moderation came on line. The first try fixed a lot of problems, but caused a few new ones, and eventually, folks figured out how to abuse the system. Hence M2. It could be pretty bad without moderation, in fact I think the fact that moderation exists even helps those who don't use it at all, because it probably discourages a lot of people from even trying crap.

    "The number of suckers born each minute doubles every 18 months."
  • by Fastolfe (1470)
    This is brought up relatively frequently, but the problems still stand:

    How do you plan on banning these people? Just ban the user account?

    What about AC's? Ban them by IP? How will you deal with dynamic IP addresses, HTTP proxies and IP masquerading/NAT?
  • I noticed that as well.. I wrote it off as being related to the 'server farm' that was supposed to be in operation..
  • Here's what my feedback would end up being:

    1) MetaModeration is good. Once a day, 10 random comments.

    2) Moderation points / likelyhood SHOULD be based on overall Karma rating. High Karma means you give more to the community, and have more of a right to shape it. Not only that, but you're probrably a little smarter then the cowards.. ;-P

    3) Allow higher Karma posters to automagically post at at a Score of '2' IF THEY CHOOSE. Shouldn't ALWAYS be 2, only when they think it's needed. The ratio of self-ups should be checked, and possibly taken away if Self Moderating on to many posts..
  • While I don't feel that context is so terribly important for meta moderation in general, it is apprent to me that it is impossible to properly metamoderate any comment moderated as "Redundant" without context.

    --

  • Err... sort of... I frankly DONT like the idea of making only SOME comments moderatable. The system works well enough now, and will be better when M2 is a part of the article, but restricting moderators further seems unneccesary. Most of us are big kids now and know what and how to moderate, and the system can fix any evils done by the BAD PEOPLE, by way of the GOOD PEOPLE and their moderation points. To make some comments un-moderatable would hurt the GOOD PEOPLE exactly as much as the BAD PEOPLE, and in the long run won't really do anything.
    ---
  • One, the entire moderstion system, including documentation, needs to be documented on the site. Looking through all of the posts here I'm astounded at how many people don't seem to understand the system, and therefore are requesting features which already exist. I think that if the whole system were very thoroughly documented, these debates about how best to implement a moderation system would st least slow down if not stop, because there would be one page where people could go, read up on the system, and understand it.

    Second, there are complaints of a "lack of moderation." There's a reason for that, but increasing the number of moderators is not going to help. The problem lies in the fact that you can't post in a thread you've moderated, and vice versa. People who post their insignts to a thread lose the ability to moderate. People who consider moderating back down because they might want to post something. I remember the debates about the idea of "moderate XOR post" before it was implemented, and I agreed with it at the time, but I see now that it just isn't working.

    At least in that aspect, we should go back to the idea that you can moderate and post in the same thread, but you cannot moderate your own posts. At least for people with karma over, say, 15 (enough to show a history of good, responsible posting) this would end that problem.

    Third: It's already required for a person to send in an e-mail address when registering. Perhaps it would be best if only one account were allowed per e-mail address. Couple this with the idea posted earlier in this thread that a person whose post gets moderated to Flamebait or Troll (this would have to be changed to require a consensus of three moderators in order to change a person's post to Troll status) cannot post anything more until the next day. The result would help to cut down on trolling, since a person cannot simply create another account and continue on. Since there's already a "Post Anonymously" checkbox there isn't a need for more than one account per person anyway. E-mail addresses can't be faked, since the password is sent to the e-mail address given when a person registers (so an account with a faked address is useless since you can't log on).

    Finally, the thing about keeping Karma on a person's User Info page: people like to monitor how they're doing. I myself was quite surprised when I looked at my own score. It's a nice thing to see for those of us with high Karma, and if Karma starts to drop it's a good wake-up call. For those with low Karma, there's something to be said for watching one's progress as one claws one's way out of that trap. It's for the best that people can see what their Karma score is; please don't change it.
  • Many moderations are questionable, but rarely do I read one and think "What a moron". But they are there, and I want to try to make it happen less.

    The stupid moderations are easy to metamoderate.

    But it is difficult for me if I get a comment presented for metamoderation that got an interesting or insightful mod and I can't find it exciting at all.

    Such happened with an Amiga posting and a Linux posting this morning. In both cases I decided not to change anything, despite thinking that the posts were just normal level and tried very hard to see the postings not from my personal interests but more how Amiga or Linux users might see them. But I am not sure if this is the right thing.

  • So here are the ideas: ... 2.When you have moderator access, only display moderate controls on say 20% of the comments.

    It makes sense: negative karma users would never moderate, and more karma would give you more choice in your moderation. It would cripple good moderators because they might see a bad comment but be helpless to moderate it, However since there are 5x more moderators, the odds are someone else will get it.

    No, it doesn't make sense. I hate this idea. When I moderate, it tends to take up a lot of time, browsing at -1, reading all the comments to recognize duplicates, and reading all the replies (I read FLAT not threaded) to catch the gems. But I only do this for news that interests me because I just have a limited amount of time available. With only 20% or so of news stories available, chances are I'd never use my points at all. In the three days given to use your points right now, I usually just get my points used before they expire.

  • It seems like ratings are getting very bi-polar around here. Posts that are a little bit better-than-average are spiking out at +5, while ones that are just slightly sub-par are getting -1's. Why? Because moderators are interested in showing how much they agree with the other moderators, in hopes of increasing their karma.

    I gave this some thought yesterday while I was mowing my lawn, and I came up with:

    • If someone is logged in with moderator status, or they are surfing as A.C.'s, then don't display numeric ratings on the posts. This way moderators can assigned ratings based purely on a post's merits (or lack thereof) and not based on what another moderator has to say.
    • Don't show the actual karma rating on the user pages; the sign of the number should be sufficient. That way, you could see whether your karma is good or bad, but not the magnitude. This isn't a race, after all..

    That's just my dollar/50. BTW, I've been a moderator a few times recently, so I'm not taking potshots at a different group of people.

  • why not just assume that the 80% good people will outweigh/undo the effects of the 20% bad?

    Admittedly, 5 moderator points/day helps limit the damage that bad moderators can cause. However, it also limits the benefit that good moderators can bring. I don't think it fundamentally changes anything. It is sort of like a minimizing lens; the overall ratio is the same, but the effect is reduced.

    I don't remember why the moderator point limit was implemented. It's possible it may have been done in reaction to abuse that was not being corrected. If that's the case, disregard this idea:

    Why not try a week or two with UNLIMITED moderator points? That is, if you are selected as a moderator, you have no limit on how many points you use. Keep the number of moderators about the same, just give them unlimited, tyrannical power. :-) But track M2 points carefully -- if people get more than a few negative M2 points, sharply decrease their frequency of moderation.

    I think what will happen is that yes, there will be some abuse, but that good moderators will correct that abuse, and that the meta-moderation will (after awhile) shut down the abusers.

    If that's no good, your idea of 'a chance' to moderate any given comment is a possibility -- I'm thinking a 5% chance per positive karma point. That way, 20 karma means you can moderate any post. Most likely, anyone who has 20 karma is going to have a clue. :-)

    I have meta-moderated a few times, and I don't think I have chosen 'unfair' once. In some cases I probably would have chosen differently, but I have not seen ANY examples of grievous boneheadedness. (I don't mark down moderators for making different choices than I would, as long as their choice seems at least faintly reasonable.)

    It also seems like the slant on pro-open-source and pro-Linux tends to artificially inflate some posts. I have written posts pointing out limitations of open source (most recently in regard to open docs), and also have written ones in favor of it where I think it really helps. Even though (in my opinion), the open-docs-won't-happen argument was better thought out than most of what I've written, it got no positive moderation at all. Other posts I have put up have gotten a lot of positive moderation, even though they weren't nearly as good, IMO. Maybe I'm too close to the problem, but personally I thought the negative-slant article should have scored higher.

    Strikes me as a good idea to remind moderators to be aware of their own biases. I try to correct for mine when I'm moderating. I'm not sure how well I succeed, mind you, but I do try. :-)
  • by luge (4808) <slashdot@ti e g uy.org> on Tuesday September 14, 1999 @06:25AM (#1682400) Homepage
    Yeah. I hate to be elitist about this, but it seems pretty reasonable. I might also suggest that humor points not be counted for karma purposes- while I enjoy reading funny posts, the authors of said posts (which are much more likely than a good, serious post to get a 5) really shouldn't get so much karma credit for it.
    Keep up the good work, Rob- someday, people will study your code to see how to make internet conversations that work for large numbers of people. I do believe that this is the first time anyone has seriously tried to make anything like this work- and the quasi-experimentation that you have done will be a source for the next group that tries to apply this technique to more serious fields of endeavour.
  • Well you are in luck because I have even been thinking of a scalable implementation of the rating network stuff. I'll try to layout some database tables for you but please excuse me if my explanation isn't perfectly clear.

    One table holds all of the comments. The table has the userid of the author, the comment id of the comment, a score, and the total number of votes cast.

    Another table has all of the authors with a userid, a score, and the total number of votes cast for that author.

    A third table holds the individual scores for each author by each user. The worst case for this table is that it grows to n^2 where n is all of the users on slashdot. In practice it will not grow to n^2 because not everyone will vote for every author.

    A fourth table holds all of the individual votes for an article. Assuming there are n readers and p articles, the potential size of this table is p*n. That shouldn't be too bad as most people don't post articles and only some people will vote for every article.

    There are two things which pin the whole system together. For purposes of calculating an article's score, the user's own votes get a weight of 10 out of 10. The user can specify the weight of the community vote on a scale from 0 to 10. So as you count the score for the article, you multiply your own vote for that author by 10. Other authors that you have previously ranked get weighted depending on how you ranked them, and the rest of everybody gets the wieght that you grant to the public at large. Any article that can't be assigned a personal score just gets the community score. Every article always has at least one vote: when the author posts the comment is automatically scored to some constant, perhaps 10/10.

    Anyway as you can see I've been modelling this for some time. With the right indexing scheme and efficient SQL, this should scale really well. It would be a bigger load over time, but over time computing power is increasing so you come out even.

    Regards,
    jwb
  • What is your opinion on this subject if you take it outside of the context of HTML delivered over HTTP? You say that moderation is a chore, and I completely agree! But the reason I think moderation is a chore is because 1) all of those select boxes make the page load and render slowly 2) When I wont to moderate I have to wait for the page to reload.

    I believe that both of these problems would go away with the use of a custom desktop client. Imagine that you are sitting in front of your computer using KSlashClient. To vote on an article, all you have to do is click on a gradient strip. Full left is 0, full right is 10. When you click, your vote is transmitted to the discussion server adn registered, but you don't have to wait for anything. The vote is delivered asyncronously while you continue to read (and rate) the rest of the comments. Suddenly moderation isn't a chore any more.

    Now, let's take your pathological case. Little Johnny has the day off from 3rd grade and is rating all of the flames 10 out of 10. Nobody else is around so them flames are the highest ranked articles in the discussion.

    When all of the hard-working rational adults logon to read slashdot, they immediately see "Gnome SUCKS KDE RULES!!!!! (Score 10)" and they think "WTF?" So they click on the zero end of their voting scale and read on. I believe that there are a lot more good, thinking people on slashdot then there are vapid flamers, so the result would be an immediate and decisive downgrading of the flames to the bottom of the list.

    As a benefit, since Johnny's personal score is 0.2 and the rational adults all have 6-10, Johnny's votes are easily overpowered.

    What do you think?

    -jwb
  • Actually, wild fluctuations are what happens now! A moderator moderates an article up to 5, then it goes back down to three, then up to four, down again, until the moderators get bored.

    In my system, the scores are *much* more stable. For example, suppose 1000 people have given an article an average score of 5/10. Now another user comes along and gives the article a ranking of 10/10. What is the article's score? Still 5/10.

    One of the best parts of this system is it's stability.

    -jwb
  • by Jeffrey Baker (6191) on Tuesday September 14, 1999 @06:33AM (#1682415)
    Rob,

    Please accept my humble criticism of slashdot's comment system. I have been thinking about this system a lot lately and designing a replacement for it. I have no delusions of replacing slashdot: slashdot is a community that would be hard to duplicate. The only problem I have with slashdot is the comment mechanism, and I will present those problems and some possible solutions.

    Problem: Moderation race conditions. Joe and Jane are both moderators reading a slashdot article. They both see a comment that they think is interesting, but not incredible. They both choose to spend one moderator point to raise the article to "Score: 2 (interesting)". They continue to read the comments and click the "moderate" button a few minutes later. Now the article is rated 3, but neither of them thought it was worthy of 3. Moderation has failed for both of them. One of them could choose to moderate it back down to 2, but then the article would be rated "Score: 2 (overrated)", which does not reflect their true opinion of the article.

    Solution: Instead of incremental ratings (+1, -1), allow the moderator to score the article on a range, say from 0 to 5. Then if two moderators assign a score of 2, the article's score is 2, not 3.

    Problem: Only a few people have a few moderation points at once. This leads to several worst-case failures. One of them is that no moderators will read a given article, and the comments in that article will be clustered around 0 and 1, making the "Show highest scores first" setting much less useful.

    Solution: Allow all readers to be moderators all of the time. There will never be a deficiency in the number of moderators with this method. The number of people scoring articles will be directly proportional to the number of people reading that article. When 100% of the participants can cast votes, there is no outlying possibility if lack of moderation.

    Problem: The community ranking does not allow for individual preferences. The individual may have a strong opinion for or against a certain author, but the score given by the moderators is unlikely to reflect his preference.

    Solution: Use a network of preferences to tailor the rankings that a user sees and to delegate the ranking of articles.

    I believe that the ideal system would allow the user to cast a vote on every comment posted. The comment display system would use the user's voting history to score and sort the comments on every new article. Take an example:

    Joe, John, Jack, Jerry, Jane, and Jennifer are all users. In the past, Joe has given articles by Jack an average rating of 4.2 (out of 5), and has given John an average ranking of 3.8. Joe has consistently rated Jerry at 0. Joe, John, and Jerry all have overall ratings (the average rating given by all the system's users) of 2.

    In a new article, Jane, who has never posted a comment before, posts an unusually insightful comment. John, Jack, and Jerry all post their usual stuff. All three of them rate Jane's article at 5, but the overall community ranking for Jane is only 3. When Joe reads the comments for the article, the article display system will give John and Jacks votes more weight than the community vote, because Joe has rated those two authors highly. The system will completely ignore Jerry's votes because Joe doesn't think much of him.

    The result is that Jane's article (along with John's and Jack's) are listed at the top of the page, instead of lost in the middle of the pack. Other users with completely different preferences will get completely different displays. When there are no personal votes for an author or article, the community ranking can be used to score the article.

    Whew, I hope that example was clear. It really isn't very complex or even original. It is just like what firefly was doing 5 years ago, but with a little less complexity and a lot more scalability. If slashdot were to implement a reputation management system like this, I think it would be a much better place.

    Regards,
    jwb
  • by Ignatius (6850) on Tuesday September 14, 1999 @07:29AM (#1682416)
    It stikes me, that moderators get 10 M2 points per day but merely about 5 regular mod. points per month(!) to do their job in the first place. Using 98% of the pontential ressources for control and merely 2% for the actual work might be acceptable when handling nuclear weapons or biotoxins, but is IMHO total overkill when it comes to managing voluntary moderator work for an online forum, esp. since the exactly same information as provided by the M2 scheme could easily been extracted by better analysing comment moderations. Here's how:

    Hypothesis 1: The majority of moderators are fair and take thir job seriously.

    Hypothesis 2: If one moderator rates a certain comment up, and another moderator rates the same comment down, it is reasonable to assume, that they would equally inclined to consider the other's decision as unfair.

    Conclusion 1: The ammount of mutually nullifying moderator decisions on a certain post provides a measure for the total 'unfairness' p of a comments moderation, where p = 2*min(#_of_ups,#_of_downs). If hypothesis 2 is true, then this p should be proportional to the karma penalty generated by the current moderation scheme.

    Conclusion 2: Since we assume that there are more good moderators than bad ones, it is reasonable to assume that in a contoversial comment moderation the majority has a higher propability to be "right" than the minority (provided that there are more than 2 moderation decisins involved)

    Proposed Solution: In the case of a controversial comment (i.e. a comment with up *and* down votes), generate a (very small) karma penalty proportional to p and distribute #ups/(#ups+#downs) of the penalty among the down-voters and #downs/(#ups+#downs) among the up-voters.

    Example: A comment gets 4 votes up and 2 votes down. Each up-voter gets 1/12 (33% total) and each down voter gets 1/3 (67% total) of the total penalty which ammounts to 2*min(4,2)=4 penalty points (which should IMHO be somewhere around 1/5 to 1/10 of a current karma point)

    In order for this system to work, it would be necessary to increase the total ammount of moderation points floating around and it might be reasonable to give some karma reward for each moderation to compensate for the total penalty generated by this system (so that in the end only the guys who regularily make minority decisions take the punishment).

    Of course, the used formulas are only a proposal and fine tuning could certainly increase the overall efficiency of the system, once empirical data is available.
  • The problem is that Slashdot's population grew, as did the number of the lamers. A long time ago, the noise was virtually nonexistent, and Rob could just weed out the little bits of it that did exist. Now, when Slashdot has become almost a family name, the amount of lamers is bigger than ever.

    --

  • Say you normally get your comments moderated up to, say, 3. Pretty good. But then suppose you just want to provide a little bit of information, or a brief comment; something that wouldn't warrant a 3. It would get the 3 anyway; which might also trigger the Highlight Threshold for some people (depending on individuals' settings for this).

    One perhaps better way would be to take the average of past comments and create a separate variable, maybe call it AverageScore. Then give users a choice in their prefs whether to add the AverageScore to the Score, or add a fraction of the AverageScore to the Score (I envision 1/2 or 1/3 of AverageScore, rounded to an integer), or do nothing with AverageScore.

    Another consideration with such a system is that Slashdot would have to find the AverageScore of every user who posted on a given page, do manipulations based on the viewer's prefs, and use that in the dynamic generation of the page. This could be more load on the server than any value we might get out of it.

    Personally, I don't see a need to have a historical score influencing a present score. The moderation works pretty well at bringing out the good stuff and eliminating the crap. Adding a predictor value like AverageScore seems redundant at the least, and it could interfere in an undesirable way with the moderation process.
  • You raise an interesting and valid point, which is that the {Over,Under}rated specifier is redundant with respect to M2. I think the solution should be to remove OR/UR from the regular moderation menus, and let MetaModerators determine how well or poorly a post was moderated.
  • A poster before me has asked "how" you propose to do this. I'd like to ask "why?"

    At the risk of seeming like a pedantic pointer-out of the obvious, you can "limit the amount of noise and increase the signal" by your very own self, by setting the filter to (e.g.) +2.

    Personal aside: Your tone, as evinced by comments like "Ever since /. has started getting overly popular", smacks of arrogance and old-timer-ism. Does anyone care how few digits are in your Slashdot uid? (I don't.) Rather than moaning about how everything sucks now that the newbies are here--shades of Usenet 1993!--why not moderate and meta-moderate (as applicable), contribute stories, post sensible things that are likely to be well regarded, and otherwise do your part to make Slashdot a better place?
  • M2 moderation acts as a disincentive for me to moderate as my karma is (was) over 10.

    I have been wondering why my karma has been gravitating towards 10, now that you mention M2 moderation doesn't take someones score outside the -10..10 range I understand why.

    Since my karma is (was) over 10 meta-moderation applied to my moderation can only drive my karma down towards 10 (well it tends to oscillate between 9 and 10). When people disagree with my moderation I lose points, when someone agrees with my moderation nothing happens.

    Actually I remember on a previous moderation story an AC complainined about their karma rapidly dropping (a few points a day) and that after a week the would hit zero. They wrote back the next day saying that there score seemed to have stopped decreasing, I guess they finally dropped to 10 as well.
  • I'm quite sure that the karma does not have a half-life.

    Everyone I've spoken to regarding MetaModeration has watched their karma plummet through the floor. It appears to take plunges soon after the user completes a batch of moderation - mine, for instance, dropped 3 points on the first day moderation was enabled (I was a moderator that day), and five points the next day I moderated. Other users have reported similar patterns.

    I am convinced that massive negative meta-moderation of my moderation is causing this, and as a result, I am now afraid to moderate. In the last week, I've had one +4 post and one +5 post, and my karma is still five points lower than it was seven days ago.

    What we're seeing here is moderators getting afraid to moderate (such as me - I keep doing it anyway 'cause my Karma is quite high, but I will stop when I'm within range of zero). This is not good.

    The Meta-Moderation system is flawed in that it subtracts a full point for every negative vote, but does not add an equal amount for every positive vote. There is no way around this flaw, however: to add a positive integer for every positive vote to your moderation would probably cause the opposite; people's karma rocketing skyward.

    I'm all for scrapping meta-moderation. Here's my replacement system:

    Give each moderator a number of moderation points proportional to his or her karmic level.

    The highly-ranked people have obviously got a handle on how to write a good comment. Wouldn't they, therefore, be the best people to tell good comments? Give them the weight they deserve. And keep all the other people in the system at a lower level to help out.

  • I have just one question:

    Has anybody seen or felt any improvement in the atmosphere at /. since the M2 system was implemented?

  • I disagree with the idea of hiding scores when moderating. When I moderate, I try to invest my points in posts that I feel are not scored as well as they should be. I look for AC posts that are still at 0 and really should be higher. I'd hate to waste the few points I have on something that's already a +5.
  • Not sure I see the distinction.

    The way the NetP engine that Movielens uses works is there are users and items.

    The users rate the items, and then the system uses this data to predict your ratings on other items. The "profile" would just be the set of all ratings you have done, and so yes, comparing this profile with the profiles of other users is what allows the system to predict which users you agree with.

    Now I suppose that the system you are proposing is slightly different in that you are rating other moderator's moderations instead of coments directly. But if a moderator gives a given post a +1, and then you rate that person's rating a -1, isn't that essentially the same as you rating the comment -1?

    In any event, it's a short leap from your idea to a full CF model, and that would eliminate the need for designated moderators. It is, however, a slow, expensive, non-portable thing to set up, so I doubt Rob would consider doing it. I wouldn't.
  • I don't see why I haven't been given the opportunity to moderate yet.
    Don't worry about it. I'm sure it's nothing personal. I don't post much, but have moderator points for the second time this week. There's a random factor in it, I guess.

    Either way, moderating /. should be taken seriously, and it filters out the worst noise, but it's not the be-all-end-all of the /. experience.

    It's definitely not something you should get upset about. It'll float your way sometime soon; after that, the glamour 'll wear off pretty quickly. IMO, of course.
    --

  • One more way of displaying a discussion would be nice. We can now look in order of post, or by score, either with threads or without...

    For moderating purposes, it might be useful to see the most busy threads first (or last) to see wether there is a flame-war raging, or a hot discussion (of value) brewing...

    Just a thought.
  • I very much like the idea of judging the overall worth of a posting (-1 to +5) rather than adding a point to it's value... But, a comment can't possibly go from 0 or 2 to 5 just because I said so...

    So how about a compromise:

    Moderators still get moderation quanta (I vote for fractions of points, see my rant at #33). They then get to choose what worth a post has, and their quantum adds (or subtracts) the actual value of the post in that direction.

    In this way, the overall moderator opinion of worth, coupled with their quanta towards that end, result in a post that is moderated to the average of all moderator opinions... Preferably where it started. :)

    It's another level of complexity, and should keep Rob busy for weeks in trying to figure out how to implement it. [VBSEG]
  • by jabber (13196) on Tuesday September 14, 1999 @06:40AM (#1682457) Homepage
    Some of the Funny ratings are quite pertinent to the discussion at hand though. And a humorous attitude about something tends to show understanding of the subject... Well, to me at least. So a person with enough understanding to make a joke is, in effect, making a worthwhile contribution.

    Maybe we should have an additional category? For relevant humor or offtopic humor? But here I diverge from my personal opinion that simpler is better...
  • by jabber (13196) on Tuesday September 14, 1999 @06:43AM (#1682458) Homepage
    IMHO, the 'Overrated' and 'Underrated' moderation items should have the effect of MetaModeration. These are, essencially, means by which moderators can correct one another's work.
  • by jabber (13196) on Tuesday September 14, 1999 @06:29AM (#1682459) Homepage
    First off, MM is not really necessary.

    A good comment is good. It is unlikely that a single, rabid individual, will sink a good comment into oblivion, without an equally determined benefactor raising it up again. By that same token, a conspiracy of evildoers to elevate each other's trash, whenever one of them has moderator access, is slim to none.

    Occam's Razor applies, and a system of lesser complexity is preferable to one of greater complexity. Slashdot should not hamstring itself with rules, and rules about rules, else it will become a burocracy in which nothing ever happens.

    Personally, I've never used MM, but I would rather see it go away - simply because Moderation points strike me as a rare commodity, so their negative effect seems to be outweighed by their utility to the holder. Also, with /. being a self-correcting mechanism already, they just add another layer.

    MM is an interesting exercise for Rob, it nothing else, and as such it is as useful as anything that keeps a mind from being idle. :) I, for one, would rather see Rob bring his considerable talents to bear on extending the article discussion forums. On many occasions, a great off-topic discussion has been sparked by an article. This discussion either flops for being off-topic, or dies when the article expires from the main page. It would be nice to have some quantifiable worth of such a discussion, that would enable it to live on until it dies a natural death.

    As for a redoux of the moderation philosophy, how about fractional moderation? Increase the number of moderators, certainly, maybe by a factor of four. But reduce the worth of each assignable point by that same amount. This way, several moderators would have to agree to elevate a post to the next higher integer score. The ability to abuse moderator privilege goes away, as does the confidence that a single vote of one individual will make a lot of difference.

    As moderators, we would then add our voice to a comment, not knowing if we're the first, or the decisive ones to raise the message status. Follow?

    This way, good posts will rise, bad posts will sink, and iffy posts will tent to teeter-totter on the edge of where they started.

    Ah, all this opinionated rambling has tired me out. But, before my hands completely cramp... How's about making all posts require a Preview??
  • I think that it is a bad idea to restrict moderation excessively. When I have moderator access, I change my prefs to browse at low scores, and look for new comments (i.e. ones that probably haven't haven't been moderated yet). That way comments that deserve to be moderated up get there. Comments that are at 3 or 4 don't need that much help.

    However, once I'm done moderating, I switch the preferences back to a more filtered view, and go back and re-read the articles looking for the content, as usual.

    I think it is *essential* that moderators be allowed to pick which comments they moderate during the 3 days they have their points. Giving moderator access more or less frequently based on karma is a perfectly good idea, and could achieve some of the same effect.

    Finally, I would like to just throw a reality check out there. Slashdot should be good, but there is *no* way to weed out all the a-holes, flamers, trolls, etc. Slashdot already does a very good job of tagging their comments so that people like me who don't generally want to see them don't have to. I think that, at this point, the moderation system is sufficiently effective, and time would be better spent squashing some of those SQL problems, and implementing some of the great ideas for additional forms of content that could be added.
  • by maphew (14702) on Tuesday September 14, 1999 @09:54AM (#1682467) Homepage Journal
    Actually, why not make karma be an average of post moderations?

    And what of the lurkers? Just because a person doesn't post doesn't mean they wouldn't make a good moderator.

    Actually Rob, how many users are there who haven't posted, but still read regularily?

  • by grappler (14976) on Tuesday September 14, 1999 @10:28AM (#1682468) Homepage
    if they tie it too closely, some people will make slashdot into a big game, where the object is to get as many karma points as you can so that you can have the most influence you can, both in your moderation and high initial comment scores. I know some people already think of it this way, but it's not something to be encouraged.
  • A number of comments, somewhat scattered.

    First, I don't envy you. It's a no-win situation, and it does indeed take the innocence of youth to put up with something like this.

    Second, I'm still confused. I have a post here ( The heights of low taste ) [slashdot.org] that has been moderated more strangely than any other post I've ever made. It has 1 Troll, 2 Informative, 1 Overrated, and 1 Underrated. I can understand the over rating, but the troll moderation is somewhat curious. What rates as a troll message? And please don't say "your message" :).

    Third, I think the real-time meta-moderation next to each message would be a very nice touch, if you can make sure that an even number of meta moderation tokens are handed out fairly. Having the seperate page is just fine by me, as I can go off to the various threads and check context if I have a question. Like you, I've never looked at any of the moderated messages and thought the moderator subhuman based on their moderation, and it's been my experience that the number of moderations I disagree with is around 5-10%.

    Finally, don't try to hard to make this perfect. It never will be, and trying will only make the system more convoluted, complex, and fragile. I really do have a life, and slashdot is nice to have and a lot of fun at times, but I can live without it. Just like I'm sure you can.
  • I find that a lot of posts are marked as "troll", and upon reading them, they are in fact not trolls at all. Do a lot of the people moderating not fully understand what a troll is?

    Perhaps we should quote the Jargon File:

    troll
    troll /v.,n./ [From the Usenet group alt.folklore.urban] To utter a posting on Usenet designed to attract predictable responses or flames. Derives from the phrase "trolling for newbies" which in turn comes from mainstream "trolling", a style of fishing in which one trails bait through a likely spot hoping for a bite. The well-constructed troll is a post that induces lots of newbies and flamers to make themselves look even more clueless than they already do, while subtly conveying to the more savvy and experienced that it is in fact a deliberate troll. If you don't fall for the joke, you get to be in on it.

    Some people claim that the troll is properly a narrower category than flame bait, that a troll is categorized by containing some assertion that is wrong but not overtly controversial.



    Let's define FlameBait while we're at it:

    flame bait
    flame bait /n./ A posting intended to trigger a flame war, or one that invites flames in reply. See also troll.


    All I really ask is that, when a person is moderating, keep these ideas in mind. Don't mark something stupid a troll just because it's stupid. Stick to the definitions of the terms. If you simply want to mark something up or down, use over/under-rated..



    ---
  • 1.Put 5x (Numbers are Variables here: Don't complain about them!) as many moderators into the system.

    2.When you have moderator access, only display moderate controls on say 20% of the comments.

    I'm not convinced that moderator dilution is the answer here. I would expect that by spreading out the pool of moderators, Slashdot would see even more problems with badly moderated posts. What we want is instead a smaller pool of people. but people who moderate more thoughtfully.

    But how to distinguish thoughtful moderators from irresponsible ones? I like the "karma" system since it provides more personal accountability. I don't think it has been in use long enough to really show its potential, but people who regularly post articles that other people like should have a chance to participate more in the moderation process

    Here are some thoughts:

    1. Create a fixed-size "pool" of moderation points (or perhaps make it a percentage of the total number of articles?)
    2. Dole these out to a predetermined number of people with high-karma folks getting more moderation points. (Basically option 3 as mentioned in the article.)
    3. Create a system where a very few of the very best people on slashdot can serve as uber-moderators. These people would have the power to permanently ban people from participating in moderation. (Probably best to choose these people by hand.)

    With that said, I feel that the present system is working VERY well. It is extremely rare that I see a post moderated up which didn't really deserve it. Even better, by setting high thresholds, I can browse very quickly and still get some valuable insights on the articles.

    Good work, guys!

  • by lisa (19611) on Tuesday September 14, 1999 @06:12AM (#1682495) Homepage Journal
    I *really* like the idea of integrated M2 moderation in the comments section.

    I can't tell you how many times I've thought the moderators-in the context of a comments thread, were either clueless or not paying attention or just plain dumb. However, with the current M2 setup, I don't understand the context and don't really want to take the time to go look. I am already taking the time to do the extra moderation.

    When I'm not a moderator, it would still be nice to feel I am having some sort of role in the moderation process-even if its just keeping a check on the moderators.

    Problems I foresee-Someone moderates a comment, and immediately the moderation is moderated (jeez, i should choose a new word) to be Unfair. So does the moderator go back and change it? Is that unfair pressure? And if you M2 a comment in a thread, are you unable to participate in that discussion?

    I like the idea of Karma. Though I think it would be cooler to see it on a scale when presented on the user info page. I pay quite a bit of attention to this site, but I still don't remember the scale offhand. A graphical bar would be nice. And instead of just numbers, relate the the numbers to real world behavior. ie. karma of -1 is called 'troll' or something like that. Be creative!

    -Lisa
  • by AeiwiMaster (20560) on Tuesday September 14, 1999 @07:46AM (#1682496)
    I have noticed some small problems with the moderation system.
    They might be connected.

    1) Good messages not posted fast after the news are not moderated up.

    2) Points used on 5 points messages don't add to the story points
    but decrees your moderation points.
    This mean lesser points for messages in 1)

    3) There is no way to view unmoderated messages
    when you moderate.


    I think that a solution to 2 and 3 will help on 1.

    Knud

    Aeiwi [aeiwi.com] is a search tool with a unique interface.
    Which let users add more search terms,
    until they have a small number of results.


  • Permit me to disagree. First, nobody is going to spend a moderation point to downgrade a -1 post, or upgrade a +5 post. Second, I don't know how typical I am, but I moderate posts when my opinion of the post disagrees with its rating (as in: this is at least a +3 post and it's only at +1 -- upgrade!). If I like the post, but it is already at what I believe is its proper level, I'll just let it be. Given that, I think that the suggestion to let moderators directly set the rating (on a -1 to +5 scale) has merit.

    And, BTW, what is karma's expiration period? Clearly there is one, but how long is it?

    Kaa
  • by the_ed (22189) on Tuesday September 14, 1999 @06:16AM (#1682504) Homepage
    Mostly, this sounds like a good thing. I completely agree that no one user should have too much power, but my gripe is here:

    As a moderator, you should have the ability to moderate where it is needed, and not have your moderating ability limited to randomly selected comments.

    If a moderator sees a particularly outstanding comment, they should be able to moderate it, and not be hindered by a random 20% factor. If you are going to increase the number of moderators, do so at the expense of moderation points, not at the expense of the ability to moderate.

    And I do have to agree with Rob: if you don't like the system he's worked so hard to build, you can get around it without asking him to take it away.

  • ...is to have Rob ban, for just a day or two, trolls and flamebait. Maybe not based on some moderators comments, but on what Rob/Hemos/Katz/crew actually read. Ever since /. has started getting overly popular (ie, stories in some major online publications), there has been an increasing amount of noise in here. Something that will limit the amount of noise and increase the signal would be much appreciated. I think I'm going to puke if I see anymore comments that are trying to contribute nothing to the discussion except insults and flames.

    Phew... that felt good to get that off my chest. Other than that, I think the moderation is going OK.

  • Accually I'm quite surprised you havn't been able to moderate.. Are you sure the moderation option is checked in you user prefereces? But then again, maby it is the obsessive thing. As Rob did say that can be a reason for no receiving moderation access.
  • Yes, but generally in a mass of posts one will get more negative moderation than positive. So it works out fine in the end.
  • "I don't care if you turn stuff off. It won't hurt my feelings at all!"

    The people who complain about the point system aren't complaining because they don't like to look at the numerals 0-9, so turning it off won't assuage them. Their complaint is that certain people can't be heard because of
    • Insufficient time spent on /.
    • An unwillingness to reveal their name/address OR lie about it
    • Bad moderation
    • Other

    Obviously the users who complain can still view those comments, but that's not the issue. The issue is that Slashdot is a forum and if, for any reason, certain people can't be heard from then the forum becomes self-defeating.

    Now then, at this point I'd like you to re-read my subject line. "Devil's Advocate" means I don't necessarily hold these views myself. As it happens I did and to some extent still do. But I've come to realize that high noise levels defeat the forum as well and something had to be done.

    However, I think it is important to remember that we don't have to create rules just to have rules. Nor do we have to create rules just to have something to write a program about. All that said, I have a comment about M2.

    I don't think we need it. Bad Moderators have only one effect: Lowering the scores of good comments. The solution is two-fold:
    1. Have "default score viewing level" be a criterion for choosing moderators (i.e. pick some moderators who read at the -1 level)
    2. Increase the number of moderators

      1. The first step ensures that there are moderators who are reading the unfairly moderated comments at the -1 level. And the second step ensures that there are enough moderators to raise those scores up.
        ---
        Put Hemos through English 101!
        "An armed society is a polite society" -- Robert Heinlein
  • Moderating is a heavy responsibility

    This article was posted at 12:11. By the time I got to it, there were -1:163 comments. When I got through those and refreshed with "newest first" there were 50+ more.

    So, this post will probably get completely lost in the shuffle. My only hope is that Rob may read it, since he posed the question and actually cares about his community. The up-side is that I have all the previous comments to try to pull together.

    First, when I've noticed that I had moderator points, I have tried to switch to -1, flat mode to be as fair and impartial as possible, but, if I were to try to moderate a topic such as this I'd have to quit my day job. A couple of things would help this:

    • Highly visible moderator notice: Turn the banner red, auto-switch prefs to -1, flat, oldest first (this should be a requirement for moderation anyway), or else-wise slap me to get my attention. I'd make more effort to go off the headlines and into the comments.
    • Don't expire points so quickly: Those of us who hit /. throughout the workday are more likely to glance at the headlines and get back to work. Give us a few evenings and weekends to spend our points, or am I the only one who actually uses the "Older Stuff" box?. I dispute the argument that moderators, or anyone else, usually only see the first 50 or so posts. I RARELY get to a story on /. that quickly. I usually tag the headlines during the workday and pick up the comments later, when I have time. If anyone is checking them ONLY that quickly, and not coming back later, they are missing some great comments!

    Random Moderation

    Nope. Been said all over this page. Moderation would be less of a responsibility and less compelling for those of us who care about the impact to the /. community. It wouldn't be worth the effort.

    Meta-Moderation

    I think M2 may work in 2 directions. Instead of just ranking the moderators, it could also rank your opinion of the moderators and be another factor in your preferences. I'd like to see the M2 boxes on all the comments. I don't think they'd be used that much, but when they were used, they'd be used on comments that people really care about. This should address the problem of moderators with agendas. Let's face it, if someone has an agenda, that happens to agree with mine, than that's a good thing to me.

    Fractional Moderation

    This is a good idea, both for moderation and for M2. It would take at least 2 moderators to change the moderation.

    Unscored Moderation

    Some people have pointed out that the categories available are not necessarily complete (like missing stupid), approriate (OR/UR being more relevant to M2), or positive or negative (some people LIKE Trolls, some don't want Funny). So:

    • First expand the list of tags available. Forget -1:+5, and just keep a list of tags, and the number of each type of tag on the comment.
    • Allow users to specify both categories and thresholds in their prefs. e.g. I'll take Interesting:2, Funny:4, Insightful:3, Normal:0 and Troll:5, in that order.

    Karma

    Rob says it doesn't factor that much anyway, but I think it should be affected by only the most positive and negative moderations and M2. I know I said no numbers, but, while one person's Flamebait is another person's Funny, Offtopic is not good. Redundant is irrelevant, since everyone has different views (prefs., NOT opinions) it's probably not redundant to readers with different preferences.

    Maybe instead of adding +1 to future posts, Karma could add personality tags. If a poster consistently gets Interesting:2 or higher, then that person is probably interesting. These tags should be clearly distinct from moderation tags, so the reader and moderators knows that it applies to the author and, not necessarily, to the post.

    A Changing Face

    This should keep Rob busy for a year or so, but imagine the possibilities. For a discussion about webcams in public restrooms, I could view at Funny:2, Interesting:3. For a discussion on security, I could use Insightful:2, Interesting:2.

    Since this would put my viewing entirely into the hands of moderators, I'd probably add Normal:1 to any view. Or, I could read on pass one a topic for Interesting and Insightful comments, and a second pass for Funny.

    Conclusion

    The whole point of moderation and meta-moderation is to filter the /. experience for the sake of the individual participant. Being a moderator is a pain in the [insert your desired portion of anatomy here] and it is difficult to do fairly. Try to remember that if it weren't for their efforts, you wouldn't be able filter your view at all.

    And thanks, Rob. It just keeps getting better.

  • That's not a bad point. I find that I often forget to check the box to not add my automatic +1 to posts that I'm just spitting out (like this one :-) )in response to someone else's discussion that probably, although not a bad or off-topic post, probably isn't a +2.

    Also, for other reasons, I _do_ want to see scores when I get points to moderate. Nothing I hate worse than seeing a -1 post that is neither flamebait or off-topic, just merely an opposing opinion. As long as its someone's opinion with some valid arguments, it should have as much chance to be seen as any other 'good' post. Also, posts that are marked up waaay too much can be knocked down.

    My feeling is that meta-moderation can be used to correct this problem, if there were only additional info in the m2 page. Sequence of moderation.

    This might be real tough to do but here's what I'd like: When presented with a comment on m2 page that has had multiple moderations, show what the post was scored at the time the moderator marked it.

    So, if a post ends up +5, when it shows up in M2 the first time, it might have an original score of 2 and went to 3. I would mark that as 'fair'.

    The next time it shows, the original score was 3 and went to 4. Maybe I leave this alone, maybe 'unfair'.

    The third time I see the post and it goes from 4 to 5 and its not a 5 comment and the moderator is just doing a 'mee-too' like has been suggested, it is markeed 'unfair'.

    Now, the moderator who put the post over the top loses karma and maybe gets less points.

    I know, but it was a thought...

    (oh and Rob, showing me my Karma points is bad, makes me hit reload waaay too much to look at my post history to see what I did right/wrong) =D

    As always, appreciate your work, Rob!

  • The problem with giving high-karma people more moderation power is that, even though they can't moderate up their own comments, they can selectively moderate up comments of their friends or those they agree with. This, in turn, increases their friends' moderation power, which will end up giving them more karma, etc.

    You make an example of a bad (GNOME-bigot) moderator moderating down 5 KDE-related posts. I think it's more likely (and I've seen on slashdot many times) that the moderator will moderate up 5 rather uninspiring or moderately interesting GNOME-related posts, and ignore more insightful KDE-related ones. I metamoderate and think "well, that wasn't THAT insightful. But I can't really say it was unfair either."

    Personally, I don't see that as an easily solvable problem; you can't metamoderate taste in what people find interesting or funny. No matter how you randomly choose people, you will probably always end up getting the same ratio of, say, pro-GPL to anti-GPL moderators (whatever the ratio is in the /. reading community). But it would make matters worse if more prevalent views (and thus, higher karma) could feedback into more moderating power.

    Oh, and I'd like to point out that I like your idea of metamoderating in the context of an article. If it wouldn't slow things down too much!

    JMC

  • by CocaCola (30016) on Tuesday September 14, 1999 @07:56AM (#1682531)
    While I agree that integrated M2 is fundamentally correct and wanted, 'random M2' has some very important properties as well: it in a sense _forces_ and ensures the moderator to take up 'independent' issues/comments and judge them. An M2 almost by definition only ever sees articles in which he is interested - this again creates inhomogenity in the distribution of M2's. Randomness distributes M2's much better between moderations, and this also makes it sure that 'bad moderators' will be caught sooner or later. Eg. if a 'malicious moderator' goes back to a KDE vs. Gnome thread, and moderates down KDE-positive comments, then this skews the _historic record_. Newcomers, analysts (and history) will be manipulated. It has a very low likelyhood that such malicious moderating will ever be detected - M2's by definition do not read old articles. (but there _are_ people who read old articles, but they are not likely to M2 moderate, or even _notice_ that the historic record has been manipulated with.)
    All in one, I very much like the current 'daily random M2', because 1) it gives me a warm fuzzy feeling that even such random samples are usually 90% well-moderated 2) I can be sure conceptually that the bad guys will be caught sooner or later, and it might be me who catches one of them. M2 is really like law enforcement, you both want to patrol on the streets but you also want to do larger scale (and more random) searches.
    The solution? Honestly I dont really know. Maybe this hybrid method: there would be a 'get me 1 random comment to M2-moderate' link right at the top, but also it would be possible to M2-moderate 'in the field'. Thus I could eg. random-M2 5 comments, and realtime-moderate another 5 comments. The 'split' between random and realtime M2 is thus up to the M2-moderator.
  • How's about making all posts require a Preview??

    How about being able to edit your own boneheaded posts? A lot of boards do this with a quick (edited by: username, time) stamp on it. Bye, bye spelling/grammatical correction posts! (note: I have largely avoided the MM discussions so if this has already been shot down I apologize)

    I think meta-moderation is a good idea (except that I get asked to meta-moderate first thing every day), even though I guess I was a bad moderator and had my karma dropped to a point where my comments no longer start at a "2"(which could have been me posting comments without the +1, which I do for quick nitpicks or other things that would most likely be moderated down to 1 for being "too high", which could be a bug or a seriously nested parenthetical aside). I am actually right on the line, if someone moderates this (or any of my other comments) up, I will return to the blessed 2 status.

    Increasing the limit could also help for the sig/noise ratio. With 4-5x as many moderators and limits up to 50 or 100 there would be a better stratifation (new word!) of comments and could actually make a Top 10 posts of the Week mini-site useful. I think there is a lot of great content here that goes unnoticed (and a lot of crap to wade through to find it).
  • by Dugan (31865) on Tuesday September 14, 1999 @10:17AM (#1682538)
    From my earliest Slashdot-lurking days, I've wished that the interesting, lively debates didn't get whisked off the front page and into oblivion so soon.

    Imagine a page or pages, linked to Slashdot's front page, called:

    - Current "Liveliest" Discussions: ordered by comments/minute
    - Week/Month/Year's Hottest Debates: ordered by total no. of comments
    - Active Debates: "old" articles with recent comments (ordered by "recency"?)

    Etc... other criteria are possible.

    Wouldn't this add a fascinating static element to Slashdot? Eventually certain discussions would emerge as "Slashdot's All-Time Great Debates" , with thousands of comments from a period of years. Set your threshold to +30 to savor the distilled, democratic wisdom of 30000 slashdotters ;-)

    What do you think?
    Dugan
  • by fremen (33537) on Tuesday September 14, 1999 @06:29AM (#1682546)
    The M2 thing seems to be going nicely, but I would like to propose an additional idea. When you moderate, you're limited to having very few subjects that detail how you feel about the message. Sometimes they don't do justice to your feelings about the post. That's why I think that it would be nice to have a one line comment detailing why you think the moderation you just made was fair. This is something that I have actually wanted in the past, and it might even help with the M2 moderation.

    Writing a comment would be optional, as many moderations are simply self explanatory. But, it could be useful to say "I moderated this down because it was repeated above in comment #xyz," or "I moderated this down because we aren't discussing foo here, only bar." Of course, positive feedback would be good, too. Good posters could be rewarded with feedback like "Good post," or "Nice job researching this subject."

    This information could then be made available on the M2 moderation page, permitting people to get an idea about why a moderation took place. A comment that might look perfectly normal could have been grossly off topic. Further, it would give the original poster an idea about why he was moderated up or down. Everybody wins.

    Of course, the moderation comments would have to have limits. Perhaps the complete history of the moderation comments would only be available to the original poster, while everyone else only saw the most recent. This would prevent silliness like whole discussion threads on why a post was moderated in such and such of a way.

    Anyway, it's just a thought.
  • by dermond (33903) on Tuesday September 14, 1999 @06:15AM (#1682548)
    when i moderated i read through an article and if i find one (or 2 really) good posts i moderate them up.. i assume most people do it that way. the result of the new system would be that probably peopel see a good comment and can not moderate them ..

    so they search for a similar one that does express a similar idea (or simply repeats most of the idea) that they can moderate up.. and the will moderat that up. so the whole moderation process would loose sharpness it would be somehow blurred. it would not help to find the perls form the posts...

    mond.
  • The way things are set up now, with a potentially infinite number of levels of meta-moderation (what if the meta-moderation isn't fair? do we need meta-meta-moderation?), is too complicated. I propose a simpler system.

    Set up a directed graph, with the nodes being the slashdot users, and the weighted connection between user A and user B being A's average opinion of B's posts. (Maybe put in an "uncertainty" factor, as well-- only having rated one post means the rating has a high uncertainty, while having rated dozens means the rating is quite accurate.) Now, use paths through the graph to determine whether a post should be seen. If I like B's posts, and B likes C, then I should be likely to see C's posts. This way, there is no one universal standard for a "good" post. If I happen to think that it's really funny when people flame each other and post a bunch of offtopic stuff, I should be able to adjust my ratings so that is mostly what I see. On the other hand, if I want to see well-balanced, reasoned posts, I can do that, too.

    This kind of system would mimic the kinds of trust systems you see in real life-- you trust the opinions of your closest friends, those of friends of your close friends a little less, and those of random people off the street not very much at all.

    You could also set it up so if there is no path between yourself and a given user, you just look at the overall average of paths coming into that user and use that to build a reputation score. You go back more than one level deep, though, so if 10 people rate user A highly, but each of these 10 people has received a poor rating from 10 other people, A's rating should not be that high. (In effect, the first extra level deep is your "meta-"moderation and the extra levels are even more exponentially diminishing "meta-" levels.)

    Of course this system is based on scoring per individual, not per post, but I think it's safe to assume that the overall quality of a given user's posts is going to be fairly consistent.

    I'll admit that this may be more expensive in terms of implementation complexity and resource usage, but I think it's a conceptually clearer system than what exists now.
  • by Kyrrin (35570) on Tuesday September 14, 1999 @08:18AM (#1682551) Homepage
    That would not then have the function of raising/lowering the score, though, which is what Over/Underrated is useful for, no?
  • Having read a significant proportion of this discussion so far and seen a significant number of people arguing against M2, I'd like to put in an argument for it.

    M1 is in place for several reasons:
    1. Removing spam and flames
    2. Adding notice to well written articles

    Where exactly does this fail? several points have been raised:
    1. People push their own agenda, moderating up articles by friends or whose topic they agree with, rather than the reasoning behind it.

    2. Its a simple fact that some people have very different views from others.

    In a collaborative moderation scheme such as this, it is entirely necessary to limit these bad effects as much as possible. The first attempt here on slashdot was to minimise the damage moderators could do, by giving them an extremely limited amount of power.

    I believe M2 represents a better solution. While it cannot have effect instantly, since it will take a short amount of time for karma ratings to settle properly and for people to get used to the system, it now gives us the ability to not only limit those moderators who moderate inappropriately, but also to give more power to those who regularly moderate well, thus reducing the need to have more and more people moderate (reducing its special status and making it a chore) while at the same time, increasing the likelyhood that any given moderation point awarded is from a moderator whose moderating methods meet the approval of the community.

    In this case, the presence of M3 or higher would never be necessary, M2 represents the will of the community as a whole, and M1 represents the average of that will, with a human intellect.

    All in all, an excellent system, and as a collaborative system (as opposed to personal moderation, which is another issue entirely) I believe it only requires some tweaking to achieve excellent results.
  • by El Volio (40489) on Tuesday September 14, 1999 @06:36AM (#1682557) Homepage
    In response to the people who are upset about /. getting "anal" and the users needing to "get over themselves":

    Look, I usually have neither the time nor inclination to read a bunch of "m1cr0s0ft sux" posts, nor "First!", nor posts that are nothing but flamebait. Sometimes I do, and I read the comments with my threshold set at -1. It's at 4 most of the time, but I still go to -1 when I can to get the flavor.

    But I'm not being anal. This isn't about what I should or shouldn't read, though I do skip posts I find useless. This is about trying to filter the massive amounts of information that come my way every day. If you have the time and desire to read it all, have at it! I'm actually jealous, since Lord knows I would like to do so.

    Sadly, I can't, and neither can a lot of other /.'ers. If Rob et al. have the time and desire to improve the system for us, great. If not, I'll live. I'll still read /., post when I can, and continue with my life when I can't. But for cryin' out loud, don't act like somebody stepped on your toes when the end result of all this can be totally turned off, or even selectively turned off, by anyone who so chooses.

    There, I feel so much better now. :)
  • The present system of moderation always seemed a little arbitrary and too easy to abuse. So here's a strange idea that I think just might work:

    On each comment, have three radio buttons for Good, Bad, and Average, set to Average by default. Each comment gets a score, which starts at 0. Now, whenever someone gets moderator status, have them pick a button for every comment. When they click "Moderate", "Good" posts get their score incremented, "Bad" posts get it decremented, and "Average" posts are left alone.

    Now for the important part: When the page is displayed, posts with scores in the top 15% get "Very good", next 20%, "Good", next 30%, "Average", next 20%, "Bad", and last 15% "Very Bad". Or those names can be changed to whatever. Since this system needs more raw feedback to work right, increase the number of moderators or frequency someone gets moderator access.

    Another idea is to let everyone moderate all the time, but then people might not take it seriously, or not do it carefully.

    Note that the method of determining the final category of a comment by definition averages out the individual moderations, so it is very difficult to abuse the system.

    Possible problems would be the complexity of the code, especially keeping track of which comments a moderator has already moderated (because they get to moderate all comments). However, it doesn't seem much harder to implement than the metamoderation system.
  • I think the problem here is the moderating up of "funny" one-liners, or "funny" posts that are borderline flamebait. One example that comes to mind, despite my completely having forgotten the context, was a "They Killed Kenny!"-type one-liner going all the way up to 5, and being the top post in the thread for those of us who sort by score. (Like me.)


    However, the one time I can recall being moderated up for amusement value (I think the post in question made it up to #4) was for a carefully-done parody of the article I was posting to. It was no one-liner, and it required about a half-hour of thought and editing on my part -- probably more raw effort than I've put into most of my /. posts. :) (For those playing along at home, this was my "How to Flame a Slashdot Post" guide. I'll have to dig that out of the archives, because I think that despite being silly and sarcastic, a lot of what I had to say there is pretty relevant to many of the discussions I participate in.)


    Butanyway. The problem with "funny" is that it's easy to abuse, but I don't think the solution is to get rid of "funny" as a moderation option. I'm not sure how to encourage better-quality satire, however.

  • by fable2112 (46114) on Tuesday September 14, 1999 @06:39AM (#1682567) Homepage
    Lately it seems as though every time someone says "and of course I'm gonna get moderated down for this" or something similar, the comment gets moderated up. Either moderators are trying to prove how good they are by moderating up self-described "controversial" comments (so nobody will think they are biased), or posters are really overdoing the false-modesty thing. Or perhaps a little bit of both.


    In any case, it's getting a bit tedious IMNSHO. It's not that I generally have anything against the comments contained in posts that also have this statement, but the statement would make me, personally, less-inclined to moderate the post in *either* direction.

  • by fable2112 (46114) on Tuesday September 14, 1999 @06:55AM (#1682568) Homepage
    [Speaking of moderation, someone moderate that up please??]


    Of course, there is the slight potential problem of moderators who can't otherwise post to the discussion trying to get their two cents in that way, but limiting it to one line will cut down on that.


    In addition, it would help pick out problem moderators. If, for instance, someone is moderating a post down and puts as a comment "You're a loser. Go away!" or something similar, we might have a clue that there's some personal bias going on. Well, that's not quite universal. I used to be a sysop on a YAWC BBS, and we used to put comments on people's Aidelines like "twitted for bad info." One such comment by another 'op was "Abusive Loser. I hate that."


    Perhaps a better example would be the inevitable "Go away Katz" posts, if they were being moderated up with a "Yeah! Katz sucks, heh heh heh," or something similarly unproductive. Or moderating a Linux criticism down with "Are you nuts? Linux r00lz, dude!" You get the idea. Also, I know that a few times when I've moderated, I've wished for a "misinformation" category with which to mod something down,sort of like the opposite of "informative."


    Like I said, though, the above poster has an excellent idea!

  • by nano-second (54714) on Tuesday September 14, 1999 @07:34AM (#1682581)

    Although the improved context aspect is appealing. The problem I forsee with integration is due to our threshholds.

    I don't mind taking 5 minutes to meta-moderate and then be done with it. I don't mind, in that context, reading a few loser comments, which have likely been appropriately moderated.

    However, if M2 was integrated, to properly participate, I would have to do it throughout the day as I read articles, with which I would probably lose patience. Or if many of the randomly chosen comments had low scores, I wouldn't even see them at all, since I like to read articles with a threshold of 2.

    Furthermore, how many randomly chosen comments would be set for meta-moderation? As it is, I only can meta-moderate 10 per day, would this be kept track of if the comments were integrated?


    ---

  • Alright. Or maybe they read Slashdot so obsessively that in the mass of posts they make, one strikes a moderator as insightful or funny and karma goes up. :)

    Actually, why not make karma be an average of post moderations? Every time it hits a integer, that score is added to your posts by default.

    So, for instance, if your posts are in average moderated to 2, you get +1 on your posts; if your posts tend to be moderated to -1, you get -1. In the rare event when your posting average is +4, you get to post posts rated 5 by default! (Like that would ever happen.)

    Don't know; it doesn't sound perfect at all. But the idea of alloting karma with regard to post quality average instead of blunt total sounds good.

    "There is no surer way to ruin a good discussion than to contaminate it with the facts."

  • That's easy to take care of. Don't take the average into account until a fixed number of posts is made, say 5 or 10.

    So the odds of some asshole scoring 5 on 5 successive posts are pretty slim.

    "There is no surer way to ruin a good discussion than to contaminate it with the facts."

  • by Enoch Root (57473) on Tuesday September 14, 1999 @06:05AM (#1682585)
    I would love to see moderation capability tied more closely to karma. In general, high karma individuals have a history of posting intelligent comments, and that would go along with being able to recognise intelligent comments. So increase the proportion of moderation power and probability for high karma! I have high karma, have been here for a while, and unless you consider reloading the Slashdot page 10 times on a workday to be obsessive behaviour, then I don't see why I haven't been given the opportunity to moderate yet.

    "There is no surer way to ruin a good discussion than to contaminate it with the facts."

  • by jflynn (61543)
    I like the idea of multiple meta-moderators per moderation. If there are enough volunteers, having three meta-moderators per moderation and only implementing "Unfair" if two out of the three agree might be useful to filter out mistaken or biased meta-moderating. Moderation mostly works, so meta-moderation should have small or positive effects on most moderator's karma or we risk losing good moderators from the pool.

    I'm worried if moderators are losing huge chunks of karma from meta-moderation. If that's because they're not moderating well, fine, but if it's because people don't understand or abuse meta-moderation it's not. Could there be another reason for 20 point drops, such as a newly implemented time horizon, or changed rules for karma totalling?
  • by jflynn (61543) on Tuesday September 14, 1999 @06:43AM (#1682590)
    I don't agree that "Funny" should be treated differently. If anything, slashdot discussions need more and better humor. The tendency to take ourselves too seriously is nicely punctured by that occasional post poking fun at one or more sides. More Shoeboy, not less!

    Disclaimer - never gotten a "Funny" karma point yet -- but appreciate those who can and do.
  • First of all, I think the idea of moderation, and the use of filtering by the individual user, is a good idea. Obviously, you guys are trying to make the best of an anarchistic situation at best, and (IMHO) I think you've done an admirable job at it.

    That having been said, I think that there might be one area where moderation might be improved:

    Instead of (or perhaps in addition to) giving someone more moderation points based on karma, how about increasing moderation based on the total score of all their previous posts in a given topic? Say for instance, that a person has posted a number of comments under a given topic, all of which have been moderated to a four or five, since they have all proven to be relevant, informative, insightful, etc. When later on, this user gets their moderation points, they get X points for general moderation, but Y points (where Y is a function of their total posts' moderation points) under the topic that they have been posting under? The point here is that if someone's posting under a given topic, and they have regularly posted well (and this could include some kind of deduction if they similarly post flames, trolls, etc.), they probably have a good understanding of the topic, and should perhaps be given a little more leeway and/or (dare I use this word?) power under that topic.

    This power would, of course, be subject to revocation if the meta moderation found them suddenly becoming Mr. Bad Moderator (an obvious flaw in my suggestion, I know.)

    Just a thought.

  • In general I agree with the proposed changes. With one exception. The 20% rule.

    You mention that you believe that in general the majority of people are good and behave in a manner consistant with this evaluation. Ergo, the majority of people when given moderation powers will exercise them with the best possible judgement they can (we're not perfect and moderation is an exercise in personal perference to some degree or another).

    So then why limit to 20% of the comments. By your own argument this restriction would do more harm then good, as there are more good moderators who would be hindered by the restriction then there are bad moderators who would abuse the ability.

    Add to this the idea of giving karma more overall effect in the moderation process and I think you've pretty much solved the problem. Bad moderators will not gain moderation status as often, thus further raising the proportion of good moderators. Adding the 20% rule to this would only exasberate the problem. In my opinion :).
  • by ucblockhead (63650) on Tuesday September 14, 1999 @06:55AM (#1682596) Homepage Journal
    Solution: Allow all readers to be moderators all of the time. There will never be a deficiency in the number of moderators with this method. The number of people scoring articles will be directly proportional to the number of people reading that article. When 100% of the participants can cast votes, there is no outlying possibility if lack of moderation.


    The trouble with this is that most people will soon cease bothering to moderate. When it is a rare occurance, people will take it as an honor, and work to do a good job. When it is a constant duty, people will slack off and not moderate. The only people who will moderate are those with very strong opinions on the subject, and I'd suggest that they are the least likely to be objective.

    If you take KDE vs. Gnome as an example, with random moderation, you have to assume that a large percentage of the moderators will be in that largest class of people who doesn't greatly care one way or another and moderation will be fair. If everyone has the power to moderate, most people (the moderate moderators) will be sick of moderating and probably won't bother. The only ones who will are those who think it is really important to moderate posts up or down. Those are likely to be the more fanatical posters, and you won't get objective moderation.

    Moderation will just turn into a new sort of internet poll, and we all know how accurate those are!

    Ideally, you'd always want moderators who had no opinion on the subject. Since that isn't likely possible, choosing at random is likely the next best bet.

    I do like your other ideas, though. The averaging of scores would help one problem, which is that in popular topics you seen pages and pages of fours and fives while in a lesser used topic, you might see only a couple of twos and threes. Obviously this has little to do with post quality!
  • I don't know about everybody else, but I like moderation. It allows me to skim through articles looking at stuff in the 3+ range so I can catch the major points very quickly. Without moderation, I have to wade through offtopic, irrelevant, redundant, and just plain wrong information. I, and many other readers, are not experts in all subjects, and so we need knowledgable moderators to separate the fact from the fluff.
  • by HSinclair (64082) on Tuesday September 14, 1999 @06:19AM (#1682598) Homepage
    Although I do appreciate all the time you've spent coding in all this crazy moderation stuff, rob, I am somewhat concerned about some things.

    A large portion of comments deserve no more or no less than the 1 (or 0 for AC's) that they get.. when you only display moderation controls on only part of the coments, almost all the comments with controls won't really be deserving of a moderation up or a moderation down. I think when this happens, some moderators, esp. the new ones will be just "itchy" to spend their points on something before they go away, and will moderate all the posts they can, regardless of whether they are deserving. (I may be wrong on this though)

    What I feel might be a better solution than only having the moderation controls on a few of the comments.. Have a lot of moderators with only 1 or 2 points so your exampled Gnome bigot won't be able to moderate down all the KDE comments he sees , yet the "good" moderators will still be able to put the deserved -1 on those "Let's petrify nitrozac and hump her leg" posts.
  • How's this for an idea, if a user's comments are consistantly moderated as good, then new ones default to being rated good, until moderated elsewise. Same idea for a bad user's comments, as long as there is still some hope of an anomolous good comment by a bad user getting to the masses.
    It could be as simple as the default value for your comment being the arithmetic average of past comments (within limits), or it could be more complex.
  • Well, as I see it :

    The opportunity to see what is your Karma value tend to let you think "I have to improve my Karma skill" or "I'm chaos devoted, i've to be the worst"

    Perhaps renaming it with a no-meaning name would prevent this sort of behavior, or just seeing as a binary value : can moderate/can't

    ( and I apologize for my poor english-writing skill, need to improve it ;-)
  • I don't know where you get this from.

    +5s are pretty rare, 2s are very common, 3s less so, 4s fairly unusual. I might seem that -1 is assigned rather too often, but that might just be an illusion based on the fact that the bell curve ends at -1 on the minus side, so you have a "piling up" of all bad posts in the -1 category.

    If what you are saying happens, you'd expect to see more 5s than 4s. Especially considering that the bell curve ends at 5 on the plus side, you'd expect to see more 5s than 4s ANYway, but I've not noticed this to be the case.

    How about it Rob? What's the statistical breakdown on all Comments?

  • by JordanH (75307) on Tuesday September 14, 1999 @09:08AM (#1682614) Homepage Journal
    I like the idea of more Moderators with 1 or 2 points only.

    Conversely, I feel that truly old hands with very high Karma should have more points. This will aid /. in maintaining it's culture and flavor.

    What I'm thinking is something like:

    [Moderator Points] = (10*([age of your account]/[age of slashdot])*[Karmic Factor])

    [Karmic Factor] = MAX( [Karma]/[Nirvana Threshold], 1)

    All of the above are self explanatory except the [Nirvana Threshold]. This could be some threshold that's quite high, possibly the 4th standard deviation positive from the distribution of all Karmas. Maybe a factor could be thrown in to lessen the influence of account age, but you get the idea. I'm not sure whether to round up or down the moderator points assigned. Of course, no fractional points should be allowed (hmmmm.... no that was dumb, no fractional points allowed).

    This system has the significant advantage of being very geeky, but still understandable to someone without more than High School Algebra.

    I really feel that your /. seniority should have a real weight in the matter, not just a cutoff (no newbies). I respect the old-timers who have been consistently moderated up in the past.

    The problem with basing ability to moderate and moderation points on Karma alone is that newbies , and I'm probably guilty of this, might be working too hard to increase their Karma. This might tend to generate a lot of popular Comments that are really just there to get applause rather than real meat.

    An advantage to allowing new moderators just a few points might encourage these new moderators to take great care in assigning them.

    Another benefit to penalizing new accounts on a sliding scale is that it would tend to discourage people from creating new accounts just because they are in a Karmic hole.

    One potential problem is that someone with only 1 point might be more likely to only moderate up. I know I wouldn't want to waste my point on some Troll. Possibly assigning points in pairs (+/-) would help here. I like this. Moderation points are actually quantum fluctuations.

    Here's a question for Rob, What is the "average" moderation? + or -? If you assigned in pairs the "average" moderation would tend to be zero. Not exactly zero as some people might not use all of their points before expiration. Would this be a good thing or a bad thing?

    In general, the idea is to increase the reliability of moderation, while allowing a broad cross-section to participate. A sliding scale, based on your Karma and seniority would help to promote this, I think.

  • by torrent (77284) on Tuesday September 14, 1999 @07:20AM (#1682616)
    If someone is logged in with moderator status, or they are surfing as A.C.'s, then don't display numeric ratings on the posts. This way moderators can assigned ratings based purely on a post's merits (or lack thereof) and not based on what another moderator has to say.
    I suspect that hiding the scores would actually increase what you want to prevent. As it is, a moderator can see whether a post has been moderated appropriately. But if you hide the scores, every moderator might increase the score on a post that is slightly above average. You would see a post that might deserve a +2 turn into a +5 because every moderator decided that it was worthy of a little extra attention, and they were unaware that the post had already been given that attention. Leaving it the way it is allows moderators to move the score towards what they think is ultimately appropriate.
  • by John Regehr (77784) on Tuesday September 14, 1999 @06:40AM (#1682619) Homepage
    Currently, everybody sees posts that are ranked on the same scale: what the moderators think. I don't agree that this is (always) the right thing to do.

    Rather, each reader should have the ability to see a customized view of /. based on their preferences.

    For example, if I see a post moderated up to "5" and I think's it's a bonehead post, I would like to de-value the opinions of the moderators who liked that post. This is different from meta-moderation: I'm not saying that these moderators are bad and should be prevented from moderating, I'm saying that I don't want to see the results of their moderations.

    Similarly, when I read /. (and the linux kernel dev list, and comp.lang.perl.misc) there are some people whose posts I consistently value. I would like a way to always read their posts. Conversely, I would like a /. "killfile" so that I don't have to read posts by people who have irritated me in the past. Again, I'm not talking about reducing their karma so they can't post - I want a filter so I don't see their posts.

    Eventually, each user will build up a profile of posters and moderators whose opions they trust and distrust. One can picture people trading these profiles, merging profiles with other people's, etc.

    As the amount of information on the net grows, developing a systematic way to figure out which of it you trust and which of it you don't is going to become a very important problem.

    Didn't Brin have some sort of reputation management system in Earth? Reading up on this might be a good place to start.

    Anyway, the point I want to make is the people have different tastes and want to read different subsets of all posts. There's no reason to force the same view of moderation on everyone.
  • by Ticker (79929) on Tuesday September 14, 1999 @07:56AM (#1682622) Homepage
    Unfortunately, alot of newsgroups do suck now, because there wasn't a decent retro-moderation system in place when the NNTP protocol was first built. It's either nothing, or all the way. When it was first designed, NNTP really wasn't intended for use by the general public at all, like alot of the Internet protocols. Now look at what a haven for spam newsgroups are. Out of 30,000 of them, only several thousand are probably useful now.

    So obviously it's important for Slashdot to have a good moderation system, otherwise we might end up with a SNR equal to that of newsgroups.

    The good thing about Slashdot is that we can add and take away as many features as we want, because there's no protocol involved here, only code. So we can try one thing, see how it works, and if it doesn't work out, try something else. I think though, that we have to try not to make the moderation system *too* fancy. If we make moderation too difficult and subject to review after review after review, then I think moderators will begin to ask themselves "what is the point?". I would hate to see something like this [segfault.org], for example, become reality.

    On another topic, I think that meta-moderation in the comments list itself is a bad idea. Let's say I'm person A, or a friend of person A. Person A gets their post moderated down. I might choose to meta-moderate that post. Even if we ban the poster themselves from meta-moderating where their own posts are involved, we can't stop them from telling all of their friend to go there and do it for them.

  • Slashdot was mentioned as one of the shining examples of 'Reputation Managers' by Jakob Nielson in his latest Alertbox column [useit.com] on USEIT.COM [useit.com]. In the article Nielson is basically saying that Reputation Management systems are up and coming ways of adding important value to the web. Only thing is, he has one rather pithy comment about useability on /. (he is, after all, a usability guru). Quote: "Unfortunately, the ability to filter out poorly rated comments is not turned on by default, so only diligent users who study the slightly confused user interface will discover this useful feature."

    Personally I don't find the /. system that difficult to understand, but I do remember having some difficulty finding documentation on how to use the moderation/scoring/karma features. I know there is some stuff in the FAQ and all that, but a more formal set of documentation might be in order.

    I know, I know. Documentation is always the last thing a programmer gets around to writing. But look at it this way, with /. being a continual work-in-progress there is little chance any one will get around to doing some simple 'man pages' for /. anytime soon unless it is made a priority.

    Perhaps we users should take this task on for ourselves. No, I am not volunteering, or at least not to do all the work...

    Jack

  • When M2 appeared, the first thing I thought it that it opens up a pandora's box of moderation levels. The moderation system tends to work well, and the M2 system should compensate for rogue moderators. but there will be calls against "bad" M2 moderators. This is inevitable. But what can be done to moderate the moderator's moderators?

    Simple - instead of (possibly) adding additional M's of moderation, maybe it would be better to solicit feedback from the people who post to /. On a separate page - accessable only to users with accounts - allow all users to view a sample of M2 moderated posts and comment on their fairness.

    This could work kinds like the quizzes that appear on the sidebar of the main page. There might be five or ten M1 and M2 moderated comments that people could look at and vote on fairness. If an M2 moderator consistently (over the space of weeks, months?) is out of line according to the /. users, then their priviledge to M2 moderate will diminish accordingly.

    Now here's a (possible) catch. In addition to real M2 moderated posts, a few fake posts - the specifics of this are a little beyond me right now - would be included. These posts would serve to gauge the response of the /. users (some would be obviousally biased, others not so much so) and would just throw some randomness into the equation.

    There's not going to be any one final solution to keeping any forum free of worthless posts, but this system would do much to bring the /.'ers back into the fold of self-administration. Any thoughts?

  • ...is to have Rob ban, for just a day or two, trolls and flamebait.

    Having had my very first /. posting (a polite statement of opinion, butressed by a few facts based upon my direct experiences with the item being discussed) labeled 'flamebait' makes me want to disagree. Dare I?

    (Ok, so maybe I wasn't completely polite: I used the word 'sucks.' However, it was in reference to an inanimate object.)

    I'm not new to this. I've posted in chatrooms|forums|newsgroups|whatever for over 10 years, and was completely surprised by instant negative karma.

    Maybe not based on some moderators comments, but on what Rob/Hemos/Katz/crew actually read.

    Although it would probably have helped in my situation, I'm sure Rob/Hemos/Katz/crew probably have enough to do already. I'd prefer they spend the time much as they already have: Sifting through the contributions and posting the relevent ones.

    Hmm, the majority of my non-AC posts are about moderation. I think I'll stop now.
    --

Neckties strangle clear thinking. -- Lin Yutang

Working...