Forgot your password?
typodupeerror
User Journal

FortKnox's Journal: Moderation System 38

Journal by FortKnox
Have you read Taco's latest journal?? He speaks about some moderation ideas he's brainstorming.

What truely peaked my interest was his idea of using weights. Maybe I'm going WAY overboard here, but here's my thought:
  1. I like the idea of all moderations affecting the post. If I start at score:2, get 4 upmods in a row, and 1 downmod, I should still be at 5.
  2. Pending moderation queues. I don't know how feasable this is, but what about making a moderation 'pending' and put into the m2 queue right away, and let people m2 their hearts out. Moderations go from pending->moderated if the m2 works well, or pending->removed if m2 says its bad (and the moderator still loses the point).
  3. Neural Net intelligent scoring. Based on M2 (for pending AND after moderating), weigh the moderators mod'ing ability. If they are constantly getting approved moderations, the weight will increase the moderation count (it'd mean scoring going into floats instead of ints, and there'd have to be a weight max, of course). Its the AI way of doing things.

I'd even be willing to code out the neural net neuron for the user. What do you guys think? Any other ideas (besides using K5's system)??

This discussion has been archived. No new comments can be posted.

Moderation System

Comments Filter:
  • I don't know how much this is like K5, but...

    I wish that there was more group decision-making in the process, just as you say, so that it won't be so easy to mod down. To encourage people to mod up, maybe they should make a rule like "You need more people to agree with you to mod down, and it will cost you 2 mod points instead of 1.". I M2 on a regular basis, & I get the impression that there are people going around, and only modding down. Perhaps they are trying to get rid of the noise, or discourage group think, but still. I don't think that it just a mistake either, because a lot of them seem to be opinion based.

    Another suggestion is to be able to mod up or down without affecting a person's karma. There are times, when a message could be moderated differently, but it's nothing to do with the person or opinion.

    I definitely like the idea of a queue.

    Your idea of +this -that = +5 might be better expressed as "vectored moderation": the positives & negatives shouldn't eliminate each other. The totals should cancel each other out, just like Physics & Algebra.

    Keep up the good work.
    • On plastic it costs 2 points to mod down. That reduces Modbombing, but the flip side is that long, offtopic threads about Men's Rights and Israel on the Vibrators story don't get modded into oblivion. So it cuts both ways.

      I like sllort's old proposal to ask the moderator the desired score, and then if Current Score Desired Score mod up, otherwise leave it alone. (The converse applies for downmods.)

  • He should enable comments in his journal.
    especially when he makes statements like "I know I'm gonna get flames over this." Geez, open it up for some discussion.
    Anyway, On the part of moderation, I can see how the friends/foes thing will have a big effect on how overall moderation goes.
  • log (n) vs linear (Score:3, Interesting)

    by daoine (123140) <moruadh1013&yahoo,com> on Friday November 08, 2002 @03:56PM (#4627843)
    I always wanted to see comment scoring more like a logarithmic scale rather than a linear scale -- now, I haven't worked out the math to actually see how this would work, but it intuitively makes sense.

    A couple +1's would boost you up through 2 and 3, but you'd need several more to hit 4, and significantly more to hit 5. The more people that moderated, the more stable a rating would be (in theory) Of course, you's need to implement all moderations affecting the comment, but I think it would be neat...

    • That's exactly what Chacham describes in his journal entry here [slashdot.org]. Just a simple curve which could make scores much more helpful in the long run.
    • ...now, I haven't worked out the math to actually see how this would work, but it intuitively makes sense.

      Actually, what you're describing is similar to how baseball votes for awards like MVP or Cy Young, only in reverse. In baseball, voters for a particular award fill out a ballot, ranking the top three candidates for the award. A first-place vote is worth 5 points, second is worth 3, and third is worth 1. Add up all the points, and Barry Zito beats Pedro Martinez for the AL Cy Young. <grumble />

      I like your idea. Maybe make it a function of the current score. So, if you're at +3, it would take 3 upmods to reach +4, or 3 downmods to fall to +2. If you're starting at -1, 0, or +1, it only takes one vote to move in any direction, so a well-spoken AC can get some easy upward momentum, while trolls get smacked down right quick.

      It may also address what you described in your other post: the 3-4 gap. This spreads out upmods enough that +5 is really meaningful, but allows other worthy posts to rise to +3 or +4. Upmods from 0 or +1 and posters with the Bonus tend to create a dense pack at +2.

    • Why not just let the scores run up to 20? If you're going to let people spend the points, why not just make the system more finely grained?

      • the point of log is that it requires exponentially more +1 moderations to get to the next level -- which stops comments from immediately shooting up to +5 from 5 moderations -- you could pick a larger constant (like 50) of moderations to get to the top rating.
        • by JMZero (449047)
          you could pick a larger constant (like 50) of moderations to get to the top rating.

          So why not just have the top rating be 50?

          the point of log is that it requires exponentially more +1 moderations to get to the next level

          Either way, those who have been modded up the most times will show up first. Why bother hiding how many times people have been modded up in some log system? Why not just show how many times the thing has been modded?
          • Why not just have the score be -128...128 or -32768...32767 or more depending on how many bits Slashdot uses for the score.

            Use the same system for karma, and give people lots of mod points. Something like karma/2 per week. Let the karma flow towards neutral. The bigger the absolute value of your karma, the faster it approaches 0. Like one point every 30/abs(karma) days.

            And semi-related: new negative moderation reasons. At least give us "incorrect". Maybe "stupid" and "not funny" are a bit too inflammatory. "Crapflood" and "cut-n-paste" would be good as well.
            • I, too have been disappointed with the moderation reasons... both positive and negative. "Incorrect" would be a good start. "Lame" would work for 'not funny'... "Obvious Karma Whoring" and "Cut-n-Paste" are both too long to go in a drop-down, I'm sure.

              My favorite choices for negatives would be:

              • Didn't read article (Uninformed?)
              • Metoo
              • Lame
              • Bitching about site being slashdotted. (Whiner?)
              • Running gag (ie, hot grits, natalie portman, beowulf cluster)
              Maybe some people like the running gag things, but I think it's a waste of space; someone could've voiced their opinion, but instead went for a cheap laugh.
              • I disagree with Lame. That's way too broad imho and could apply to about everyone/everything you don't like or agree with..

                Not Funny, on the other hand, is enough self-explanatory that I could picture it in there ! I like your other ideas too (Incorrect, Whiner, Karma Whore, etc.)

          • Well, one could think that 50 is cumbersome - What's the difference between a score of 50, or 49? It's a Leichert scale gone wrong, because the intervals have no meaning.

            Using a log scale isn't hiding the number of people that agree -- it's the forcing factor that makes more people agree. You want the good comments to be seen quickly, but the best comments have to be voted on by a lot of people -- you can't really do that with a scale of 50, or any linear scale.

            • Well, one could think that 50 is cumbersome - What's the difference between a score of 50, or 49? It's a Leichert scale gone wrong, because the intervals have no meaning.

              50 is cumbersome? You don't think people can grok the meaning of 50?

              One is easy to figure out, and one isn't. 49 means 49 people have voted this up - and 50 means fifty people have. If you want, I could explain 47 and 23.

              If you want confusing, try being a moderator on a logarithmic scale (I used up all my points, but none of the scores changed?)

              Using a log scale isn't hiding the number of people that agree

              Actually, that's precisely one of the effects this method would have.

              -- it's the forcing factor that makes more people agree. You want the good comments to be seen quickly, but the best comments have to be voted on by a lot of people -- you can't really do that with a scale of 50, or any linear scale.

              OK, you understand that either method would function exactly the same way, right? Each logarithmic score would just be a way of displaying the true, linear score kept in the database. Any other way of implementing this would function exactly the same, and would be more cumbersome.

              Either way, a comment would/could require the same number of votes to be at the maximum rating. Either way, comments that had got more positive votes would show up first. Only the displayed numbers would be different, and the log score's sole benefit would be that it's more confusing.

            • Why bother hiding how many times people have been modded up in some log system? Why not just show how many times the thing has been modded?

            Because if we limit a strict high-score to 5, the usual filters apply: you may want to read your comments at +3 or above; not necessarely +10 or +20 or +2000. That just wouldn't work because it's impossible to guess how much points would a comment ever have, on the other hand it's easy to define how "good" the comment ultimately becomes if we limit the highest score to +5.

            Anyway I prefer the logarithmic system, but that's imho of course.

            Btw: the upper limit (perfect 5 score) should only be achieved per story, with the comment achieving the single best metric. All the other comments should be computed relative to it, eg. we should have the next ones down weighted at 4.97, 4.89. 4.45, 4.0, etc. (just throwing numbers in to make my point - not actual computations.. I hope you get the idea :)

            • by JMZero (449047)
              Btw: the upper limit (perfect 5 score) should only be achieved per story, with the comment achieving the single best metric. All the other comments should be computed relative to it, eg. we should have the next ones down weighted at 4.97, 4.89. 4.45, 4.0, etc. (just throwing numbers in to make my point - not actual computations.. I hope you get the idea :)

              That does change things... Would probably work well.
    • Now that I think about it, I like my idea [slashdot.org] of limiting the single most-modded-up comment to +5 and computing all the other comments scores from it more and more. In this view, it doesn't really matter if the computation is linear or logarithmic, in fact it's probably better if it stayed linear.

      We'd have to limit the scores granularity to 2 decimal places though, for clarity sake :)

  • I always liked the reinforcement method of Google where a site/post is bumped up IF the user goes to it and bumped down if the user returns to the same search. Of course it would be much more difficult to do on /. but a similar thing happens on other BBSes (the 'bump' posts to pump up the amount of replies to a thread thus making it more popular and more likely someone will read it).

    So how could /. implement this? Allow everybody to agree or disagree with moderation to a post. Heck it could be something of a radio button of agree with moderation or disagree with moderation. And then a submit.

    You could then cap out moderator moderation at 3 or 5 and then only allow 5 if enough users agree with the post.

    I'd assume you could do likewise with modding down (although that would be more difficult as it is possible a post gets modded down and enough people don't see it).

    Some implementation problems:

    * uneven user-level moderation (ie some people do it consistently, others don't). I guess then you could make it so users couldn't take away from a mod up, only cap it from getting a 5.

    * web-load from gigantic amounts of HTTP-POST/GETs being thrown a /. I don't know if this could be a big problem or not.

    The gain: now meta-moderation or background heuristics are less important as this occurs at runtime. And instead of having some neural net chugging in the background for each user, it could just be a simple function over the amount of bumps a post gets (I also think # of Replies to a post should count as bumps as well. How can a post get 12 responses but not be modded up? Starting dialogue is as important as making good points).
    • (I also think # of Replies to a post should count as bumps as well. How can a post get 12 responses but not be modded up? Starting dialogue is as important as making good points).

      I think that's a really good point -- all my posts on /. seem to be schizophrenic like that. They're either modded up to 5, or they stick around at 2 and have tons of response. Strangely enough, it never seems to be both.

    • I also think # of Replies to a post should count as bumps as well. How can a post get 12 responses but not be modded up? Starting dialogue is as important as making good points

      What if the post is totally inane or completely off-topic, yet one of its responses is interesting and generates a mass of replies? What about flame-wars? This would cause problems with trolls replying to each other, thus causing their posts to be bumped up.

      Now if the moderators karma (and moderator record - based on how many fair/poor moderations via M2) was used as a modifier, so the "good" moderators' moderations were worth more than moderations than a bad moderator, things could get interesting...
      • Ehh.. I still think a root post that starts a flurry of topics is important just to see. Often there is a later post that is at 3 or 4 and you have to go searching for the post it is responding to. Now I'm not talking about modding up more than the base one so, at most, only one troll would get the bonus. Heck even if it didn't add to their Karma, just that it was bumped up for us to see.
  • Too many people have the +2 bonus, it should be much harder to get than the current situation. There are several ways of doing this, probably the easiest is to double the karma cap and increase the amount required to receive the +2 bonus.

    The biggest problem with the current moderation system is giving editors unlimited moderation points. Slashdot should be a 100% user moderated system instead of being to prone to editorial censorship.

    • The biggest problem with the current moderation system is giving editors unlimited moderation points. Slashdot should be a 100% user moderated system instead of being to prone to editorial censorship.
      Hell, I'd be happy with just getting rid of Michael's moderation capabilities. As far as what I have seen, only criticism of him and his postings are what really get abused. I think that editors should only be eligible for mods just like the rest of the users though. Let their accounts get weighted for mod points just like everyone else.
      • Re:+2 bonus (Score:3, Insightful)

        by cscx (541332)
        Yeah, amen to that. Michael is an asshole. That having been said, I think that "Overrated" gets abused even more... it doesn't show up in Meta-Mod! What kind of crap is that? Basically, meta-mod means nothing. You can mod up a "first post" comment +5 Underrated and never get caught.
        • That has been reported [sourceforge.net] in the past, but Taco's answer was "intentional design decision"........

          Amen. :(

          Btw, didn't we used to be friends ? ;)

  • Taco hit on the idea of time degredation and "matched" moderation. If two users mod the same, it has a higher point value. I don't think that +5 is a high enough cap. Unless you move to points (4.5, etc) or expand it to 10, maybe 20, weights work much better.

    Your idea on the pending moderation status is a great one. But it would require more people to M2... Maybe have it so the first moderation in any category (Say a comment gets +1, funny, then -1, Offtopic) doesn't count, but the second one counts for one (then each additional is +1) so you'd have a comment with base score of 1, with +2, Funny, -1 Offtopic, and it would be scored at 2. If there were another -1 Offtopic, it would go to 1. It would neutralize the first moderation done, which I think would A) Help with too many +5 comments and B) require more cooperative moderation.
    • Taco hit on the idea of... "matched" moderation. If two users mod the same, it has a higher point value.

      I think this is a flawed idea, because the labels for the moderation are totally objective and arbitrary. It doesn't make any sense to do this.

      Think about how easy it is for something to be informative and interesting, or funny and insightful, or troll and flamebait. And so on. I think it could be argued that a funny and insightful comment should be given MORE value than something that is simply funny or interesting.

      Having two moderations the same means one of the following happened: (a) two people who moderated a post happened to choose the same objective label for the post, or (b) the second person to moderate checked the stats of that post to see what the previous mods were and selected a mod that so they agree.

      Could someone explain how this would make moderation more effective (and not, perhaps, even less effective)?
  • What about CF? (Score:3, Interesting)

    by rnd() (118781) on Friday November 08, 2002 @05:53PM (#4628934) Homepage
    One thing that rarely gets discussed in relation to moderation is Collaborative Filtering.

    With a CF system, every user would have the option to moderate any comment he/she wanted. The CF system would take the preference data that it had for each user (based on each user's moderations) and recommend other posts for viewing. Every user would have his/her own personal Slashdot.. Rating of posts would be handled through the intelligence contained in everyone's moderation ideas and weighted based on how similar two people's views had been in the past.

    If you just opened an account and didn't have an accumulation of preference data for the system to use to determine how you were likely to moderate new posts, then the system could resort to using the aggregate moderation data to generate a list of popular posts. Also, individual users could choose to view Slashdot this way if they preferred it over the CF recommendations. Note, every time a user 'corrected' a recommendation with either a higher or lower moderation, the system would 'learn' more about that user's preferences.

    Also, stories and story submissions could be handled by CF as well. With a well-designed CF system, there would really be no need for editors whatsoever, and Slashdot would branch off in a thousand different directions into a web of overlapping niche communities. The interwovenness and breadth of each community defined by each user's preferences. All and all it would be a far more rich system than what is in place today and would take Slashdot to the next level.

    There is a system called MovieLens (just google for movielens and you'll find it) that uses this kind of system for moving ratings. I've found it to be extremely accurate -- far more accurate than any of my closest friends' recommendations.

    • I don't know about MovieLens, but isn't this roughly how IMDB computes their scores for movies ?

      Anyway I like your idea a lot. But how about ease of implementation... If I were a programmer maintaining Slashcode, I wouldn't know where to start ;)

      • I'm not sure whether or not there is some CF hidden in IMDBs score computation, but to my knowledge a movie will only have one score (say, a 6.7 out of 10 stars) for everyone who looks at the score. With a CF recommender system, the score wouldn't be the average of all of the votes. Instead, it would be an estimate of the score that a particular individual would give the movie based on his/her rankings of all of the other movies he/she has seen.

        Ease of implementation: This is a tough one for me to answer, as I have not written any code to do CF. Most of the current work in CF originated at research universities, but now there are some companies such as LikeMinds [macromedia.com] that have a commercial product. LikeMinds is now owned my Macromedia, and is not likely getting much marketing attention (many links on the Macromedia site pertaining to it are dead).

        In fact, Likeminds (pre-Macromedia) created a site called www.moviecritic.com (now defunct) that worked extremely well. The Moviecritic project was a research project done by LikeMinds, probably to provide a real-world test for their technology.

        There are a lot of resources (white papers, etc) about CF on the web, particularly if you go to MovieLens and look at the GroupLens Research Project at the U of MN. [umn.edu]

        As for actually writing the code, I would guess that the hardest parts would be initially creating the CF engine (I don't know of any GPL'ed general purpose CF engines, but hey, that's an idea), and then making it perform adequately. Slashdot gets a bit sluggish sometimes, and adding CF won't make the load on the DB any lighter.

        I'd sure like to see it happen, though.
  • He had a journal entry where he talked about one of his posts getting down-modded once it was off the main page. Basically this comes down to a successful attack on those who don't think like you (by icing their Karma).

    I think that, once a thread is off the main page it should then take on different moderation rules. Or maybe moderation can only occur if people are posting to the thread at a specific rate?
  • Neural Net intelligent scoring. Based on M2 (for pending AND after moderating), weigh the moderators mod'ing ability. If they are constantly getting approved moderations, the weight will increase the moderation count (it'd mean scoring going into floats instead of ints, and there'd have to be a weight max, of course). Its the AI way of doing things.

    I understand and like the idea of having moderation ability somehow related to M2. What I don't understand is what you intend to use the neural net for? What is the adaptive part? The function that maps M2 results to mod-points?

    If that is what you meant, why is this adaption necessary (or useful)?

    Another way to use neural techniques to improve the moderation system is to categorize moderators into different groups (frequent posters, frequent M2:ers etc.) using some kind of unsupervised learning methods.

10 to the 12th power microphones = 1 Megaphone

Working...