Moderation Ideas 322
First Some Comments on M2 and Karma
As a whole it seems to be working pretty well. Some people are really irritated, other people like it, and some people simply are fretting over their karma way to much. Let me just remind everyone that Karma isn't that important. You should expect to see some karma come and go since your activity is subject to both M1 and M2 moderation. As an aside, M2 moderation will not push you out of the -10...10 range (in other words, being a bad moderator will never cause you to get the Comment Penalty). The only thing karma affects is eligibility for moderation participation.I really wish I didn't feel obligated to display peoples' karma on their user info page. It tends to be a distraction. It's not supposed to be an integer determining anyone's value as a human being, it merely functions as a sort of 'Risk Factor' for moderation related activities. High Karma means you are a low risk, negative karma means you shouldn't be given tons of moderator points and turned loose on the world ;)
"The Problem"
As I see it, the major problem on Slashdot is bad people gaining moderator points. The M2 system is one way to counteract that: An unfair moderator is (theoretically) weeded out. They are still free to post and participate as always, but they can no longer moderate if their karma drops below 0.
"The Assumption"
I'm an optimist. I assume that there are more good people than bad people in the discussion groups. I assume that the good people will participate, and help weed out the bad ones. I also assume that eventually there will be world peace and harmony amongst all races and creeds and religions and genders. I'm young and an idealist here, gimme a break ;) As a whole, I think this is true: I've Meta Moderated many times, and I rarely find that more than 20% of moderation is bad. Many moderations are questionable, but rarely do I read one and think "What a moron". But they are there, and I want to try to make it happen less.
"The Solution"
On an abstract level, the solution is to restrict the amount of power that any single user has within the system. Certain restrictions are in place already: moderators occasionally get 5 points, they and they lose them after 3 days, can't post and moderate the same discussion etc. Each of limits is designed to reduce the risk that any given person can screw with the system.Each of these limits suck for a good moderator. This is given: the naughty people force rules to exist for the good people. If those 84 year old blue haired ladies who drive on the yellow line at 20MPH with their left turn signal on weren't around, maybe we wouldn't have speed limits either (Note: I also tend to drive 5-10 MPH under the speed limit and regularly forget to turn off my blinker, so I am part of the problem ;)
"The Idea"
The idea is inspired by M2 moderations removing of the ability to choose what you M2 moderate. (This is flawed of course because you can reload and get 10 new comments but I'll fix that eventually). It also is a pain because you have no context for many of the comments. By not choosing which comments you can moderate, the risk of doing something inappropriate is greatly reduced: eg, when Mr. Bad Moderator gets 5 points he waits until GNOME article appears and then moderates all the pro-KDE comments down.The solution is to reduce the probability that Mr. Bad Moderator can moderate a comment that pushes his agenda. This is what the point limits and the time limits are designed to accomplish today, and what this idea is designed to accomplish should we decide to implement them.
So here are the ideas:
- Put 5x (Numbers are Variables here: Don't complain about them!) as many moderators into the system.
- When you have moderator access, only display moderate controls on say 20% of the comments.
- Increase the percentage of moderatable comments based on karma.
It makes sense: negative karma users would never moderate, and more karma would give you more choice in your moderation. It would cripple good moderators because they might see a bad comment but be helpless to moderate it, However since there are 5x more moderators, the odds are someone else will get it. Side benefits include more people given "a shot" at moderating and more people will be allowed to contribute. If the M2 stuff works, the bad moderators will lose karma and no longer be allowed to moderate, and the better moderators will be given a larger percentage of all comments to moderate.
Another Sort of unrelated idea:
- Integrate M2 into the comments display tree: this solves the context problem that the existing metamod page has. Simply randomly select the occasional moderated comment for M2 moderation. Plus its much more "Real Time" and becomes less of a secondary chore. Plus it allows more people to participate since M2 moderation has much less restrictions on it. The downside is that this might be a pretty serious performance hit. I haven't really thought through it yet, but its an interesting thought. Comments on it?
Ordering (Score:1)
so I like reading comments, but after the first x odd, I get bored. So then I cull the boring comments by upping the threshold to 3 or 4. Fine.
The problem is that moderators feel this way too. Any comment past the (say) first 50 is effectively lost.
Unfortunately, the very people who have insightful comments are likely to also have better things to do all day than reloading
I'm not proposing any fix, just pointing out a percieved problem.
I dialed in to find my IP is blocked from posting! (Score:1)
I hope not (Score:1)
Just a question here (and not meant to be flamebait) but do you think that you're continuously losing karma because your moderation practices are at fault?
It's possible, although I don't think so. (Obviously, I'm biased) I would say that at least half of the moderations I've done should be obvious (marking down obvious trolls and marking up obvious gems.) Perhaps I've been a little too aggressive, but I don't think it's been that bad.
There are a couple of things to keep in mind, though. First, I have no way of finding out which moderations got me marked down. So if M2 is simply knocking me down for doing a crappy job, I have no way of going back and changing my ways. Secondly, I think the volume is excessive in any case. Even if I've been doing on balance a crappy job (say 60% unfair and 40% fair), losing 20 points in a week is ridiculous. I may be a special case since I get to moderate a lot more than most people, but it's still excessive. If I had to guess, I'd say I've doled out about 20 moderation points in the last couple of weeks, so that would mean that every moderation a moderator does (if they moderate similarly to me) they will lose one karma point.
I also have talked to at least one other user who had a similar problem, so it's possible we are flukes, but it would be interesting to find out whether the average M2 is up or down, and how much the average moderator's karma has changed (perhaps karma/total moderation point doled out)
Default settings (Score:1)
>asked to moderate. Click 'No Scores' and you won't see scores again. Change
>your threshold to -1 and forget that they even exist.
Wouldn't it be easier for non-registered ACs if the defaults were set up this way? That way, if a troll doesn't see his post, then he won't be tempted to continue, unless he actually knows how to change the user preferences. If someone wants to read another's trolling, let them modify their user preferences.
Re:Asking to be moderated down (Score:1)
Oh wait! This discussion is about moderation.
Mark it up then.
In any case, use up those moderation points, because it makes you feel powerful.
I moderated once ages ago before I quit using Linux and decided it was more fun to just cause trouble on Slashdot by adding less-than-dogmatic posts. I've thrown away three nicks so far out of boredom with the whole concept of meritocracy.
Last week one of my AC posts got moderated up to 3, and I was trying to make fun of Linux with it (people thought it was funny... oh well) The post about putting black electric tape over your USB ports to make them Linux compatible.
I guess I'm not a very good troll.
Re:The slashdot moderation system is flawed (Score:1)
score threshold (Score:1)
Moderate this up! It's good medicine. (Score:1)
--
Re:Moderation and karma (Score:2)
I dislike it as well, as i've been here for over a year and have high karma, but have never, not even once, been given moderator access, since I read slashdot two to three times a day and click the "read more" on nearly all the stories.
A possible change in using/interpreting moderation (Score:1)
What I mean is, if a comment gets a 5 rating, should it be interpreted as being an exceptional comment, or merely a comment 5 people thought was a little better than the usual fare? Right now it seems like the second, but in my view it should be the first.
After all, look at the disproportionate number of comments that manage to get a 5 rating because they're 'Funny'. Sure, they may be amusing, but are they really the best that Slashdot has to offer?
The problem, as I see it, is that we are asked to moderate without knowing the current rating of a comment -- it takes a while to read through and moderate all comments to some articles, by which time the values the comments had when you loaded the page are no longer valid. So if you decide to moderate up some minorly amusing AC's comment as 'Funny', it suddenly ends up a 5, because 4 other people thought the same thing -- even though you might have thought it deserved no better than a 1 to 2.
There's a nice solution to this problem, but it would be a bit more complicated than the current method. A similar problem exists in stock markets -- by the time a buy order you've submitted gets taken care of, the price of a stock could have moved way past what you wanted to spend. So along with your buy, you specify a price range -- if the stock is out of that range when the buy gets cleared, you're not left holding some overpriced stock.
Anyway, here's the idea. Instead of moderating up or down by points, you should be able to specify exactly what rating you think a message should have. Your moderation would count as a vote in favor of any rating up to the one you specify -- past that, it doesn't count.
Thus, if you think a comment deserves a rating of 2, but not better, you could moderate it to 2. If 10 other people do the same thing, the comment would still have a rating of 2 -- as opposed to the current situation, where it would have shot up to 5.
(Mathematically, you pick the highest rating such that the number of votes for that rating exceeds the rating. To get a rating of 1, you just need 1 person to vote in favor of a 1 or better. To get a 2, you need 2 people to vote it a 2 or better. Etc.)
Any thoughts?
JRaven
Re:Having everyone always moderate is dangerous (Score:1)
I think that the current system of assigning moderator points does a good job of distributing the work of moderating. The main idea for a change that I can think of would be to assign moderation points separately for each story (something like, there have been 10 comments, the next eligible moderator to enter the comments section gets a point). I think that this would be a better way of preventing the possible problem of not having any moderators read the comments for an article, but it might not be as good as the current system in avoiding moderator bias.
Re:Watch out (Score:1)
At least as long as they don't get obnoxious about their high karma. And even that would probably be mostly self-correcting.
Averages can fix the bad moderator problem... (Score:1)
1. The moderators score an article on a scale of -1 to 5 (or what ever).
2. Slashdot takes the list of scores for an article and sorts them.
3. Slashdot discards the top and bottom 20% (or whatever, aka 'alpha') of the scores from the sorted list. (This is the trimming bit).
4. Slashdot them averages the remaining list of scores and uses the result as the actual score for the article. (statistical mean bit here).
Problem solved. Articles get moderated. Bad moderators have no effect. No meta-moderation is needed. No race conditions either. A few more moderators and moderator points would need to be handed around.
As a bonus, a background process can hunt for bad moderators and eliminate them. It would just have to find moderators that constantly score articles way different from thier peers. (Insert statistically analysis here).
Whatdoyathink?
--
Simon
Re:Intelligent... (Score:1)
And what of the lurkers? Just because a person doesn't post doesn't mean they wouldn't make a good moderator.
Actually Rob, how many users are there who haven't posted, but still read regularily?
Well, I've made a grand total of about 8 posts in the last year, and I've gotten moderator status t wice. (Karma is 6, I guess only making useful posts helps).
Well, as long as I'm posting, I might as well make my own observations.
Just a few thoughts. Maybe we should hand out positive karma points for well formatted HTML (Check out my
s!).
-Dave, going back to sleep
Re:Intelligent... (Score:1)
User behaviour Flaws, Software cannot fix all (Score:1)
When I am meta moderating, the intuitive thing to do is to glance at the comment, read the first line, and then check fair if the comment has been moderated up for 'interesting' or 'informative'. Sometimes when large amounts of text is being read on subjects that people aren't particularly interestd in, they try to skim. I don't curretly have an answer to this problem. I'm also not too sure this is a bad thing, as everything I've marked as one of the two I's has actually BEEN interesting when I read down further. My intuitions were right.
Is the voting question comments included in meta moderation? I have yet to see one.
The idea of the different reasons for marking up and marking down is interesting. Articles are posted for different reasons. For example, when Stanley Kubrick died [slashdot.org], there may have been a funny comment posted about him, and it may have been moderated up for such, but a funny comment is not necessarily the thing people would want to see moderated up to 5. However, it may deserve a 5.
One solution to this problem would be drop down boxes next to all the moderation types for the poster of the news article. The poster would select a priority from 1 to the number of moderation reasons. When the user reads the messages, a '3' that is informative would be read before a '3' that is funny.
This could be prioritized at the user level, but I think it would be more appropriate for the article poster to decide which the priorities should be.
When The Autism Article [slashdot.org]'s comments were read by me, I was dismayed to find a humour post being moderated to the top of the posts, when this was a potentially sensitive and touching issue. (Although there is a place for humour, even there.)
Me too! ;-) (Score:1)
I personally have become much more sensitive to that urge I get to NOT post a quick smartass comment, because I want my karma to go up. I'm specifically being more careful and thoughtful about what I post. I think a lot of other people are too, and you can see that in the way the quality of comments has improved in the past few days (and the volume generally declined).
I don't think this is the end-all, eventually, someone's going to figure out how to abuse this new system, but for now, it's working great.
"The number of suckers born each minute doubles every 18 months."
Re:Ya know... (Score:1)
"The number of suckers born each minute doubles every 18 months."
How? (Score:2)
How do you plan on banning these people? Just ban the user account?
What about AC's? Ban them by IP? How will you deal with dynamic IP addresses, HTTP proxies and IP masquerading/NAT?
Re:SQL glitches (Score:1)
Jabberings.. (Score:1)
1) MetaModeration is good. Once a day, 10 random comments.
2) Moderation points / likelyhood SHOULD be based on overall Karma rating. High Karma means you give more to the community, and have more of a right to shape it. Not only that, but you're probrably a little smarter then the cowards..
3) Allow higher Karma posters to automagically post at at a Score of '2' IF THEY CHOOSE. Shouldn't ALWAYS be 2, only when they think it's needed. The ratio of self-ups should be checked, and possibly taken away if Self Moderating on to many posts..
Context for MetaModeration (Score:2)
--
We Don't Like Change... (Score:2)
---
A few things... (Score:2)
Second, there are complaints of a "lack of moderation." There's a reason for that, but increasing the number of moderators is not going to help. The problem lies in the fact that you can't post in a thread you've moderated, and vice versa. People who post their insignts to a thread lose the ability to moderate. People who consider moderating back down because they might want to post something. I remember the debates about the idea of "moderate XOR post" before it was implemented, and I agreed with it at the time, but I see now that it just isn't working.
At least in that aspect, we should go back to the idea that you can moderate and post in the same thread, but you cannot moderate your own posts. At least for people with karma over, say, 15 (enough to show a history of good, responsible posting) this would end that problem.
Third: It's already required for a person to send in an e-mail address when registering. Perhaps it would be best if only one account were allowed per e-mail address. Couple this with the idea posted earlier in this thread that a person whose post gets moderated to Flamebait or Troll (this would have to be changed to require a consensus of three moderators in order to change a person's post to Troll status) cannot post anything more until the next day. The result would help to cut down on trolling, since a person cannot simply create another account and continue on. Since there's already a "Post Anonymously" checkbox there isn't a need for more than one account per person anyway. E-mail addresses can't be faked, since the password is sent to the e-mail address given when a person registers (so an account with a faked address is useless since you can't log on).
Finally, the thing about keeping Karma on a person's User Info page: people like to monitor how they're doing. I myself was quite surprised when I looked at my own score. It's a nice thing to see for those of us with high Karma, and if Karma starts to drop it's a good wake-up call. For those with low Karma, there's something to be said for watching one's progress as one claws one's way out of that trap. It's for the best that people can see what their Karma score is; please don't change it.
Interesting is harder to metamoderate than troll (Score:2)
The stupid moderations are easy to metamoderate.
But it is difficult for me if I get a comment presented for metamoderation that got an interesting or insightful mod and I can't find it exciting at all.
Such happened with an Amiga posting and a Linux posting this morning. In both cases I decided not to change anything, despite thinking that the posts were just normal level and tried very hard to see the postings not from my personal interests but more how Amiga or Linux users might see them. But I am not sure if this is the right thing.
20% of zero is? (Score:2)
No, it doesn't make sense. I hate this idea. When I moderate, it tends to take up a lot of time, browsing at -1, reading all the comments to recognize duplicates, and reading all the replies (I read FLAT not threaded) to catch the gems. But I only do this for news that interests me because I just have a limited amount of time available. With only 20% or so of news stories available, chances are I'd never use my points at all. In the three days given to use your points right now, I usually just get my points used before they expire.
Karma -> Grade Inflation (Score:5)
It seems like ratings are getting very bi-polar around here. Posts that are a little bit better-than-average are spiking out at +5, while ones that are just slightly sub-par are getting -1's. Why? Because moderators are interested in showing how much they agree with the other moderators, in hopes of increasing their karma.
I gave this some thought yesterday while I was mowing my lawn, and I came up with:
That's just my dollar/50. BTW, I've been a moderator a few times recently, so I'm not taking potshots at a different group of people.
well, if 80% of the users are good... (Score:2)
Admittedly, 5 moderator points/day helps limit the damage that bad moderators can cause. However, it also limits the benefit that good moderators can bring. I don't think it fundamentally changes anything. It is sort of like a minimizing lens; the overall ratio is the same, but the effect is reduced.
I don't remember why the moderator point limit was implemented. It's possible it may have been done in reaction to abuse that was not being corrected. If that's the case, disregard this idea:
I think what will happen is that yes, there will be some abuse, but that good moderators will correct that abuse, and that the meta-moderation will (after awhile) shut down the abusers.
If that's no good, your idea of 'a chance' to moderate any given comment is a possibility -- I'm thinking a 5% chance per positive karma point. That way, 20 karma means you can moderate any post. Most likely, anyone who has 20 karma is going to have a clue.
I have meta-moderated a few times, and I don't think I have chosen 'unfair' once. In some cases I probably would have chosen differently, but I have not seen ANY examples of grievous boneheadedness. (I don't mark down moderators for making different choices than I would, as long as their choice seems at least faintly reasonable.)
It also seems like the slant on pro-open-source and pro-Linux tends to artificially inflate some posts. I have written posts pointing out limitations of open source (most recently in regard to open docs), and also have written ones in favor of it where I think it really helps. Even though (in my opinion), the open-docs-won't-happen argument was better thought out than most of what I've written, it got no positive moderation at all. Other posts I have put up have gotten a lot of positive moderation, even though they weren't nearly as good, IMO. Maybe I'm too close to the problem, but personally I thought the negative-slant article should have scored higher.
Strikes me as a good idea to remind moderators to be aware of their own biases. I try to correct for mine when I'm moderating. I'm not sure how well I succeed, mind you, but I do try.
Re:Moderation and karma (Score:5)
Keep up the good work, Rob- someday, people will study your code to see how to make internet conversations that work for large numbers of people. I do believe that this is the first time anyone has seriously tried to make anything like this work- and the quasi-experimentation that you have done will be a source for the next group that tries to apply this technique to more serious fields of endeavour.
Re:The slashdot moderation system is flawed (Score:2)
One table holds all of the comments. The table has the userid of the author, the comment id of the comment, a score, and the total number of votes cast.
Another table has all of the authors with a userid, a score, and the total number of votes cast for that author.
A third table holds the individual scores for each author by each user. The worst case for this table is that it grows to n^2 where n is all of the users on slashdot. In practice it will not grow to n^2 because not everyone will vote for every author.
A fourth table holds all of the individual votes for an article. Assuming there are n readers and p articles, the potential size of this table is p*n. That shouldn't be too bad as most people don't post articles and only some people will vote for every article.
There are two things which pin the whole system together. For purposes of calculating an article's score, the user's own votes get a weight of 10 out of 10. The user can specify the weight of the community vote on a scale from 0 to 10. So as you count the score for the article, you multiply your own vote for that author by 10. Other authors that you have previously ranked get weighted depending on how you ranked them, and the rest of everybody gets the wieght that you grant to the public at large. Any article that can't be assigned a personal score just gets the community score. Every article always has at least one vote: when the author posts the comment is automatically scored to some constant, perhaps 10/10.
Anyway as you can see I've been modelling this for some time. With the right indexing scheme and efficient SQL, this should scale really well. It would be a bigger load over time, but over time computing power is increasing so you come out even.
Regards,
jwb
Re:Having everyone always moderate is dangerous (Score:2)
I believe that both of these problems would go away with the use of a custom desktop client. Imagine that you are sitting in front of your computer using KSlashClient. To vote on an article, all you have to do is click on a gradient strip. Full left is 0, full right is 10. When you click, your vote is transmitted to the discussion server adn registered, but you don't have to wait for anything. The vote is delivered asyncronously while you continue to read (and rate) the rest of the comments. Suddenly moderation isn't a chore any more.
Now, let's take your pathological case. Little Johnny has the day off from 3rd grade and is rating all of the flames 10 out of 10. Nobody else is around so them flames are the highest ranked articles in the discussion.
When all of the hard-working rational adults logon to read slashdot, they immediately see "Gnome SUCKS KDE RULES!!!!! (Score 10)" and they think "WTF?" So they click on the zero end of their voting scale and read on. I believe that there are a lot more good, thinking people on slashdot then there are vapid flamers, so the result would be an immediate and decisive downgrading of the flames to the bottom of the list.
As a benefit, since Johnny's personal score is 0.2 and the rational adults all have 6-10, Johnny's votes are easily overpowered.
What do you think?
-jwb
Re:Having everyone always moderate is dangerous (Score:2)
In my system, the scores are *much* more stable. For example, suppose 1000 people have given an article an average score of 5/10. Now another user comes along and gives the article a ranking of 10/10. What is the article's score? Still 5/10.
One of the best parts of this system is it's stability.
-jwb
The slashdot moderation system is flawed (Score:5)
Please accept my humble criticism of slashdot's comment system. I have been thinking about this system a lot lately and designing a replacement for it. I have no delusions of replacing slashdot: slashdot is a community that would be hard to duplicate. The only problem I have with slashdot is the comment mechanism, and I will present those problems and some possible solutions.
Problem: Moderation race conditions. Joe and Jane are both moderators reading a slashdot article. They both see a comment that they think is interesting, but not incredible. They both choose to spend one moderator point to raise the article to "Score: 2 (interesting)". They continue to read the comments and click the "moderate" button a few minutes later. Now the article is rated 3, but neither of them thought it was worthy of 3. Moderation has failed for both of them. One of them could choose to moderate it back down to 2, but then the article would be rated "Score: 2 (overrated)", which does not reflect their true opinion of the article.
Solution: Instead of incremental ratings (+1, -1), allow the moderator to score the article on a range, say from 0 to 5. Then if two moderators assign a score of 2, the article's score is 2, not 3.
Problem: Only a few people have a few moderation points at once. This leads to several worst-case failures. One of them is that no moderators will read a given article, and the comments in that article will be clustered around 0 and 1, making the "Show highest scores first" setting much less useful.
Solution: Allow all readers to be moderators all of the time. There will never be a deficiency in the number of moderators with this method. The number of people scoring articles will be directly proportional to the number of people reading that article. When 100% of the participants can cast votes, there is no outlying possibility if lack of moderation.
Problem: The community ranking does not allow for individual preferences. The individual may have a strong opinion for or against a certain author, but the score given by the moderators is unlikely to reflect his preference.
Solution: Use a network of preferences to tailor the rankings that a user sees and to delegate the ranking of articles.
I believe that the ideal system would allow the user to cast a vote on every comment posted. The comment display system would use the user's voting history to score and sort the comments on every new article. Take an example:
Joe, John, Jack, Jerry, Jane, and Jennifer are all users. In the past, Joe has given articles by Jack an average rating of 4.2 (out of 5), and has given John an average ranking of 3.8. Joe has consistently rated Jerry at 0. Joe, John, and Jerry all have overall ratings (the average rating given by all the system's users) of 2.
In a new article, Jane, who has never posted a comment before, posts an unusually insightful comment. John, Jack, and Jerry all post their usual stuff. All three of them rate Jane's article at 5, but the overall community ranking for Jane is only 3. When Joe reads the comments for the article, the article display system will give John and Jacks votes more weight than the community vote, because Joe has rated those two authors highly. The system will completely ignore Jerry's votes because Joe doesn't think much of him.
The result is that Jane's article (along with John's and Jack's) are listed at the top of the page, instead of lost in the middle of the pack. Other users with completely different preferences will get completely different displays. When there are no personal votes for an author or article, the community ranking can be used to score the article.
Whew, I hope that example was clear. It really isn't very complex or even original. It is just like what firefly was doing 5 years ago, but with a little less complexity and a lot more scalability. If slashdot were to implement a reputation management system like this, I think it would be a much better place.
Regards,
jwb
Meta Moderation is Redundant (Score:4)
Hypothesis 1: The majority of moderators are fair and take thir job seriously.
Hypothesis 2: If one moderator rates a certain comment up, and another moderator rates the same comment down, it is reasonable to assume, that they would equally inclined to consider the other's decision as unfair.
Conclusion 1: The ammount of mutually nullifying moderator decisions on a certain post provides a measure for the total 'unfairness' p of a comments moderation, where p = 2*min(#_of_ups,#_of_downs). If hypothesis 2 is true, then this p should be proportional to the karma penalty generated by the current moderation scheme.
Conclusion 2: Since we assume that there are more good moderators than bad ones, it is reasonable to assume that in a contoversial comment moderation the majority has a higher propability to be "right" than the minority (provided that there are more than 2 moderation decisins involved)
Proposed Solution: In the case of a controversial comment (i.e. a comment with up *and* down votes), generate a (very small) karma penalty proportional to p and distribute #ups/(#ups+#downs) of the penalty among the down-voters and #downs/(#ups+#downs) among the up-voters.
Example: A comment gets 4 votes up and 2 votes down. Each up-voter gets 1/12 (33% total) and each down voter gets 1/3 (67% total) of the total penalty which ammounts to 2*min(4,2)=4 penalty points (which should IMHO be somewhere around 1/5 to 1/10 of a current karma point)
In order for this system to work, it would be necessary to increase the total ammount of moderation points floating around and it might be reasonable to give some karma reward for each moderation to compensate for the total penalty generated by this system (so that in the end only the guys who regularily make minority decisions take the punishment).
Of course, the used formulas are only a proposal and fine tuning could certainly increase the overall efficiency of the system, once empirical data is available.
Re:Ya know... (Score:2)
--
one problem with that (Score:2)
One perhaps better way would be to take the average of past comments and create a separate variable, maybe call it AverageScore. Then give users a choice in their prefs whether to add the AverageScore to the Score, or add a fraction of the AverageScore to the Score (I envision 1/2 or 1/3 of AverageScore, rounded to an integer), or do nothing with AverageScore.
Another consideration with such a system is that Slashdot would have to find the AverageScore of every user who posted on a given page, do manipulations based on the viewer's prefs, and use that in the dynamic generation of the page. This could be more load on the server than any value we might get out of it.
Personally, I don't see a need to have a historical score influencing a present score. The moderation works pretty well at bringing out the good stuff and eliminating the crap. Adding a predictor value like AverageScore seems redundant at the least, and it could interfere in an undesirable way with the moderation process.
Re: Karmic disturbances (Score:2)
Re: What we really need... (Score:2)
At the risk of seeming like a pedantic pointer-out of the obvious, you can "limit the amount of noise and increase the signal" by your very own self, by setting the filter to (e.g.) +2.
Personal aside: Your tone, as evinced by comments like "Ever since
A complaint (Score:2)
I have been wondering why my karma has been gravitating towards 10, now that you mention M2 moderation doesn't take someones score outside the -10..10 range I understand why.
Since my karma is (was) over 10 meta-moderation applied to my moderation can only drive my karma down towards 10 (well it tends to oscillate between 9 and 10). When people disagree with my moderation I lose points, when someone agrees with my moderation nothing happens.
Actually I remember on a previous moderation story an AC complainined about their karma rapidly dropping (a few points a day) and that after a week the would hit zero. They wrote back the next day saying that there score seemed to have stopped decreasing, I guess they finally dropped to 10 as well.
Meta-Moderationg Causing En-Masse Karma Plummet (Score:2)
I'm quite sure that the karma does not have a half-life.
Everyone I've spoken to regarding MetaModeration has watched their karma plummet through the floor. It appears to take plunges soon after the user completes a batch of moderation - mine, for instance, dropped 3 points on the first day moderation was enabled (I was a moderator that day), and five points the next day I moderated. Other users have reported similar patterns.
I am convinced that massive negative meta-moderation of my moderation is causing this, and as a result, I am now afraid to moderate. In the last week, I've had one +4 post and one +5 post, and my karma is still five points lower than it was seven days ago.
What we're seeing here is moderators getting afraid to moderate (such as me - I keep doing it anyway 'cause my Karma is quite high, but I will stop when I'm within range of zero). This is not good.
The Meta-Moderation system is flawed in that it subtracts a full point for every negative vote, but does not add an equal amount for every positive vote. There is no way around this flaw, however: to add a positive integer for every positive vote to your moderation would probably cause the opposite; people's karma rocketing skyward.
I'm all for scrapping meta-moderation. Here's my replacement system:
Give each moderator a number of moderation points proportional to his or her karmic level.
The highly-ranked people have obviously got a handle on how to write a good comment. Wouldn't they, therefore, be the best people to tell good comments? Give them the weight they deserve. And keep all the other people in the system at a lower level to help out.
Have we Seen an Improvement? (Score:2)
I have just one question:
Has anybody seen or felt any improvement in the atmosphere at /. since the M2 system was implemented?
Re:Karma -> Grade Inflation (Score:2)
Re:This is called collaborative filtering... (Score:2)
The way the NetP engine that Movielens uses works is there are users and items.
The users rate the items, and then the system uses this data to predict your ratings on other items. The "profile" would just be the set of all ratings you have done, and so yes, comparing this profile with the profiles of other users is what allows the system to predict which users you agree with.
Now I suppose that the system you are proposing is slightly different in that you are rating other moderator's moderations instead of coments directly. But if a moderator gives a given post a +1, and then you rate that person's rating a -1, isn't that essentially the same as you rating the comment -1?
In any event, it's a short leap from your idea to a full CF model, and that would eliminate the need for designated moderators. It is, however, a slow, expensive, non-portable thing to set up, so I doubt Rob would consider doing it. I wouldn't.
Re:Moderation and karma (Score:2)
Either way, moderating /. should be taken seriously, and it filters out the worst noise, but it's not the be-all-end-all of the /. experience.
It's definitely not something you should get upset about. It'll float your way sometime soon; after that, the glamour 'll wear off pretty quickly. IMO, of course.
--
And another thing... :) (Score:2)
For moderating purposes, it might be useful to see the most busy threads first (or last) to see wether there is a flame-war raging, or a hot discussion (of value) brewing...
Just a thought.
Comment worth vs moderation increment (Score:2)
So how about a compromise:
Moderators still get moderation quanta (I vote for fractions of points, see my rant at #33). They then get to choose what worth a post has, and their quantum adds (or subtracts) the actual value of the post in that direction.
In this way, the overall moderator opinion of worth, coupled with their quanta towards that end, result in a post that is moderated to the average of all moderator opinions... Preferably where it started.
It's another level of complexity, and should keep Rob busy for weeks in trying to figure out how to implement it. [VBSEG]
Karma for Funny points (Score:3)
Maybe we should have an additional category? For relevant humor or offtopic humor? But here I diverge from my personal opinion that simpler is better...
Karmic disturbances (Score:4)
Moderate thoughts on Moderation (Score:5)
A good comment is good. It is unlikely that a single, rabid individual, will sink a good comment into oblivion, without an equally determined benefactor raising it up again. By that same token, a conspiracy of evildoers to elevate each other's trash, whenever one of them has moderator access, is slim to none.
Occam's Razor applies, and a system of lesser complexity is preferable to one of greater complexity. Slashdot should not hamstring itself with rules, and rules about rules, else it will become a burocracy in which nothing ever happens.
Personally, I've never used MM, but I would rather see it go away - simply because Moderation points strike me as a rare commodity, so their negative effect seems to be outweighed by their utility to the holder. Also, with
MM is an interesting exercise for Rob, it nothing else, and as such it is as useful as anything that keeps a mind from being idle.
As for a redoux of the moderation philosophy, how about fractional moderation? Increase the number of moderators, certainly, maybe by a factor of four. But reduce the worth of each assignable point by that same amount. This way, several moderators would have to agree to elevate a post to the next higher integer score. The ability to abuse moderator privilege goes away, as does the confidence that a single vote of one individual will make a lot of difference.
As moderators, we would then add our voice to a comment, not knowing if we're the first, or the decisive ones to raise the message status. Follow?
This way, good posts will rise, bad posts will sink, and iffy posts will tent to teeter-totter on the edge of where they started.
Ah, all this opinionated rambling has tired me out. But, before my hands completely cramp... How's about making all posts require a Preview??
Restricting Moderation Too Much (Score:4)
However, once I'm done moderating, I switch the preferences back to a more filtered view, and go back and re-read the articles looking for the content, as usual.
I think it is *essential* that moderators be allowed to pick which comments they moderate during the 3 days they have their points. Giving moderator access more or less frequently based on karma is a perfectly good idea, and could achieve some of the same effect.
Finally, I would like to just throw a reality check out there. Slashdot should be good, but there is *no* way to weed out all the a-holes, flamers, trolls, etc. Slashdot already does a very good job of tagging their comments so that people like me who don't generally want to see them don't have to. I think that, at this point, the moderation system is sufficiently effective, and time would be better spent squashing some of those SQL problems, and implementing some of the great ideas for additional forms of content that could be added.
Re:Intelligent... (Score:3)
And what of the lurkers? Just because a person doesn't post doesn't mean they wouldn't make a good moderator.
Actually Rob, how many users are there who haven't posted, but still read regularily?
Watch out (Score:3)
An evolving system (Score:2)
First, I don't envy you. It's a no-win situation, and it does indeed take the innocence of youth to put up with something like this.
Second, I'm still confused. I have a post here ( The heights of low taste ) [slashdot.org] that has been moderated more strangely than any other post I've ever made. It has 1 Troll, 2 Informative, 1 Overrated, and 1 Underrated. I can understand the over rating, but the troll moderation is somewhat curious. What rates as a troll message? And please don't say "your message"
Third, I think the real-time meta-moderation next to each message would be a very nice touch, if you can make sure that an even number of meta moderation tokens are handed out fairly. Having the seperate page is just fine by me, as I can go off to the various threads and check context if I have a question. Like you, I've never looked at any of the moderated messages and thought the moderator subhuman based on their moderation, and it's been my experience that the number of moderations I disagree with is around 5-10%.
Finally, don't try to hard to make this perfect. It never will be, and trying will only make the system more convoluted, complex, and fragile. I really do have a life, and slashdot is nice to have and a lot of fun at times, but I can live without it. Just like I'm sure you can.
Perhaps explanations are needed. (Score:2)
Perhaps we should quote the Jargon File:
Let's define FlameBait while we're at it:
All I really ask is that, when a person is moderating, keep these ideas in mind. Don't mark something stupid a troll just because it's stupid. Stick to the definitions of the terms. If you simply want to mark something up or down, use over/under-rated..
---
Moderator Dilution (Score:2)
1.Put 5x (Numbers are Variables here: Don't complain about them!) as many moderators into the system.
2.When you have moderator access, only display moderate controls on say 20% of the comments.
I'm not convinced that moderator dilution is the answer here. I would expect that by spreading out the pool of moderators, Slashdot would see even more problems with badly moderated posts. What we want is instead a smaller pool of people. but people who moderate more thoughtfully.
But how to distinguish thoughtful moderators from irresponsible ones? I like the "karma" system since it provides more personal accountability. I don't think it has been in use long enough to really show its potential, but people who regularly post articles that other people like should have a chance to participate more in the moderation process
Here are some thoughts:
With that said, I feel that the present system is working VERY well. It is extremely rare that I see a post moderated up which didn't really deserve it. Even better, by setting high thresholds, I can browse very quickly and still get some valuable insights on the articles.
Good work, guys!
Integrated M2 moderation (Score:5)
I can't tell you how many times I've thought the moderators-in the context of a comments thread, were either clueless or not paying attention or just plain dumb. However, with the current M2 setup, I don't understand the context and don't really want to take the time to go look. I am already taking the time to do the extra moderation.
When I'm not a moderator, it would still be nice to feel I am having some sort of role in the moderation process-even if its just keeping a check on the moderators.
Problems I foresee-Someone moderates a comment, and immediately the moderation is moderated (jeez, i should choose a new word) to be Unfair. So does the moderator go back and change it? Is that unfair pressure? And if you M2 a comment in a thread, are you unable to participate in that discussion?
I like the idea of Karma. Though I think it would be cooler to see it on a scale when presented on the user info page. I pay quite a bit of attention to this site, but I still don't remember the scale offhand. A graphical bar would be nice. And instead of just numbers, relate the the numbers to real world behavior. ie. karma of -1 is called 'troll' or something like that. Be creative!
-Lisa
Moderation problems (Score:3)
They might be connected.
1) Good messages not posted fast after the news are not moderated up.
2) Points used on 5 points messages don't add to the story points
but decrees your moderation points.
This mean lesser points for messages in 1)
3) There is no way to view unmoderated messages
when you moderate.
I think that a solution to 2 and 3 will help on 1.
Knud
Aeiwi [aeiwi.com] is a search tool with a unique interface.
Which let users add more search terms,
until they have a small number of results.
Re:The slashdot moderation system is flawed (Score:2)
Permit me to disagree. First, nobody is going to spend a moderation point to downgrade a -1 post, or upgrade a +5 post. Second, I don't know how typical I am, but I moderate posts when my opinion of the post disagrees with its rating (as in: this is at least a +3 post and it's only at +1 -- upgrade!). If I like the post, but it is already at what I believe is its proper level, I'll just let it be. Given that, I think that the suggestion to let moderators directly set the rating (on a -1 to +5 scale) has merit.
And, BTW, what is karma's expiration period? Clearly there is one, but how long is it?
Kaa
Random Mass Moderation (Score:4)
As a moderator, you should have the ability to moderate where it is needed, and not have your moderating ability limited to randomly selected comments.
If a moderator sees a particularly outstanding comment, they should be able to moderate it, and not be hindered by a random 20% factor. If you are going to increase the number of moderators, do so at the expense of moderation points, not at the expense of the ability to moderate.
And I do have to agree with Rob: if you don't like the system he's worked so hard to build, you can get around it without asking him to take it away.
What we really need... (Score:2)
Phew... that felt good to get that off my chest. Other than that, I think the moderation is going OK.
Re:Moderation and karma (Score:2)
Re:Intelligent... (Score:2)
Devil's Advocate (Score:2)
The people who complain about the point system aren't complaining because they don't like to look at the numerals 0-9, so turning it off won't assuage them. Their complaint is that certain people can't be heard because of
Obviously the users who complain can still view those comments, but that's not the issue. The issue is that Slashdot is a forum and if, for any reason, certain people can't be heard from then the forum becomes self-defeating.
Now then, at this point I'd like you to re-read my subject line. "Devil's Advocate" means I don't necessarily hold these views myself. As it happens I did and to some extent still do. But I've come to realize that high noise levels defeat the forum as well and something had to be done.
However, I think it is important to remember that we don't have to create rules just to have rules. Nor do we have to create rules just to have something to write a program about. All that said, I have a comment about M2.
I don't think we need it. Bad Moderators have only one effect: Lowering the scores of good comments. The solution is two-fold:
The first step ensures that there are moderators who are reading the unfairly moderated comments at the -1 level. And the second step ensures that there are enough moderators to raise those scores up.
---
Put Hemos through English 101!
"An armed society is a polite society" -- Robert Heinlein
My head hurts.... (Score:2)
This article was posted at 12:11. By the time I got to it, there were -1:163 comments. When I got through those and refreshed with "newest first" there were 50+ more.
So, this post will probably get completely lost in the shuffle. My only hope is that Rob may read it, since he posed the question and actually cares about his community. The up-side is that I have all the previous comments to try to pull together.
First, when I've noticed that I had moderator points, I have tried to switch to -1, flat mode to be as fair and impartial as possible, but, if I were to try to moderate a topic such as this I'd have to quit my day job. A couple of things would help this:
Random Moderation
Nope. Been said all over this page. Moderation would be less of a responsibility and less compelling for those of us who care about the impact to the /. community. It wouldn't be worth the effort.
Meta-Moderation
I think M2 may work in 2 directions. Instead of just ranking the moderators, it could also rank your opinion of the moderators and be another factor in your preferences. I'd like to see the M2 boxes on all the comments. I don't think they'd be used that much, but when they were used, they'd be used on comments that people really care about. This should address the problem of moderators with agendas. Let's face it, if someone has an agenda, that happens to agree with mine, than that's a good thing to me.
Fractional Moderation
This is a good idea, both for moderation and for M2. It would take at least 2 moderators to change the moderation.
Unscored Moderation
Some people have pointed out that the categories available are not necessarily complete (like missing stupid), approriate (OR/UR being more relevant to M2), or positive or negative (some people LIKE Trolls, some don't want Funny). So:
Karma
Rob says it doesn't factor that much anyway, but I think it should be affected by only the most positive and negative moderations and M2. I know I said no numbers, but, while one person's Flamebait is another person's Funny, Offtopic is not good. Redundant is irrelevant, since everyone has different views (prefs., NOT opinions) it's probably not redundant to readers with different preferences.
Maybe instead of adding +1 to future posts, Karma could add personality tags. If a poster consistently gets Interesting:2 or higher, then that person is probably interesting. These tags should be clearly distinct from moderation tags, so the reader and moderators knows that it applies to the author and, not necessarily, to the post.
A Changing Face
This should keep Rob busy for a year or so, but imagine the possibilities. For a discussion about webcams in public restrooms, I could view at Funny:2, Interesting:3. For a discussion on security, I could use Insightful:2, Interesting:2.
Since this would put my viewing entirely into the hands of moderators, I'd probably add Normal:1 to any view. Or, I could read on pass one a topic for Interesting and Insightful comments, and a second pass for Funny.
Conclusion
The whole point of moderation and meta-moderation is to filter the /. experience for the sake of the individual participant. Being a moderator is a pain in the [insert your desired portion of anatomy here] and it is difficult to do fairly. Try to remember that if it weren't for their efforts, you wouldn't be able filter your view at all.
And thanks, Rob. It just keeps getting better.
Re:Karma -> Grade Inflation / Default good? (Score:2)
Also, for other reasons, I _do_ want to see scores when I get points to moderate. Nothing I hate worse than seeing a -1 post that is neither flamebait or off-topic, just merely an opposing opinion. As long as its someone's opinion with some valid arguments, it should have as much chance to be seen as any other 'good' post. Also, posts that are marked up waaay too much can be knocked down.
My feeling is that meta-moderation can be used to correct this problem, if there were only additional info in the m2 page. Sequence of moderation.
This might be real tough to do but here's what I'd like: When presented with a comment on m2 page that has had multiple moderations, show what the post was scored at the time the moderator marked it.
So, if a post ends up +5, when it shows up in M2 the first time, it might have an original score of 2 and went to 3. I would mark that as 'fair'.
The next time it shows, the original score was 3 and went to 4. Maybe I leave this alone, maybe 'unfair'.
The third time I see the post and it goes from 4 to 5 and its not a 5 comment and the moderator is just doing a 'mee-too' like has been suggested, it is markeed 'unfair'.
Now, the moderator who put the post over the top loses karma and maybe gets less points.
I know, but it was a thought...
(oh and Rob, showing me my Karma points is bad, makes me hit reload waaay too much to look at my post history to see what I did right/wrong) =D
As always, appreciate your work, Rob!
Problem: Karma feedback loop? (Score:2)
You make an example of a bad (GNOME-bigot) moderator moderating down 5 KDE-related posts. I think it's more likely (and I've seen on slashdot many times) that the moderator will moderate up 5 rather uninspiring or moderately interesting GNOME-related posts, and ignore more insightful KDE-related ones. I metamoderate and think "well, that wasn't THAT insightful. But I can't really say it was unfair either."
Personally, I don't see that as an easily solvable problem; you can't metamoderate taste in what people find interesting or funny. No matter how you randomly choose people, you will probably always end up getting the same ratio of, say, pro-GPL to anti-GPL moderators (whatever the ratio is in the /. reading community). But it would make matters worse if more prevalent views (and thus, higher karma) could feedback into more moderating power.
Oh, and I'd like to point out that I like your idea of metamoderating in the context of an article. If it wouldn't slow things down too much!
JMC
Integrated and Random M2 moderation (Score:3)
All in one, I very much like the current 'daily random M2', because 1) it gives me a warm fuzzy feeling that even such random samples are usually 90% well-moderated 2) I can be sure conceptually that the bad guys will be caught sooner or later, and it might be me who catches one of them. M2 is really like law enforcement, you both want to patrol on the streets but you also want to do larger scale (and more random) searches.
The solution? Honestly I dont really know. Maybe this hybrid method: there would be a 'get me 1 random comment to M2-moderate' link right at the top, but also it would be possible to M2-moderate 'in the field'. Thus I could eg. random-M2 5 comments, and realtime-moderate another 5 comments. The 'split' between random and realtime M2 is thus up to the M2-moderator.
Re:Moderate thoughts on Moderation (Score:2)
How about being able to edit your own boneheaded posts? A lot of boards do this with a quick (edited by: username, time) stamp on it. Bye, bye spelling/grammatical correction posts! (note: I have largely avoided the MM discussions so if this has already been shot down I apologize)
I think meta-moderation is a good idea (except that I get asked to meta-moderate first thing every day), even though I guess I was a bad moderator and had my karma dropped to a point where my comments no longer start at a "2"(which could have been me posting comments without the +1, which I do for quick nitpicks or other things that would most likely be moderated down to 1 for being "too high", which could be a bug or a seriously nested parenthetical aside). I am actually right on the line, if someone moderates this (or any of my other comments) up, I will return to the blessed 2 status.
Increasing the limit could also help for the sig/noise ratio. With 4-5x as many moderators and limits up to 50 or 100 there would be a better stratifation (new word!) of comments and could actually make a Top 10 posts of the Week mini-site useful. I think there is a lot of great content here that goes unnoticed (and a lot of crap to wade through to find it).
Agreed: keep "living" discussions visible! (Score:3)
Imagine a page or pages, linked to Slashdot's front page, called:
- Current "Liveliest" Discussions: ordered by comments/minute
- Week/Month/Year's Hottest Debates: ordered by total no. of comments
- Active Debates: "old" articles with recent comments (ordered by "recency"?)
Etc... other criteria are possible.
Wouldn't this add a fascinating static element to Slashdot? Eventually certain discussions would emerge as "Slashdot's All-Time Great Debates" , with thousands of comments from a period of years. Set your threshold to +30 to savor the distilled, democratic wisdom of 30000 slashdotters
What do you think?
Dugan
More ideas... (Score:5)
Writing a comment would be optional, as many moderations are simply self explanatory. But, it could be useful to say "I moderated this down because it was repeated above in comment #xyz," or "I moderated this down because we aren't discussing foo here, only bar." Of course, positive feedback would be good, too. Good posters could be rewarded with feedback like "Good post," or "Nice job researching this subject."
This information could then be made available on the M2 moderation page, permitting people to get an idea about why a moderation took place. A comment that might look perfectly normal could have been grossly off topic. Further, it would give the original poster an idea about why he was moderated up or down. Everybody wins.
Of course, the moderation comments would have to have limits. Perhaps the complete history of the moderation comments would only be available to the original poster, while everyone else only saw the most recent. This would prevent silliness like whole discussion threads on why a post was moderated in such and such of a way.
Anyway, it's just a thought.
possible results of the suggested moderation syst. (Score:3)
so they search for a similar one that does express a similar idea (or simply repeats most of the idea) that they can moderate up.. and the will moderat that up. so the whole moderation process would loose sharpness it would be somehow blurred. it would not help to find the perls form the posts...
mond.
Use a reputation system. (Score:2)
Set up a directed graph, with the nodes being the slashdot users, and the weighted connection between user A and user B being A's average opinion of B's posts. (Maybe put in an "uncertainty" factor, as well-- only having rated one post means the rating has a high uncertainty, while having rated dozens means the rating is quite accurate.) Now, use paths through the graph to determine whether a post should be seen. If I like B's posts, and B likes C, then I should be likely to see C's posts. This way, there is no one universal standard for a "good" post. If I happen to think that it's really funny when people flame each other and post a bunch of offtopic stuff, I should be able to adjust my ratings so that is mostly what I see. On the other hand, if I want to see well-balanced, reasoned posts, I can do that, too.
This kind of system would mimic the kinds of trust systems you see in real life-- you trust the opinions of your closest friends, those of friends of your close friends a little less, and those of random people off the street not very much at all.
You could also set it up so if there is no path between yourself and a given user, you just look at the overall average of paths coming into that user and use that to build a reputation score. You go back more than one level deep, though, so if 10 people rate user A highly, but each of these 10 people has received a poor rating from 10 other people, A's rating should not be that high. (In effect, the first extra level deep is your "meta-"moderation and the extra levels are even more exponentially diminishing "meta-" levels.)
Of course this system is based on scoring per individual, not per post, but I think it's safe to assume that the overall quality of a given user's posts is going to be fairly consistent.
I'll admit that this may be more expensive in terms of implementation complexity and resource usage, but I think it's a conceptually clearer system than what exists now.
Re: Karmic disturbances (Score:3)
M2 approval. (Score:2)
M1 is in place for several reasons:
1. Removing spam and flames
2. Adding notice to well written articles
Where exactly does this fail? several points have been raised:
1. People push their own agenda, moderating up articles by friends or whose topic they agree with, rather than the reasoning behind it.
2. Its a simple fact that some people have very different views from others.
In a collaborative moderation scheme such as this, it is entirely necessary to limit these bad effects as much as possible. The first attempt here on slashdot was to minimise the damage moderators could do, by giving them an extremely limited amount of power.
I believe M2 represents a better solution. While it cannot have effect instantly, since it will take a short amount of time for karma ratings to settle properly and for people to get used to the system, it now gives us the ability to not only limit those moderators who moderate inappropriately, but also to give more power to those who regularly moderate well, thus reducing the need to have more and more people moderate (reducing its special status and making it a chore) while at the same time, increasing the likelyhood that any given moderation point awarded is from a moderator whose moderating methods meet the approval of the community.
In this case, the presence of M3 or higher would never be necessary, M2 represents the will of the community as a whole, and M1 represents the average of that will, with a human intellect.
All in all, an excellent system, and as a collaborative system (as opposed to personal moderation, which is another issue entirely) I believe it only requires some tweaking to achieve excellent results.
Anal retentive? I don't think so (Score:3)
Look, I usually have neither the time nor inclination to read a bunch of "m1cr0s0ft sux" posts, nor "First!", nor posts that are nothing but flamebait. Sometimes I do, and I read the comments with my threshold set at -1. It's at 4 most of the time, but I still go to -1 when I can to get the flavor.
But I'm not being anal. This isn't about what I should or shouldn't read, though I do skip posts I find useless. This is about trying to filter the massive amounts of information that come my way every day. If you have the time and desire to read it all, have at it! I'm actually jealous, since Lord knows I would like to do so.
Sadly, I can't, and neither can a lot of other
There, I feel so much better now.
a really crazy idea (Score:2)
On each comment, have three radio buttons for Good, Bad, and Average, set to Average by default. Each comment gets a score, which starts at 0. Now, whenever someone gets moderator status, have them pick a button for every comment. When they click "Moderate", "Good" posts get their score incremented, "Bad" posts get it decremented, and "Average" posts are left alone.
Now for the important part: When the page is displayed, posts with scores in the top 15% get "Very good", next 20%, "Good", next 30%, "Average", next 20%, "Bad", and last 15% "Very Bad". Or those names can be changed to whatever. Since this system needs more raw feedback to work right, increase the number of moderators or frequency someone gets moderator access.
Another idea is to let everyone moderate all the time, but then people might not take it seriously, or not do it carefully.
Note that the method of determining the final category of a comment by definition averages out the individual moderations, so it is very difficult to abuse the system.
Possible problems would be the complexity of the code, especially keeping track of which comments a moderator has already moderated (because they get to moderate all comments). However, it doesn't seem much harder to implement than the metamoderation system.
What's funny? (Score:2)
However, the one time I can recall being moderated up for amusement value (I think the post in question made it up to #4) was for a carefully-done parody of the article I was posting to. It was no one-liner, and it required about a half-hour of thought and editing on my part -- probably more raw effort than I've put into most of my
Butanyway. The problem with "funny" is that it's easy to abuse, but I don't think the solution is to get rid of "funny" as a moderation option. I'm not sure how to encourage better-quality satire, however.
OK, what's up with this?? (Score:3)
In any case, it's getting a bit tedious IMNSHO. It's not that I generally have anything against the comments contained in posts that also have this statement, but the statement would make me, personally, less-inclined to moderate the post in *either* direction.
A Most Excellent Idea!@ (Score:3)
Of course, there is the slight potential problem of moderators who can't otherwise post to the discussion trying to get their two cents in that way, but limiting it to one line will cut down on that.
In addition, it would help pick out problem moderators. If, for instance, someone is moderating a post down and puts as a comment "You're a loser. Go away!" or something similar, we might have a clue that there's some personal bias going on. Well, that's not quite universal. I used to be a sysop on a YAWC BBS, and we used to put comments on people's Aidelines like "twitted for bad info." One such comment by another 'op was "Abusive Loser. I hate that."
Perhaps a better example would be the inevitable "Go away Katz" posts, if they were being moderated up with a "Yeah! Katz sucks, heh heh heh," or something similarly unproductive. Or moderating a Linux criticism down with "Are you nuts? Linux r00lz, dude!" You get the idea. Also, I know that a few times when I've moderated, I've wished for a "misinformation" category with which to mod something down,sort of like the opposite of "informative."
Like I said, though, the above poster has an excellent idea!
Problems with Integration (Score:3)
Although the improved context aspect is appealing. The problem I forsee with integration is due to our threshholds.
I don't mind taking 5 minutes to meta-moderate and then be done with it. I don't mind, in that context, reading a few loser comments, which have likely been appropriately moderated.
However, if M2 was integrated, to properly participate, I would have to do it throughout the day as I read articles, with which I would probably lose patience. Or if many of the randomly chosen comments had low scores, I wouldn't even see them at all, since I like to read articles with a threshold of 2.
Furthermore, how many randomly chosen comments would be set for meta-moderation? As it is, I only can meta-moderate 10 per day, would this be kept track of if the comments were integrated?
---
Re:Intelligent... (Score:2)
Actually, why not make karma be an average of post moderations? Every time it hits a integer, that score is added to your posts by default.
So, for instance, if your posts are in average moderated to 2, you get +1 on your posts; if your posts tend to be moderated to -1, you get -1. In the rare event when your posting average is +4, you get to post posts rated 5 by default! (Like that would ever happen.)
Don't know; it doesn't sound perfect at all. But the idea of alloting karma with regard to post quality average instead of blunt total sounds good.
"There is no surer way to ruin a good discussion than to contaminate it with the facts."
Re:Intelligent... (Score:2)
So the odds of some asshole scoring 5 on 5 successive posts are pretty slim.
"There is no surer way to ruin a good discussion than to contaminate it with the facts."
Moderation and karma (Score:5)
"There is no surer way to ruin a good discussion than to contaminate it with the facts."
Re:IMHO (Score:2)
I'm worried if moderators are losing huge chunks of karma from meta-moderation. If that's because they're not moderating well, fine, but if it's because people don't understand or abuse meta-moderation it's not. Could there be another reason for 20 point drops, such as a newly implemented time horizon, or changed rules for karma totalling?
Re:Moderation and karma (Score:3)
Disclaimer - never gotten a "Funny" karma point yet -- but appreciate those who can and do.
Some thoughts about moderation... (Score:2)
That having been said, I think that there might be one area where moderation might be improved:
Instead of (or perhaps in addition to) giving someone more moderation points based on karma, how about increasing moderation based on the total score of all their previous posts in a given topic? Say for instance, that a person has posted a number of comments under a given topic, all of which have been moderated to a four or five, since they have all proven to be relevant, informative, insightful, etc. When later on, this user gets their moderation points, they get X points for general moderation, but Y points (where Y is a function of their total posts' moderation points) under the topic that they have been posting under? The point here is that if someone's posting under a given topic, and they have regularly posted well (and this could include some kind of deduction if they similarly post flames, trolls, etc.), they probably have a good understanding of the topic, and should perhaps be given a little more leeway and/or (dare I use this word?) power under that topic.
This power would, of course, be subject to revocation if the meta moderation found them suddenly becoming Mr. Bad Moderator (an obvious flaw in my suggestion, I know.)
Just a thought.
The 20% rule (Score:2)
You mention that you believe that in general the majority of people are good and behave in a manner consistant with this evaluation. Ergo, the majority of people when given moderation powers will exercise them with the best possible judgement they can (we're not perfect and moderation is an exercise in personal perference to some degree or another).
So then why limit to 20% of the comments. By your own argument this restriction would do more harm then good, as there are more good moderators who would be hindered by the restriction then there are bad moderators who would abuse the ability.
Add to this the idea of giving karma more overall effect in the moderation process and I think you've pretty much solved the problem. Bad moderators will not gain moderation status as often, thus further raising the proportion of good moderators. Adding the 20% rule to this would only exasberate the problem. In my opinion
Having everyone always moderate is dangerous (Score:3)
The trouble with this is that most people will soon cease bothering to moderate. When it is a rare occurance, people will take it as an honor, and work to do a good job. When it is a constant duty, people will slack off and not moderate. The only people who will moderate are those with very strong opinions on the subject, and I'd suggest that they are the least likely to be objective.
If you take KDE vs. Gnome as an example, with random moderation, you have to assume that a large percentage of the moderators will be in that largest class of people who doesn't greatly care one way or another and moderation will be fair. If everyone has the power to moderate, most people (the moderate moderators) will be sick of moderating and probably won't bother. The only ones who will are those who think it is really important to moderate posts up or down. Those are likely to be the more fanatical posters, and you won't get objective moderation.
Moderation will just turn into a new sort of internet poll, and we all know how accurate those are!
Ideally, you'd always want moderators who had no opinion on the subject. Since that isn't likely possible, choosing at random is likely the next best bet.
I do like your other ideas, though. The averaging of scores would help one problem, which is that in popular topics you seen pages and pages of fours and fives while in a lesser used topic, you might see only a couple of twos and threes. Obviously this has little to do with post quality!
Re:Ya know... (Score:2)
More Moderation Madness (Score:5)
A large portion of comments deserve no more or no less than the 1 (or 0 for AC's) that they get.. when you only display moderation controls on only part of the coments, almost all the comments with controls won't really be deserving of a moderation up or a moderation down. I think when this happens, some moderators, esp. the new ones will be just "itchy" to spend their points on something before they go away, and will moderate all the posts they can, regardless of whether they are deserving. (I may be wrong on this though)
What I feel might be a better solution than only having the moderation controls on a few of the comments.. Have a lot of moderators with only 1 or 2 points so your exampled Gnome bigot won't be able to moderate down all the KDE comments he sees , yet the "good" moderators will still be able to put the deserved -1 on those "Let's petrify nitrozac and hump her leg" posts.
Karma Feedback leads to Nirvana (Score:2)
It could be as simple as the default value for your comment being the arithmetic average of past comments (within limits), or it could be more complex.
Karma effect to turn slashdot in mud ? (Score:2)
The opportunity to see what is your Karma value tend to let you think "I have to improve my Karma skill" or "I'm chaos devoted, i've to be the worst"
Perhaps renaming it with a no-meaning name would prevent this sort of behavior, or just seeing as a binary value : can moderate/can't
( and I apologize for my poor english-writing skill, need to improve it
Re:Karma -> Grade Inflation? (Score:2)
+5s are pretty rare, 2s are very common, 3s less so, 4s fairly unusual. I might seem that -1 is assigned rather too often, but that might just be an illusion based on the fact that the bell curve ends at -1 on the minus side, so you have a "piling up" of all bad posts in the -1 category.
If what you are saying happens, you'd expect to see more 5s than 4s. Especially considering that the bell curve ends at 5 on the plus side, you'd expect to see more 5s than 4s ANYway, but I've not noticed this to be the case.
How about it Rob? What's the statistical breakdown on all Comments?
Re:More Moderation Madness (Score:3)
Conversely, I feel that truly old hands with very high Karma should have more points. This will aid /. in maintaining it's culture and flavor.
What I'm thinking is something like:
All of the above are self explanatory except the [Nirvana Threshold]. This could be some threshold that's quite high, possibly the 4th standard deviation positive from the distribution of all Karmas. Maybe a factor could be thrown in to lessen the influence of account age, but you get the idea. I'm not sure whether to round up or down the moderator points assigned. Of course, no fractional points should be allowed (hmmmm.... no that was dumb, no fractional points allowed).
This system has the significant advantage of being very geeky, but still understandable to someone without more than High School Algebra.
I really feel that your /. seniority should have a real weight in the matter, not just a cutoff (no newbies). I respect the old-timers who have been consistently moderated up in the past.
The problem with basing ability to moderate and moderation points on Karma alone is that newbies , and I'm probably guilty of this, might be working too hard to increase their Karma. This might tend to generate a lot of popular Comments that are really just there to get applause rather than real meat.
An advantage to allowing new moderators just a few points might encourage these new moderators to take great care in assigning them.
Another benefit to penalizing new accounts on a sliding scale is that it would tend to discourage people from creating new accounts just because they are in a Karmic hole.
One potential problem is that someone with only 1 point might be more likely to only moderate up. I know I wouldn't want to waste my point on some Troll. Possibly assigning points in pairs (+/-) would help here. I like this. Moderation points are actually quantum fluctuations.
Here's a question for Rob, What is the "average" moderation? + or -? If you assigned in pairs the "average" moderation would tend to be zero. Not exactly zero as some people might not use all of their points before expiration. Would this be a good thing or a bad thing?
In general, the idea is to increase the reliability of moderation, while allowing a broad cross-section to participate. A sliding scale, based on your Karma and seniority would help to promote this, I think.
Re:Karma -> Grade Inflation (Score:3)
a modest proposal (Score:5)
Rather, each reader should have the ability to see a customized view of
For example, if I see a post moderated up to "5" and I think's it's a bonehead post, I would like to de-value the opinions of the moderators who liked that post. This is different from meta-moderation: I'm not saying that these moderators are bad and should be prevented from moderating, I'm saying that I don't want to see the results of their moderations.
Similarly, when I read
Eventually, each user will build up a profile of posters and moderators whose opions they trust and distrust. One can picture people trading these profiles, merging profiles with other people's, etc.
As the amount of information on the net grows, developing a systematic way to figure out which of it you trust and which of it you don't is going to become a very important problem.
Didn't Brin have some sort of reputation management system in Earth? Reading up on this might be a good place to start.
Anyway, the point I want to make is the people have different tastes and want to read different subsets of all posts. There's no reason to force the same view of moderation on everyone.
Newsgroups/usenet does suck (Score:4)
So obviously it's important for Slashdot to have a good moderation system, otherwise we might end up with a SNR equal to that of newsgroups.
The good thing about Slashdot is that we can add and take away as many features as we want, because there's no protocol involved here, only code. So we can try one thing, see how it works, and if it doesn't work out, try something else. I think though, that we have to try not to make the moderation system *too* fancy. If we make moderation too difficult and subject to review after review after review, then I think moderators will begin to ask themselves "what is the point?". I would hate to see something like this [segfault.org], for example, become reality.
On another topic, I think that meta-moderation in the comments list itself is a bad idea. Let's say I'm person A, or a friend of person A. Person A gets their post moderated down. I might choose to meta-moderate that post. Even if we ban the poster themselves from meta-moderating where their own posts are involved, we can't stop them from telling all of their friend to go there and do it for them.
Moderation, Reputation Managers and Slashdot (Score:2)
Slashdot was mentioned as one of the shining examples of 'Reputation Managers' by Jakob Nielson in his latest Alertbox column [useit.com] on USEIT.COM [useit.com]. In the article Nielson is basically saying that Reputation Management systems are up and coming ways of adding important value to the web. Only thing is, he has one rather pithy comment about useability on /. (he is, after all, a usability guru). Quote: "Unfortunately, the ability to filter out poorly rated comments is not turned on by default, so only diligent users who study the slightly confused user interface will discover this useful feature."
Personally I don't find the /. system that difficult to understand, but I do remember having some difficulty finding documentation on how to use the moderation/scoring/karma features. I know there is some stuff in the FAQ and all that, but a more formal set of documentation might be in order.
I know, I know. Documentation is always the last thing a programmer gets around to writing. But look at it this way, with /. being a continual work-in-progress there is little chance any one will get around to doing some simple 'man pages' for /. anytime soon unless it is made a priority.
Perhaps we users should take this task on for ourselves. No, I am not volunteering, or at least not to do all the work...
Jack
Another method of moderation (Score:2)
When M2 appeared, the first thing I thought it that it opens up a pandora's box of moderation levels. The moderation system tends to work well, and the M2 system should compensate for rogue moderators. but there will be calls against "bad" M2 moderators. This is inevitable. But what can be done to moderate the moderator's moderators?
Simple - instead of (possibly) adding additional M's of moderation, maybe it would be better to solicit feedback from the people who post to /. On a separate page - accessable only to users with accounts - allow all users to view a sample of M2 moderated posts and comment on their fairness.
This could work kinds like the quizzes that appear on the sidebar of the main page. There might be five or ten M1 and M2 moderated comments that people could look at and vote on fairness. If an M2 moderator consistently (over the space of weeks, months?) is out of line according to the /. users, then their priviledge to M2 moderate will diminish accordingly.
Now here's a (possible) catch. In addition to real M2 moderated posts, a few fake posts - the specifics of this are a little beyond me right now - would be included. These posts would serve to gauge the response of the /. users (some would be obviousally biased, others not so much so) and would just throw some randomness into the equation.
There's not going to be any one final solution to keeping any forum free of worthless posts, but this system would do much to bring the /.'ers back into the fold of self-administration. Any thoughts?
Re:What we really need... (Score:2)
Having had my very first /. posting (a polite statement of opinion, butressed by a few facts based upon my direct experiences with the item being discussed) labeled 'flamebait' makes me want to disagree. Dare I?
(Ok, so maybe I wasn't completely polite: I used the word 'sucks.' However, it was in reference to an inanimate object.)
I'm not new to this. I've posted in chatrooms|forums|newsgroups|whatever for over 10 years, and was completely surprised by instant negative karma.
Maybe not based on some moderators comments, but on what Rob/Hemos/Katz/crew actually read.
Although it would probably have helped in my situation, I'm sure Rob/Hemos/Katz/crew probably have enough to do already. I'd prefer they spend the time much as they already have: Sifting through the contributions and posting the relevent ones.
Hmm, the majority of my non-AC posts are about moderation. I think I'll stop now.
--