Slashdot Log In
Interesting Moderation Proposal
Posted by
michael
on Sun Oct 01, 2000 10:53 PM
from the NP-hard dept.
from the NP-hard dept.
Kuro5hin is running a story with some interesting ideas for "the perfect moderation system". I'm not sure I care for the overall system but the idea behind it (of 'balancing' out parts of the site with strategic bonuses/penalties) is intriguing.
This discussion has been archived.
No new comments can be posted.
Interesting Moderation Proposal
|
Log In/Create an Account
| Top
| 263 comments
(Spill at 50!) | Index Only
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Collaborative Filtering vs Set Moderation (Score:3)
CF can be integrated in various ways, but the method I'm thinking of would work similar to this:
1. Everyone votes on various posts at will. There is no limit to the number of posts you can vote on.
2. Your votes are compared against other users votes to find the difference in "taste" between you and every other user.
3. Posts are weighted based on the distance between you and other users. Users who vote like you and vote on posts are weighted higher than users who don't vote like you and vote on posts.
The underlying math to implement this is rather simple (with the exception of some of the baysian network models) and widely known, but as far as I know, there are only a few very limited open source implementations in existance.
Re:All Moderation is flawed (Score:3)
(FWIW I'm currently moving away from collectives such as /. and towards weblog-type things. My favourite source for the unusual and nerd-esq is Penny Arcade [penny-arcade.com]. Sure, it's mostly centered around a comic strip, but the journal entry / news that is attached to each strip is a very good read (even though it's a bit computer game centric). Otherwise I'm returning to traditional news vendors -- I have BBC, Fox, ABC (Oz), Salon & Wired all syncing to my TRGpro.)
Re:Resident Karma Whore, move over. (Score:3)
And if you want to see an example of Slashdot just before wide scale moderation and karma went in, check out: Slashdot Moderation Phase 1.1 [slashdot.org]. Even the Anonymous Cowards generally had good comments.
S/N ratio was pretty high in those days, much like kuro5hin now, but from the look of it, community moderation and a 'reputation system' has failed in it's goal to keep ratio up. Instead we've gotten a Karma Whore 'n' Troll circus with lots of anonymous flamebait on the side, which is entertaining and might generate more pageviews, but is probably not good keeping readership up in the long run. Which is what you'd expect when you let a site 'run itself'.
Unfortunately, it's probably only a matter of time before kuro5hin is 'hacked' and it's system has been rendered as useless as Slashdot's for encouraging valuable posts. You are damned if you do (let the readership moderate itself) and damned if you don't (admins end up censoring content, people rebel against 'elite' moderators.)
--
Natural Language Processing (Score:3)
I have seen their products in action and they are *extremely* impressive. What might be of most interest to you is the way their algorithms work. Now I'm sure they won't release that info, but you may be able to glean enough info from their material to get on the right track. (Start here [autonomy.com] perhaps). Basically their technology is based on Bayesian algorithms (Bayes was an 18th century English cleric who came up with some cool ideas about NLP that couldn't be proven until recently when the computing power and information volume to make them work became available - how's *that* for far-sighted!) combined with Shannon's Information Theory. It is WAY powerful in practise!
I agree that it would be really cool to see some kind of automated moderation system based on this type of principle. I'd also *love* to see some Open Bayesian NLP work.
Good luck with the experiment.
"Give the anarchist a cigarette"
There is no perfect system (Score:3)
I think that in the end, moderation is really just a personal preference issue and not worth that much effort. None is boring, some helps, but too much is worse. Though I must applaud the efforts going into this, and I do enjoy the debates the issue provokes, maybe instead of looking at it like and trying to build the "perfect" system, we might be content to just call it exploring options or do something more interested. But trying to "perfect" moderation seems silly.
How to implement Spendable Karma (Score:3)
1. the donor has more than 20 karma (e.g. is a good player);
2. the donor can push up a post only to a total max per post that levels it out at 2 points (e.g. nasty people mod down a good post to -5, nice donors can only spend it up to +2, cost of +7);
3. extra karma spent to buy moderation costs 10 karma for every karma given (e.g. +7 would cost +70 karma);
4. each donor could only spend 10 karma per story (e.g. you'd need to find 7 people to back you up in my example, and they would not be able to donate to any other post in that story);
5. a donor can only donate up to 10 karma once per day (to keep karma whores from abusing it).
Concept?
Re:Unfair or Inaccurate moderation (Score:3)
1) If you think you were unfairly moderated, challenge your moderator to a duel (Say... High noon in front of the Geek compound)
2) You beat each other until somepody passes out.
3) The loser has his picture taken in all his bloodied glory, and posted in a new 'moderation' photo gallery for all to see.
Maybe that will get the moderators to be a little less biased, having to worry about their own butt.
jdb
There's something missing... (Score:3)
All of the systems I have seen either totally lack feedback at all (for instance, Slashdot's miserable attempt called MM) or have some type of positive feedback - this proposal included. I may be wrong, but it seems that a "poster" with an "idea" can grow to wield power. Someone accumulates enough points, he or she can consistently post to the most active categories and be seen. What would this lead to? A few people who are very active and dominate the board.
I don't know what to propose as a negative feedback system for this type of situation. All I know is that I see a desperate need for someone to design a system in which all users share power equally, however cheesy that may sound. I would love to see a proposal that intends to keep power out of a select few individuals so we rarely see the same name twice. Everyone could have the power to moderate every story they see. The bad stories will quickly drop off the front page, and the good ones will stay. The site won't have the highest-quality news, but you simply can't have both.
What you will get from a site like this is a myriad of opinions and views from all sorts of places. It won't be CNN, and it won't be nearly as homogenized as
This is just an idea, but such a site would cater to the crowd that kiro5hin has attracted. Hence, I think it would provide a nice counterpoint to Slashdot, where one can get a more mainstream, less off-the-wall and imaginitive, quality news source.
Again, admittedly, I didn't inspect the original story too closely..I read the first half and skimmed the second. Just felt like giving something to think about though.
Collaborative filtering? (Score:3)
Basic idea: allow everyone to mark whether or not they like particular articles and/or authors, then use that profile for each user to find the articles and/or authors that user is probably more interested in.
This is even more useful when you compare user's profiles against each other so that if two users have been marking messages/authors in such a way that their profiles are very similar, then when one of the users marks a message/author favorably, it automatically gets a higher rating for the other user (or down, if marked unfavorably).
In effect, each user will statistically get messages rated through a moderation scheme based on their own past history, and the history of other users who happen to agree with them.
So, how come this kind of filtering/moderation system is never discussed? Is it too computationally difficult to implement on a large scale?
Re:Spendable Karma (Score:3)
Even if it is just an added option to the current system. To make this transparent the spender/s should be identified!?!!!! That would be really interesting and would prevent siggy from dominating moderation for the first couple of weeks.
Just think...do I spend on this comment and lose my +1 for the good of humanity? If it was done transparently this would be even more interesting as your moderations would tell as much about a person as their posts.
I say lets do it.
All Moderation is flawed (Score:3)
In my opinion all moderation is flawed because it relies on human nature which is inherrently flawed. I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.
Second Law of Blissful Ignorance
Re:I have it (Score:3)
I always thought it was some form of performance art.
--
Traditional Wisdom (Score:3)
If you don't like a post, just click "-1, don't like it" and be done with it. Don't like a story, moderate the story as "-1, post-IPO-esque" and you're through.
Strong opinion about a particular user? Moderate every he has posted and ever will post. Just think, with out combined might we can hand out a bitchslap even the Taco would envy.
Heck, you even moderate moderation. If you disagree, there's "-1, wrong". If you like it, it's "+0, right" (because if its right, it doesn't need to be moderated). Hell, you can moderate the whole moderation system; if it gets low enough, it will be defuncted and Anarchy shall reign.
Moderate Slashdot as a whole...negative moderation will bring more stories confiscated from Kuro5hin, positive yields more Jon Katz stuff.
It's extends to the micro-level as well. Moderate the topic list, color scheme, each others' passwords...
Moderation at the bit-level may be difficult, until we get quantum computers and get moderate an individual bit as "+5, very" rather that just "0, false" or "+1, true".
My mom is not a Karma whore!
Unfair or Inaccurate moderation (Score:3)
The moderation on a lot of articles is just plain wrong. Or opinionated. Some moderators rate articles not on the merits of the content, but on how much it angers them or tickles their fancy. Articles that follow the groupthink of the cult get good moderation.
Well it seems sufficently complex.... (Score:3)
On my reading of the summary the system doesnt seem to have a way to deal with this. Indeed it doesnt really seem to be adaptable to deal with this.
On a positive note offensive comments should go down quicker than they do on
Since this is going to lead to an inevitable discusion about
However i dont really think that the system would be an improvement on the present one, which, given the circumstances, works rather well (though i might only be saying that because i got +4 karma today ☺).
Resident Karma Whore, move over. (Score:4)
My gut feeling - it won't work. I think the ideal moderation system would be based on what is called the Delphi Effect, if I understand it correctly. Basically the more people you have moderating, the closer to the "true" rating it will be. ie: if 80% of people believe a post is +3, but 20% believe it is -1, it is +3. No averages. A sort of majority rules. It depends, however, on alot of people moderating.. to the point that there are more moderations than there are posts. But, my ideas aside...
The problem with this proposal is two-fold: One, it has no way to detect 1 person using 10, or a hundred, or a hundred thousand, accounts and thus biasing the voting system. It's a problem prevalent here on slashdot where the trolls have created throwaway accounts. Limiting on IP address doesn't work, because many are behind firewalls and hence multiple users can legitimately be on one IP. one account per e-mail address doesn't work either - e-mail addresses are easy to get.. often for free. The net result is a small group of determined attackers can destroy the system (sound familiar?).
The second problem is related to the first. Their idea of having the users rate themselves initially is a very good idea (rob, you paying attention?) but it suffers from the fact that someone can simply moderate their own posts, and gain a point advantage.. and we're right back where we started.
The key to moderation lies in accountability. You can create the best system in the world - but unless you can enforce some kind of "one person, one vote" standard, it will always be open to abuse.
Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long. The people who took down the slashdot moderation system did so in an organized and systematic fashion. These people are bored and have nothing else to do - but you DO and hence are at a disadvantage. Once the system is in place, PLEASE ADAPT IT - don't just deploy it and forget about it. It'll need to be tweaked, updated, maybe even entirely thrown out for a new system. Trust the wisdom of Strousoup(sp?) on this one - design the first implimentation to throw it away, you're going to do it atleast once anyway.
Cheers,
Signal 11
--
speaking of moderation... (Score:4)
Some of you may feel that without such a cap, it becomes a game (as in "karma whoring" AKA "Signal 11's life"), but I think that's irrelevent. The only "downside" to this "game" are more interesting, informative, and funny posts -- not much of a downside, eh? Some call it "karma whoring", but I think that it's simply the knowledge of how to be a good Slashdot citizen.
Yes, I check my karma totals, but not because I feel it's a game. I use it to gauge the overall effect my contributions to the site by observing the rate of karma growth. And if my karma drops suddenly one week, I know it's time to cut back on the flames and trolls. ;-)
And yes, it's also fun. I'll admit that.
I understand that one of the major reasons for having such a cap is to avoid abuses like this: let's say Signal 11 post interesting, insightful, and funny things for a year. From my experience, an active, consistently good poster can, on average, easily get 100-200 in a year. This means that Signal 11 could start posting Goatse.cx links, and it'd be weeks before he loses his +1 bonus.
I have a proposal to solve this: if a user is moderated down more than 20 times in one week, he loses his +1 bonus for a week. If he is modded down more than 30 times a week, he posts at 0 for week, then at 1 for a week. If he's modded down more than 40 times a month, he posts at -1 for a week, 0 for a week, and 1 for two weeks. I think this not unreasonable, considering that most users with enough karma to get the +1 bonus won't suddenly become PBG or *syringe. :-)
Who else agrees that the +50 KarmaKap has to go?
---------///----------
All generalizations are false.
Re:Resident Karma Whore, move over. (Score:5)
Anyway, the mod system described is not the one on K5. It's in Glasscode, which is the system the article is about. Which does not run K5. (Note: I'm not sure if you understood this or not, Sig11, but a lot of other people got it completely wrong, so that's for them).
Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long.
Which part of the statistical system do we obscure? If you want a full and detailed explanation of how the various K5 systems work, see our relaunch article [kuro5hin.org].
Otherwise, I agree-- the system always has to be evolving. Think about it-- you cannot create an automated system that isn't eventually susceptible to automated attack. It's that simple. You just have to make it hard enough to attack that it's not worth it, and use the lag time to keep ahead of the kiddys. And sooner or later, you lose the race, pick your ass up, and try again (c.f. this summer for K5).
--
There is no K5 [kuro5hin.org] cabal.
Moderating articles for /. (Score:5)
I don't want to see this article again and again
or
Please post this article again and again and again
so CmdrTaco and his very diligent team will know which articles we want to see redundantly.
Slashdot - News for Attention Deficit Disorder. Stuff you saw yesterday.
On a slightly more serious note - surely Slashdot must be getting very uninteresting for the Slashdot admins if they arent' reading their own site. What does that say about quality control?
For starters, just /document/ what slashdot does! (Score:5)
Very little of how /.'s moderation and meta-moderation works is documented. How come my karma never goes above 64 even though I get moderated up? Why is it that occasionally it just drops a few points even though I haven't been moderated down? (Does karma age?)
Why can't we talk about moderation somewhere on Slashdot? If it gets brought up in a normal discussion, it's -1, Offtopic. I've never tried to submit a Slashdot article that concerns Slashdot itself, but the people who have say those are rejected.
How about a new category: -1, Herdthink, for those posters who just spew the party line about "and this is why Linux is so much better." At the very least they shouldn't be getting Insightful points for copy'n'pasting stuff from the FSF or OSI's webpage verbatim.
If we had a better FAQ, it would at least contribute to more "Insightful, Interesting, and Informative" discussion about moderation.
Enh, just my two timeslices.
Re:Spendable Karma (Score:5)
The first problem I see is that the first posts to an article are the most likely to be modded up. Moderators tend to hang around the couple most recently posted stories.
Not everybody refreshes Slashdot every two hours. The people that do, are the ones that agree most with the stereotypical slashdot agenda.
Insightful posts take time. It could easily take an hour to *read the article*, do some other research, and post some meaningful commentary. Those who post fast seem more likely to spout out their gut feelings.
To sum up: The people who post first are likely to be avid slashdot readers and more zealotous. Posts that are made soon after the article goes up are not as likely to be based on facts.
On hot trigger issues such as this one, I have read comments soon, then comments later and been pleasantly suprised by a couple better posts that get moderated later. Often on looking further, I notice that there are several more that I would have modded higher than the ones that are modded higher.
Let me try to illustrate this with a graph:
PostQualityv sTime:
|high
|
| +---+
| +++--+
|+-+ +---+
|++ +---+
|++ +---------------
|++
|+-+
|++
|++
|
|low
+-----------------------------------------
time-- ->
Sumofmoderationdone ;+-----+
|more +-------------------------
| 
|+--+
|++
|+
|++
|+
|+
|+
|+
||
|
|less
+-----------------------------------------
time--->
As you can see from the graphs, I think there are a lot of good comments posted later that don't get moderated, while a lot of earlier comments that might not be quite so good, do.
I suggest the golden moderation system.
You get 5 moderator points.
2 of the are gold.
2 of them are silver.
1 of them is bronze.
gold points can be used on any post at any time. Silver points can be used on posts attached to articles that are more than 2 hours old. Bronze points can be used on posts attached to stories that are more than 1 day old.
I think this would really do wonders for Slashdo
More ideas for slashot moderation (Score:5)
+1, Troll
+1, Whore
+5, Signal 11
+1, Slashdot Already Posted This
+1, Claimed They Were Expecting -1
+1, Redundant
+1, Only Intelligent Post in the Entire Discussion
-1, Stupid
-1, Clueless
-1, Opposing view
-1, goatse.cx
+1, goatse.cx
There could be many more, of course
Spendable Karma (Score:5)
You need to be logged in to moderate
You must have karma to give karma
It would actually give some type of value to karma
;-)
Honestly, the only way I can see to abuse such a system would be transferring karma across account, but why would anyone really need to do that?