PerlJedi's Journal: Meditations on Time, Spammers, Trolls and Fame

As I've been working this week to build a supplement for our existing hall of fame, I've been making decisions (or guesses) about many things, some the I expected, and some that I didn't... We will be releasing this thing into the wild soon, and some of the decisions I've made may be reversed once we see how readers actually interact with and re-act to it. All of you that are smart/lucky/bored/crazy/unfortunate (you pick the adjective) enough to be reading and following my thoughts are invited to leave me you're thoughts here.

Time
Our current "Hall of Fame" has two parts... "All Time" and "Past Year". "All Time" is self explanatory. "This Year" could be interpreted in two ways: during the current calendar year or during the last roughly 365 days (current hall of fame uses the latter). For the new bits of the Hall of Fame, I will be breaking it down in "Weekly", "Monthly" and "Quarterly". Each of these increments of time are open for the same interpretations as "This Year", and by nature of being smaller in scale, the differences can have a much more immediate effect, and both interpretations have both strength, and weakness as I see it.

My first instinct is/was to use absolute measurements for these (weekly is Sunday-Saturday, Monthly is from the 1st on, Quarterly is January - March etc.). The strength I see in this would be that we have a clearly defined range of time, with a beginning and an end. I think that could encourage new comers to the site to engage more immediately (at least at the weekly level) as at the beginning of the time span everyone, old and new, quiet and reserved or loud and obnoxious, would start at 0. The other possible strength here would be that with a define beginning and end we can mark the end of a time period and record the winners. The weakness in this approach (which I also argue is a strength) is that at the beginning of a time span the "Hall of Fame" would be full of low scores and therefore not particularly useful until we are a significant portion of the way into it. The (perhaps less obvious) problem created here would be that the time when this has the most complete (and arguably) the most useful data is near the end of the period and by the time the data set is complete, we have moved into the next period and started all over again.

Currently I am leaning toward the "rolling window" approach, so "Weekly" will always have the last 7 days of data. This implementation is easier to code because I never need to worry about how far into a week, and month or a quarter I am. It will also mean that regardless of when a reader looks at it, it has useful data, even if it is on the first day of the month or week. The first drawback would be that with no beginning and no end, on race ever ends, and no one ever gets to claim victory. The second would be that new comers will always see the bottom of the "Hall of Fame" as where they would need to be at the end of a week or month, which could seem unreachable.

Quality
The other issue I find myself grappling with on this topic is one of quality. We all know that there is more to being a part of something than just showing up. Slashdot is a community created/supported site, and we depending on community contribution not only for submitting stories, but also for participating in discussions, and even helping to moderate the discussions. Sadly, we also all know that not all contributions are positive. Everyone can, and lots of people do, submit stories, and make comments. The bulk of these are good, but here, as everywhere, some are just not. When we recognize members of our community we want to recognize the quality, not just the quantity, of their contributions. So for example we want to recognize those who submit stories to us on a regular basis, but we do not want to recognize the "entities" that are flooding our inbox's with spam. Similarly we want to recognize those who participate in discussions, but probably not the clowns who spends time trying to hit the comment button the first just so they are can say "Hey! Look at me, I'm first!". This question can be answered in many ways, none of which will be perfect, and most of which will piss somebody off.

Another dilemma comes into play in this decision... we want people to contribute, and we want to recognize those who do so. It seems only fair to have a well defined set of criteria on which to select who we should recognize. The problem is deciding not only what criteria are fair, but ideally we like to prevent people from "gaming the system". I'd love to live in a world in which no one would do that, but the sad reality is that if we simply say "The person with the most comments in a week gets a gold star", some joker will decide to make all of there comments one word at time, just so they can have their shinny gold star.

The first incarnation of this new "Hall of Fame" will be our best attempt at answering these questions, but like all things, it will be a work in progress, and with help from the community, if it is not the "best" on day one we'll get it there in time.