SETI's Anti-Cheating Strategy

mtDNA writes: "There's an article in the New York Times about the strategies SETI is using to avoid fraudulent reports. One trick they're using is multiple analyses of the same data. Another strategy is the use of "ringer" data, where they send you fake data for which they know the results." One of the researchers has several postscript papers on his home page - Incentives for Sharing in Peer-to-Peer Networks, Uncheatable Distributed Computations, Distributed Computing with Payout. In related news, ProcessTree apparently sent out an email to participants indicating it is closing up shop, so although SETI seems to be chugging along, the idea of distributed computing as a business model is perhaps a bit premature.

To prevent cheating...

...You simply remove any incentive to cheat.

William Gibson's "Black Ice" should do nicely. Failing that, slice or dice the data in multiple directions and compare results.

(The "different slices" is important, to ensure that you aren't trying to validate one modified client against another.)

Let's say that you have a grid of data, N x M x B (where N, M is the data, and B is the number of bits per word for that data.)

The probability that one modified client is doing the rounds, and will be encountered again by chance, is non-zero. It's not high, but it's high enough that nobody is releasing their client code in a hurry.

On the other hand, you've three simple slices you can do (along each axis), and any number of more complicated ones. That means that you have to hit the correctly-modified client for the slice you've picked, for each slice in each axis, for the data to be marked "valid". Any failure by any one client to return a result that confirms the other 16 clients that would overlap with it, would signal a bogus client.

With that much redundancy, you could also simply have "client voting". The results that are returned identically by the most clients (in excess of some threshold), regardless of the direction of slice, could be regarded as "true", with a reasonable degree of certainty. (Sure, it's not 100%, but that's the price you pay for having a society that rewards the greedy and the ethically sick.)

Of course, if you want to go one stage further, there's nothing to stop you "dicing" the data. Instead of taking a single slice through the data, you take random, small chunks from all sections, and feed them in a random order to the client. Again, the server re-constitutes the "valid" results, by merging together the results from multiple clients, taking the generally-accepted results as "correct".

This would mean that, instead of needing 20+ clients, all with suitable code for cheating "correctly" along each slice, you now need !(N x M x B)/(Size of chunks) such clients. The values don't have to be large to make this a virtual impossibility.

If you then only credit "confirmed" units (whether "slices" or "chunks"), since cheating becomes impractical, short of a global Internet conspiracy which also included the researchers, nobody is going to bother modifying the clients in any way which produced inaccurate results.

They =MIGHT= modify them to produce faster, accurate results. But, in that case, who bloody cares? I'm not going to object to someone handing round an honest, genuine client that can plow through 10 times as many blocks in a second, and still deliver the true results back to the central system. And, if the scientists were being honest to themselves, I doubt they would, either. PROVIDED the results could be guaranteed.

And that gets back to why independent result reviews, using slicing, dicing, or some other method of producing non-duplicate data sets, is very important.

Processtree closing down. Where is your user info?

Distributed Science Newsletter
May 2001

Dear ProcessTree Network suppliers,
It is with sadness that I have to announce that this will be the last newsletter you receive from Distributed Science, Inc.

etc etc etc...

We will diligently negotiate the sale of the supplier database, with emphasis on the privacy policy under which you signed up. As soon as we came to a result, the new owners will be informing you about any changes they might plan, including an opt-out for those concerned about their privacy under new management.

EEP!

Re:Already a business model

Umm no you are actually quite wrong.

Render programs are free. (povray for example, many many Excellent CG films have came out of povray. Just check the Intertnational Raytracing Competition pages)

Yes some render programs cost exorberant and insane prices, but places like pixar have programmers that write the software, and most good animation houses have their own programmers, so your cost per copy goes from $30,000.00 from the development of the first one to $0.00 for every copy thereafter. (dont give me any crap that there is a cost associated with the copies afterwards, that is pure bullcocky)

Do you think that lucasfilms goes to "CG-R_US" and buys a new effect? nooo, they create it, and then they can use it on 94,999 computers for free.

CG is cheap, and distributed processing (possible in POVRAY for a really long time now) is also cheap.

hmmm, just like I've been saying all along

Their argument against open-sourcing the client has always been that this would allow cheaters and that people would use modified clients that didn't crunch the numbers right. To which I have always responded that with any distributed computational task running on untrusted clients, you would have to do this sort of redundant analysis on each data block anyway. Even a closed-source client can be hacked fairly easily if you really wanted to, so not releasing the source doesn't magically guarantee the validity of any client-side processing. It's nice to see SETI@Home finally acknowledge what some of us have known all along.

So, when will we be seeing the client source code available for download? I'm all ready to start working on an Xscreensaver [jwz.org] module for it.

Caution: contents may be quarrelsome and meticulous!

What about public ridicule?

I have an idea for how to at least reduce the amount of cheating going on with SETI: ridicule. Because let's face it if you cheat at SETI you deserve ridicule. You're a worthless mess of a human being who probably hasn't been laid in, I dunno, EVER and has to inflate their self-esteem by turning a quest for Contact into a bigger dick contest. No one respects you. Kill yourself and leave your computer running. Your computer is worth more to society than you are.

Grr. I'm way too high strung today. Where's the bong? But godDAMN people are so freaking simple minded sometimes! What do you gain by cheating at SETI? Higher rankings? So fucking what! Great, now instead of being ranked 39623 your at 32532. RaH. You're my hero. The world is a better place because you cheated. You've fed the hungry and increased our collective wisdom. L0s3r.

Dump core. And pass the bong.

- Rev.

distributed.net does the same

As far as I know this is nothing new, distirbuted.net has always done this on thier projects (RC5, DES) to make sure people are actually checking the blocks.

The reason people are cheating.

The reason people are cheating is because they decided to make a contest out of who procressed more workunits.

I for one wish they would get rid of the scorekeeping entirely. I crunch SETI units because I enjoy the idea of helping them with their science.

Any users they lose because they were to get rid of scorekeeping would be no great loss. They were probably the losers who were compromising the datapool anyway. (talk about having no self esteem, I can see it now, some geek going up to a girl to impress her with his falsified SETI numbers).

I was one of the first 10,000 people to sign up, and I'll help them with their science as loing as they need me to, scorekeeping or no.

Rich...

Re:Active punishment?

God, I loved that old feature on Telegard/Renegade and the like. Though most people figured it out when noone responded to their flames, and then made a fake account/logged on as a guest, to find out the truth. But this would work with seti, where there is no 'feedback'. Hell, they've even disabled 'see my last 10 packets' as of late, so as long as they kept on incrementing the person's records to their eyes, it wouldn't matter. As far as the problem that you present - a broken computer being innocent as compared to malicious data. That really isn't a problem. Not to sound like an arrogant fuckwad, but the end result is the same to seti. Data that's just wrong as a result of a computer going tits-up or data that's wrong from a computer being messed with - it really doesn't matter. They're going to need to reject both.

Re:Why bother?

If someone reports a hit, cant they just re-check that data?

They do. What the client programs do is something of a preliminary analysis, filtering the most interesting packets of data from the usual junk. In the further analysis it often turns out that lots of interesting signals originated on Earth, while many others are inconclusive.

--
I hit the karma cap, now do I gain enlightenment?

Re:Why bother?

If someone reports a hit, cant they just re-check that data?
You're missing the point with SETI. There is no such thing as "a hit" when analysing these massive amounts of data. Your computer will never give a message like: "Analysis detected a HOW ARE YOU GENTLEMTN, ALL YOUR BASE ARE BELONG TO US from outer space". What your computer does is just an analysis and then the SETI-folks will do the real exciting stuff with the resulting data from your computers work.

The problem before SETI@Home was that the data wasn't analysed completely to detail because these analysis take a shitload of time so they just did a rough analysis, trying to find extreme peaks but no checking for patterns over longer periods of time.

I should have known

Somebody sent me something strange from his SETI at home setup. I don't know for sure, but it looked a little like a hoax to me.

Here are some warning signs that you may have a SETI hoax on your hands:

• a midi file of the
• Close Encounters tones.
• A .gif of Leonard Nemoy as Spock with the caption "Live long and ... whatever."
• The astral baby from 2001 rendered in ASCII graphics.
• "Hello, people of earth" in a voice that sounds suspiciously like Homer Simpson.
• Anything resembling "Goatse.cx"

In other news: Bi Curious: The Senator Jim Jeffords Story [ridiculopathy.com]

AH!

But the problem is not ordinary punks hacking the client to create false positives. No, the problem are those Beowulf clusters in underground NSA facilities making all the false negatives!

--

Why?

My questions is Why anyone would want to cheat SETI? I could just see the guy now:

"LOOK! i'm high on the hours list with 31337 years of data done on my computer for SETI. I RULE! Oh god, I wish I were dead..."

Re:Active punishment?

Instead of locking out a cheater, a better solution is to continue to feed data to that cheater, but ignore any results they submit. This will help prevent the cheater from simply creating a new account, as they will be unaware that their false results have been detected.

Double Resources

I admit that I am not terribly familure with seti, but I know that they use huge amounts of collective cpu time via the distribution of processing to remote processors. My comment relates to who decides how much of a performance hit do they want to take to insure accuracy. Do you send sheets of data out twice, and reduce your net performance by half? I don't understand how sending rouge data sheets will "catch" the bad guys, wouldn't the one that get caught just change their IP/User Name and start sending bad data again? I'm afraid that if SETI really wants security, put a bunch IBM Z series boxes straight to the satellites, and let little instances of Linux churn over the data.
TEN

Shocking!

So somebody's trying to manipulate the system in order to artificially inflate a meaningless number in a database! How shocking! (Score=5, Insightful)