Overpeer Spewing Bogus Files on P2P Networks 479
nimec writes "Zeropaid.com has posted news of a company called Overpeer which is the source of all the bogus mp3 files that are popping up on the various P2P networks. Zeropaid, in the news article, said: 'If you've encountered the "loop" files, in which a section of the chorus or hook is repeated over and over, you've been tricked by OVERPEER. OVERPEER are doing this with the full knowlege and consent of Interscope and Universal Music, in fact they are under contract to Universal and other major record labels, and will be doing a LOT MORE of this type of "interdiction" in the near future.' Right now this doesn't bother me because these bogus files are few, very spread out and it is easy spot them. I'm just afraid that over time people will keep downloading these bogus mp3s and become too lazy to delete them, like they are when it comes to incomplete songs."
Re:Idea to counter this (Score:5, Informative)
Ah, the BBS days... (Score:2, Informative)
Don
There is already a system that can prevent this (Score:2, Informative)
Its a great system, Share Reactor cant get sued, edonkey2000 doesnt have centralized servers, and I get much greater speeds than in any other P2P program. Sure would be great to see other people take advantage of the great possibilities that edonkey2000 (and other P2P programs) can offer like Share Reactor does.
Needless to say, I highly recommend it.
Re:Predators are good for an ecosystem (Score:4, Informative)
And this is an interesting software engineering problem. It is the first internet protocol that has to be designed from the ground up for anonymity and resilience. And that will grow in a hostile enviroment.
The TCP/IP stack was designed for resiliency and they did a good job, but this has to be even better, and we don't have the goverment on our side!
There are a couple of attempts at this. One is www.freenetproject.org (that seems to be stalled) and the other one is gnunet [freshmeat.net].
GNUnet is a decentralized network with confidential and authenticated communication. A first service implemented on top of the networking layer allows anonymous distribution and retrieval of content. GNUnet supports accounting to provide contributing nodes with better service.
Media companies and technical counter-measures (Score:5, Informative)
So as I said, I do see this as one of the problems to be solved, although I feel it's of lesser importance. There are many ways of doing this. One of them is previewing - when downloading an audio or video file, when you're about 100k into it (100-200k if it's video), do a preview and see what you're getting. With this looping stuff you have to go farther than 100k however - preview one fourth to one third of the way into the audio files. Many Gnutella clients have a preview feature, as does Fasttrack (Kazaa).
Another method is to ban IP's and IP ranges spreading this. This is already being done - it's only a minor fix because they will always get around it, but it will help somewhat, they won't be able to have big servers spewing this stuff 24/7
The real way to fix this however is hashes. Which are already ubiquitous - they already exist and are known on Gnutella (Shareaza, Gnucleus, Morpheus, Bearshare, Limewire), Fasttrack (Kazaa) and Edonkey2000. On Gnutella (Shareaza) and Edonkey2000, you can click through or cut and paste these URI's (URLs) to files from web sites (or Usenet, IRC, e-mail, instant messengers, whatever) and start searching and downloading the files - for FastTrack (Kazaa), it is a little bit more time-consuming and complex, but worth it if you're going to be downloading a large file. The hash technology is already there, the key now is finding a trusted source for hashes which are both good and whose data is findable and downloadable on p2p networks, and for those sources to survive. I guess I'll detail how this is currently working with the various p2p networks, why not?
There are four major p2p networks - Gnutella, Fasttrack, Edonkey and Freenet. Freenet is a publishing network, the others are all file sharing networks, which is what we're concerned with. Gnutella and Fasttrack are the two largest networks. Edonkey2000 specializes somewhat in large files however, so if it's 100MB+ files you're after, Edonkey2000 is on par, and perhaps better in some ways currently, than Gnutella and FastTrack. Edonkey2000 and FastTrack are closed networks - closed source server/clients and closed protocol networks. Gnutella is open, the protocol is open, and robust open source server/clients like Gnutizen exist for it. This gives Gnutella advantages, such as a choice of multiple clients for virtually every platform, as well as other advantages. Of all the file sharing p2p networks, Gnutella is my favorite and I believe Gnutella is the future of p2p. I think competition amongst p2p networks is healthy however as every can steal everyone elses best features and innovations.
Gnutella files are hashed for HUGE with an implementation called sha1. You can read about the technical aspects here [yahoo.com] if you wish to. These hashes are useful for finding additional sources for found files so that one can resume downloads or download from multiple sources with integrity. Actually there's one caveat to that - if you are downloading from an honest client, it will tell you a truthful hash of it's data. A client could give a fake hash and then send other data - but you would have to directly download from the rogue. How clients deal with this is even more complex - Gnucleus downloads overlapping chunks - it downloads 1-2000 from one source and 1950-3950 from another - if 1950-2000 do not match from both sources, it marks both chunks as possibly bad. You can read more details about this in Gnutella documentation and discussion groups.
Aside from this usage, these hashes can be used externally as well. Currently, Shareaza [shareaza.com], which is a pretty good servent (server/client), is the only one from which URI's (URL's) can be cut, paste, and clicked through to from the web/IRC/e-mail etc. I'm sure clients like Gnucleus will have this ability in the future. If you had Shareaza installed, you could click on a link like this - which is an, I believe uncopyrighted, Chomsky speech [gnutella], Shareaza would launch (if you don't have it already) and would ask you if you want to download the file or cancel. If you select download it would connect to GnutellaNet, search for the file, and if it found a host which has the file and which has upload slots open, would start downloading it. Actually, the Slashdot "allowed HTML" filters are pulling some necessary characters out of the above link, so you can't click through on /., although you can on a normal HTML web page. I can't post an URL that you can cut and paste either since /. forces a line break after 40 characters or so, if /. didn't do this and the below was in one line, you could have cut and paste it into Shareaza, I'll show it here for an example, imagine this was all on one line for you to cut and paste, or better was just a link to cut. You can do this on any HTML page, it's just the Slashdot HTML parsing messing it up -
gnutella://sha1:HXHSJ6ATN3LQCCIOBGUEWV5FFCKP2KBL/N oam%20Chomsky%20-%20Audio%20Book%20-%20Noam%20Chom sky%20-%20At%20Johns%20Hopkins%20University.mp3/
I would give the above link a rank of "7", because the last time I searched for it, 7 people replied they had it. I have several hashes with a score of 80-90, meaning you're more likely to find or download them, but the above is the only one I have that I have enough confidence in that the data is uncopyrighted.
So now you have one link to a hash - where can you find trusted sources which tell you what hashes are ubiquitous, making it more likely you will find and be able to download them, are rated in terms of quality by multiple sources and so forth? Well for Gnutella, one source is Bitzi [bitzi.com]. You can search for data there, see what is the most reported, what things are ranked, see comments, see bit rates, file sizes, artists, titles and so forth. It is very cool. Most interaction is from Bitzi into Shareaza (the only Gnutella client that does this currently), but from within Shareaza if you find a file you can type "find Bitzi ticket" and see if the hash has been reported on already. One thing which I'm sure will soon be remedied is that Bitzi does not have direct clickthrough to Shareaza, I have to copy hashes to my clipboard, edit them to Shareaza format and paste them into Shareaza. I'm sure soon Shareaza and Bitzi will agree on a standard and remove this step so I can just click through. And soon Gnutella clients other than Shareaza will have this ability as well. Bitzi's data base is open to the public, you can read their open data policy on their web site, anyone is free to use the data as long as Bitzi is credited. Bitzi.com is the only large, good source of Gnutella hashes I know of. Edonkey2000 has had hashes for a while, and has several good, large sources for hashes such as Filenexus.com and Sharereactor.com. Since Gnutella is a larger network and it just implemented this ability, I'm sure it will have even more and larger sources in addition to Bitzi. And since Bitzi's database is open to all, if Bitzi goes down someone else can open the database up again somewhere else. I'm sure in the future, even the trusted rating system will become distributed.
Gnutella uses the sha1 hash, Edonkey2000 uses another, and Kazaa uses another. Web sites exist that centralize the hashes for these. I'm sure soon web sites will exist that coalesces and translates all of this. Gordon Mohr, who runs Bitzi, wants to see a universal p2p tag, magnet, which is agnostic about which p2p backend it is using. Why not? We can have a tag that we (more or less) trust, and can retrieve the data from Gnutella, FastTrack, Edonkey2000 or Freenet. It's a great idea.
I am less interested in other p2p networks than Gnutella but I'll discuss their hash and meta-data web sites a little. The most interesting one is Edonkey2000 [edonkey2000.com], which as I said, has come to specialize in large (100MB+) files, and which I have to admit is a pretty good way to download large files with some guarantee of integrity. There are two major meta data sites for Edonkey - Filenexus [filenexus.com] and Sharereactor [sharereactor.com]. There are other sites as well. If you're looking for large files, they do a pretty good job currently.
Fasttrack (Kazaa) uses hashing, but the Kazaa client is not that friendly to this kind of thing. So Fasttrack/Kazaa is more of a pain in this respect than any of the others. Nonetheless, you can download a program called Sig2dat [geocities.com] that helps you copy and paste FastTrack's UUhashes. The you can go to web sites [fasttrackmovies.com] that give meta data, rankings and so forth to these hashes. Kazaa/FastTrack is unfriendly to all of this so it is much more of a pain - you have to install files that help you do this (sig2dat), you have to restart Kazaa for every file you want to download in this fashion and so forth. With Kazaa, all of this is a hassle, it's much easier to do in Gnutella (Shareaza), Edonkey2000 and Freenet.
And lastly there is Freenet [sourceforge.net]. Freenet has been using hashes since the beginning. Freenet is a publishing network, not a file sharing network. That is nomenclature - file can be and are shared on Freenet - from html pages to gifs and jpgs, to mp3's, to avi's, although Freenet is the last place you want to look for large files, Freenet's bailiwick is small files. Even a 4 meg mp3 on Freenet is harder to find and slower to download than any of the other 3 networks. Small files are the domain of Freenet - HTML pages and images. The Freenet protocol is more rich than the other protocols in many ways, thus you have more than just audio and video files going over it, you have third-party applications utilizing it, thus you have things like Fproxy (A world-wide web equivalent which runs over Freenet) and Frost and Freenet message board (Usenet equivalents - both for text and binaries). One benefit of Freenet is it's hard to crack down on people for publishing information - because no one knows who data is coming from or going to. This is not absolute, but it is much safer than the file sharing p2p networks in this respect. Also, people publish data, so that what you put out is stored somewhere other than your computer, and if your web site or shared file or whatnot is popular, it will be out there all the time without your node needing to be connected. Freenet also used a lot of signatures, encryption and so forth, so you already have a pretty solid trust mechanism and data integrity. It depends on what hash is used - KSK hashes are insecure, but SSK are signed. So with Freenet there are large upsides and downsides - the downsides are downloading is much slower, since you're downloading via intermediaries, not directly, and the larger the file, the slower the download and the harder it is to find a complete file. The upshot of Freenet is that there is less of a legal risk with regards to sharing/publishing data, data is signed by the publisher which greatly helps integrity, and also Freenet's protocol allows extensions other than file sharing with it's own internal network - web and Usenet like applications, and I'm sure there will be more in the future.
Comment removed (Score:4, Informative)
Re:So, they are wasting my bandwith! (Score:2, Informative)
Re:So? (Score:2, Informative)
Re:Media companies and technical counter-measures (Score:3, Informative)
Code is law...for us too (Score:3, Informative)
But code is also law for us.
We are the one's who write the code for P2P services like Phex, LimeWire, BearShear, etc. Thus, we are the one's who create the "law" for those services.
We have the ability to code away this problem, and any other problems presented to our P2P utopia.
So how do you deal with bogus files? Well, one way to do it is by detection. Write protocols into P2P programs to detect bogus music files. How do you do that? By reverse engineering their technology. Lets say that their "bogus" files appear the same size as normal files, but about 1/4 of the way through have a hitch in them w/c causes your player to play over the part over and over again. So you write code to detect that.
Another way to deal with it is the same way we deal with spammers: block unreliable sources. If a domain-name for e-mails often gives you spam, you block that domain name. Same thing w/ P2P networks with a little bit of ingenuity.
The only thing to worry about is the red queen effect; namely, we take counter-measures to their measures, and they take counter-counter measures to our counter-measures, and so on and so forth. This results in a lot of wasted time for us, and also will eventually make our code bloated.
Another alternative is the legal route. Contrary to what some say, there is a legal option. Their actions garble up the P2P network, which will negatively affect many who are sharing non-copyrighted files. Hence, a basis for a legal restraint.
The other possibility is a counter-attack. They've screwing up our networks, so we screw up theirs and their systems. The best defense is a good offense. This would be DoS attacks on their servers, or virus'/worms aimed specifically at their computers.
Another possibility is very simple. Rather than trying to weed out untrustworthy sources, try to find trustworthy ones. This is much easier as you'll get cooperation. Real netizens of the P2P community may put tags on their files, as identification, which would securely identify them; then, those files would be rated on two categories -- quality and completeness.