Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Mozilla

Journal numbski's Journal: We need a Slash-torrent Firefox Extension 21

Journal by numbski

I need to get this idea out of my head and down on paper (so-to-speak), as it has been driving me nuts all evening.

We need someone to develop a firefox extension that affects Slashdot's rendering a bit, integrates with RSS, and can execute actions in the background when the user is away. It goes something like this:

I wish to submit a story to slashdot. I include a few links, and most of them are on webservers that in no way could handle the load of a slashdotting. Before I submit the story, I run a spider on my story text (perhaps built into the extension?) that downloads the text of the page, and any IMG tags on the page. Should probably have an option to either ignore or allow any flash or other binary filetypes (.avi, .mov, .exe, etc). When indexing, it pulls down that content, and packs it all into a single binary file, perhaps a simple .tar.gz (.bz2?), and then from there creates a .torrent file for that binary package.

With the advent of trackerless torrents, that may be the last of the work, otherwise this will need to be made availabe on a tracker site. A multitude of these exist currently. Now we simply need to convince slashdot to allow a hidden tag included in the story submission, perhaps <torrent:http://trackersite/file.torrent>

For those using the extension regularly, the extension will ping slashdot's RSS feed once every 30 minutes. For each story containing the hidden torrent tag, it will automatically join the torrent for that story. The torrent will save locally, and unpack to a directory specified by the user, and associate itself with the sid from which the link was retrieved.

When the user goes to view the slashdot site, the rendering will be intentionally altered. Stories that have not yet completed the torrent download should have an altered background color (customizable by the user?) and perhaps some additional text notifying the user that the download is not yet complete. Ones that are completed could have a pretty icon added to let you know that the links are all available.

When the user clicks the links from the altered story, it will actually point to the files on the local hard drive, and using some simple regular expressions should take care of image loading and linking off to other sites from there.

The important thing is that the story submitter include ALL urls pertinent to the story. If you have a 5 page story, you must include all 5 url's in the spidering process.

This process will also download plain text banner ads. I'm not too concerned about this however. When rendering the page later, if someone has adblock enabled, they still won't appear, and the original content owner can't really complain that we're filtering ads from their site. Besides, we're taking a load off of their server. ;)

If done correctly, this would work for all slashcode sites. It requires minimal cooperation from slashdot themselves. Allowing a single new, hidden tag to be allowed in the story submission process. If they don't wish to cooperate, this could STILL be worked around. Basing things on the originating url (slashdot.org as opposed to macslash.org for example) and the sid of the story. Then we would need to have a centralized off-site tracker to organize things, or at least a few 'known good' locations to immediately go look for the torrent files.

You could also set up an automated spider to create our binary torrent file 'ex-post-facto' by pinging the RSS feed, if no torrent is available, define the scope (which urls to include), create the torrent and away we go.

I'm not saying the above is neccessary 'easy', but it is most certainly obtainable, and I have to think very worthwhile. If done correctly, everyone using the plugin becomes a seeder, and you don't have to worry about seeders becoming leeches, unless they close the browser out entirely often, and if they're doing that, they're robbing themselves of the benefit of the plugin to begin with.

Really, this is just extending the idea of podcasting, or the RSS+Bittorrent for broadcasting idea to slashdot to completely put to rest the slashdot effect, once and for all. Those using IE will be SOL. :P Actually, we may still put it to bed because now the number of people hitting the original site will be significantly reduced. The only real impact of course will be that the server logs of the one running the original site might now think that all of their users use IE, and that Firefox just dropped off of the map entirely or something. :\

Whatever the case, I think this is the right answer to finally solve the problem. I didn't know where to put this, so I'm putting it here, and I'll probably copy the URL to this posting into every slashdot post I make for a while. Perhaps I can draw enough interest and enough coders familiar with Firefox extensions that this can come to life. The only language I know is perl, and at least last time I checked, perl wasn't the right tool for the job for extension coding. :(

This discussion has been archived. No new comments can be posted.

We need a Slash-torrent Firefox Extension More

We need a Slash-torrent Firefox Extension

Comments Filter:
  • Interesting Idea...sounds like a lot of work though. Especially since there are all sorts of mirrors and caches already available.

    Manfred

  • copyright issues? (Score:3, Insightful)

    by markjugg ( 21992 ) * <<mark> <at> <summersault.com>> on Tuesday May 31, 2005 @01:06PM (#12685307) Homepage
    How would address the copyright issues of the target pages?
    • I would think that the robot tag would be a better idea for if a page allows the content to be followed or not. If the no robot tag is present, then the spider would ignore the page for downloading. Another thing concerning banners, if the banners are hosted somewhere else, keep them hosted somewhere else. It is entirely possible to make a HTTP request and fake the referer text so that it appears that it's coming from the original source and thus that site gets the recognition it deserves (not that I've
      • Good job hitting the nail on the head. :)

        Everyone keeps yelling about banner ad hits being lost, but I don't see that as being the case.

        I just said above, set an option to follow IMG tags, and other options for flash, java, etc.

        If it's local to the site, then the links outbound will need to spoof. If it is hosted off-site, then there's no problem, because the regex's will handle this correctly. I don't see anyone crying foul about this, otherwise Google would have issues too for caching info. We're no
    • There are none. As long as you are only taking a copy for yourself to read, in the normal way that you read a webpage, then you are covered by the "Fair Use" exemption. I doubt a judge would see a complaint as valid, either. "I put this thing on the internet, and this guy looked at it! Then he sent it to a friend!" No-one w/could realistically complain, nor could they do anything about it if they did. The counter-argument is that you didn't want to carry out a DDoS attack on their website, nor use
  • You don't need to tar/gzip the site torrent, you can just create a torrent of all the files. It's much easier to use that way, and besides, the only thing that compresses really well is the HTML page, the images and other media are already compressed (and tar just concatenates files, gzip compresses the tar file).
  • Trauma team specialists refer to something called the "golden hour", any trauma victim that receives treatment within that time frame has a vastly higher chance of surviving. As I understand the bittorrent protocol, any uploaded content is hosted only by the seeder *until* a complete copy is hosted on at least one other client machine. After that point, torrents follow a common growth law. Number of hosts, in your example mirrors, grows exponentially until it hits a cap where the majority of interested part
    • What about this then?

      Instead of <torrent:http://trackersite/file.torrent>, instead we do something like this:

      <!-- torrent data -- uuencoded torrent file follows

      (several lines of uuencoded .torrent binary data

      -->

      Then have the extension pull it directly out of the story? Of course that means we need explicit cooperation from /., but it's a possibility...
  • And I think it would reduce the /. effect. But for me personally it would not work. If I access ./ at work, our firewall blocks all torrent clients. I wonder how many other ./ers are in a similar position.
    • Yeah, that problem still remains. The only workaround for that I could see is perhaps placing your slash-torrent cache in a 'world readable' directory (I don't see any security issues with this....do you?), then set up webdav on that directory and password protect it.

      At work, mount the webdav volume using whatever method fits for your OS, then set your cache to be that share, and then have an option in the extension that more or less says "use this cache location, but don't actually download anything".

      Th
  • Maybe it could be enough to find the cached page in google or force it to be cached? Google should be able to handle a load of /. users if it would cooperate. I, just like many other people don`t use torrents just because of my ISP restrictions.
  • Wouldn't it be easier to make a Firefox extension that inserts .nyud.net:8090 [coralcdn.org] in all URLs in the article? :)

Slashdot Top Deals

"More software projects have gone awry for lack of calendar time than for all other causes combined." -- Fred Brooks, Jr., _The Mythical Man Month_

Close