RSS And BitTorrent, Together At Last 267
eoyount writes "Wired has an interesting story about a really simple idea I wish I had thought of. Transferring large files across the Internet isn't easy for your average joe, but a combination of RSS and BitTorrent technology might just make it easier - Slashdot ran a previous story on the theoretical blending last year." (LegalTorrents is run by the strangely familiar simoniker, who wrote a short piece on the O'Reilly Network about how it was set up, and offers observations on how well the combination fares.) Update: 03/17 21:45 GMT by T : Ernest Miller submits two related postings he's written on RSS+BitTorrent, a combination he calls "broadcatching."
RSS + BitTorrent = Broadcatching (Score:5, Informative)
Hack your TiVo for fansubs (Score:4, Informative)
Hell, you could modify an actual TiVo with broadband for exactly this sort of thing, and it needn't be limited solely to anime either. I'm sure it'll be popular with overseas watchers of American TV as well.
The international media and internet companies need to face facts and realize that Video On Demand is a reality and is already extremely popular - but that the shows people are demanding are not the ones the companies have been providing through their own limited, misfocused, and (most importantly) redundant services. Until we see simultaneous worldwide release of all media (including DVDs released simultaneously with the theatrical release) they will find themselves losing what should have been their easiest sales - those to impatiently eager fans.
Re:Hack your TiVo for fansubs (Score:4, Insightful)
Can someone is can make subtitles near to or exceeding professional quality ones for free? Translating, editing, timing, and typesetting? They can and they do. That is why fansubs exist. Do they bother with sound? No, because dubs are very difficult to get even close to comparable with the original language, if at all, but subs are easy and require only a little quality control. In fact, the original producers would be wise to cultivate and sponsor these international volunteers to do the the translations and editing for them (because timing and typesetting are the most technical parts but can be applied to all the translations). Plus, unlike sound subtitles require a negligible amount of bytes compared to the video.
Can someone distribute content for essentially no cost to the producer? Perhaps you ought to read the article again. Obviously the answer is yes. This is why digital fansubs are far more widespread and popular than the old VHS variety ever was. This is why the parent of this thread was referring to articles that describe how indy bands and movies can make themselves known and spread their work.
Are international menus hard/expensive to make? No, unless you make it difficult for yourself in the first place. Frankly, I'm really only interested in watching the show, and a lot of the overly flashy and slow menus out there only make things annoying. Do you even need menus for online distribution? No, since generally it's just a single movie/song/album/file. How about packaging? No.
What about when electronic distribution is not available, i.e. poor countries? Well, those guys on the blankets on the sidewalks seem to be able to manage. Certainly I've seen a few bootlegs from Hong Kong in my time with laugably bad english but probably decent chinese, and they manage. In both cases they seem to be catering to people who are priced completely out of the legitimate market rather than simply unable to access it due to a lack of translation. Which is where black markets have always taken over.
Re:Hack your TiVo for fansubs (Score:2)
#AnimeJunkies, for those of you not familiar with them, is a fansubbing group that specializes in being very quick to release, very poor in quality, and very unscrupulous. They clearly don't do much in the way of editing their translations.
If file transfer is hard for the average Joe... (Score:4, Funny)
Not so bad (Score:3, Interesting)
The main problem people have using bittorrent is regressive internet connections. (Until IPv6 becomes ubiquitous this is going to be a problem for many of the internet's designed uses, not just swarming media.)
Im not so hot about RSS, but for things such a multicast or bittorrent- it really helps to get the content when everyone else is. So having a running subscription to a show you like, then have the download automatically kick-in as soon as it becomes availab
OK, newbie question (Score:3, Interesting)
Re:OK, newbie question (Score:5, Informative)
You can find more information here [harvard.edu]
--
Real-time deal updates, over 400 a day! [dealsites.net]
Re:OK, newbie question (Score:2, Informative)
Try Google, newbie :-)
Oh heck, I'll save you the trouble. Read here [xml.com]Re:OK, newbie question (Score:5, Informative)
It's also seen as a effective way to replace e-mail mailing lists. Instead of getting your newsletters in your e-mail client, open them up in your RSS client which works on a pull basis rather than a push basis, but can still present the content to the user just like an e-mail program might.
It's very different than Active Desktop... that was just the idea of letting IE browser windows be part of the Windows Desktop level so that users could have a frequently-refreshed mini-page of content on their desktop.
Re:OK, newbie question (Score:3, Informative)
Go read Slashdot's RSS feed [slashdot.org] if you still don't get it. Basically it's just an XML document that defines story "ITEM"s as having a title, link, description and other fields.
Neato (Score:2, Interesting)
A new browser protocol? Aim your browser at
bthttp://www.victim.com
and let it rip?
Re:Neato (Score:5, Informative)
No new browser protocol is needed.
The technology is already available at http://freecache.org/ [from the peeps @ archive.org]
I don't why many others have jumped on the bandwagon yet.
Re:Neato (Score:5, Informative)
Please note that you cannot submit a whole site to FreeCache as in http://freecache.org/http://www.rocklobsters.com/ This will not work as only index.html will be cached. You have to prefix every item that you want to have cached seperately.
Using the last THG article as an example, either the Slashdot story would need to point to each page individually via freecache redirection or Tom's Hardware would need to do it.
Not quite as transparent as incorporating BitTorrent into the browser.
Re:Neato (Score:2, Informative)
I have a number of video clips that are being served via freecache from my site at:
http://holden.customer.netspace.net.au/rocke t cam.h tml
The site was slashdotted on Saturday. I would like apologise for the additional burden this placed on freecache and any reduction in service levels that occured as a result.
I would be interested in learning more about how the system coped with this event.
Regards,
Mike
Re:Neato (Score:2)
Although it's cool... (Score:5, Interesting)
"Hrm, WoW is bing distributed by Bittorrent. Meanwhile, I get angry phonecalls from Vivendi to shut down Bittorrent."
Yay for technical advances, but can commercial interests fully embrace it without killing the "evils" of it?
The problem with bittorrent (Score:5, Informative)
Arguably, yes... (Score:5, Insightful)
Kjella
Re:The problem with bittorrent (Score:5, Insightful)
When you don't think of it in terms of people uploading movie files, and think in terms of companies using the technology to ease load on their web servers, now you're looking at BT the way the author intended!
Not everyone can contribute (Score:5, Interesting)
The real problem with bittorrent is that by enabling efficient transfer of large files, people are transferring larger files. And the service providers simply do not have the capacity for everyone to be sending those large files. They may advertise unlimited access but kids they really aren't set up for it. To say nothing of the fact that the way the internet is structured now is no longer geared towards everyone being as able to send as well as they are to recieve.
Really, the internet and its billing structure should be geared towards billing by amount received, and not amount served, and widespread implementation of load-sharing protocols like bittorrent. It would be far more efficient and fair, and would encourage people to limit their consumption rather than penalizing inadvertently popular unsupported sites.
Re:Not everyone can contribute (Score:3, Insightful)
So long as the provider is implementing reasonable QoS so that occasional downloaders get instant bandwidth, I don't see the problem with this at all. People have always found use for more computing power than was readily available.
If Intel were Comcast, 15 years ago Intel wo
Re:Not everyone can contribute (Score:2)
Re:Not everyone can contribute (Score:2)
Re:The problem with bittorrent (Score:2)
Besides, my Internet connection has a much faster download speed (3Mbit) than upload speed (384Kbit). I suspect that this is true of most other BitTorrent users as well (Downloading big files is probably more popular among broadband users, and all but a few very pricy broadband services give you fa
Re:The problem with bittorrent (Score:2)
BT's reliance on client-uploads limits performance on some connections. I regularly have a little transparent graphical bandwidth monitor sitting in the corner on my desktop, and I've noticed how uploading can significantly cripple download speed (although, strangely, not always). It is also highly cpu and memory intensive.
BT has it's setbacks, but it has it's uses as well. F
Re:The problem with bittorrent (Score:2)
-nod- I've found that if I run bt totally unbounded , I usually get around 120K/s down and 22K/s up. If I cap uploads to 15, I get around 320K/s down and, of course, steady 15K/s up. I've got a little script set up to run with capped upload until the download is done, then take the cap off. As a previous poster said, I usually fire this up when on my way to bed.
Doesn't it work anyway? (Score:2)
Re:Doesn't it work anyway? (Score:2, Interesting)
Yep, that's the point of BitTorrent - and what made it so special when it first came out
So the sharing still works to some extent even if people disconnect right away.
Furthermore, it's only possible to leech significantly from a peer that has already finished downloading. Any other client will notice that it's giving but not getting, and scale down it's giving appropriately. I bet the leechs love it
Re:The problem with bittorrent (Score:5, Informative)
If the download is not popular, than the orginator of the content can handle the bandwidth. Bittorrent's benefits kick in when something is popular, where there are simultaneous downloads at any given moment. If demand trickles back to one request every hour, than obviously the originator can handle it. Once it is no longer relevant, the orginator of the content can disable the tracker.
Bittorrent is a p2p network that works BETTER the more people are using it. Once everyone disconnects, then you revert to the worst case scenario, which is just straight downloading.
So don't worry, disconnecting after you finish is okay. You did your civic duty by sharing the bits while your download was in process. Enjoy your game guilt free.
I believe this is how bittorrent works. If I'm wrong, I'm sure I'll be corrected within - 3 - 2 -1 NOW
Re:The problem with bittorrent (Score:4, Informative)
Maybe...although it is much nicer if you let your u/l to d/l ratio is at least 50% - 100% is even nicer. You can can quit right after you finish, but it would be like only sharing 10% or so of your files on P2P networks that you downloaded. A step up from a true leech but not sharing completely.
Re:The problem with bittorrent (Score:2)
I agree, on an average torrent, its nice to make or break a 1.0 sh
Why not do a non-linear download? (Score:2)
I imagine this would make it l
Re:Why not do a non-linear download? (Score:2)
Re:Why not do a non-linear download? (Score:5, Informative)
No, seriously, try playing a partially complete BT download of an AVI with a player that doesn't look for the index (mplayer, DivX, etc.). The file is missing random chunks, not the end.
Re:The problem with bittorrent (Score:3, Insightful)
Re:Yeah, I'm a "leech" - so what (Score:3, Insightful)
Uphill battle? (Score:5, Insightful)
Many ISPs and college campuses block P2P ports, BitTorrent included. I'm not sure that 'news' is a compelling enough reason to have many (or any) of them change their policies.
Re:Uphill battle? (Score:2)
SuprNova.org ? (Score:3)
-molo
Re:SuprNova.org ? (Score:4, Informative)
Re:SuprNova.org ? (Score:2, Interesting)
Re:SuprNova.org ? (Score:2)
Re:SuprNova.org ? (Score:4, Informative)
Too many good uses for Bittorrent to let the warez kiddies spoil it for us.
Have you lost your mind? (Score:3, Insightful)
If I take a knife and I cut someone up with it. Does that mean that somehow you are now going to be unable to go use the knife for something you want/need to?
Honestly, if the only thing on this planet that anyone used bittorrent for was "warez", *GASP* you could still set up torrents for legal files and have all your buddies download them.
If BT becomes illegal at some point, then we all are going to have a lot more to worry
Bah. (Score:5, Interesting)
Which is effectively getting us to pay for website access/services, but instead of giving the money to the content creators we'll be giving it to ISPs instead and paying in bandwidth besides. So this is a bad idea.
Re:Bah. (Score:2)
Re:Bah. (Score:5, Insightful)
As for ISPs metering bandwidth, guess what, you have to pay for what you use anyway, otherwise the ISP doesn't stay in business. It doesn't matter whether its metered or a fixed $30 or $60 / month. It has to cover their costs. If you're complaining that your cost would go up with metering, its because you think that you use a lot more bandwidth then everyone else. So you're just trying to shift the costs to the people that don't use as much. Pot, meet kettle.
Re:You misunderstand. (Score:2)
#1 I want to download a trailer. Unfortunately, its busy and only 1% of the hits gets through.
#2 I want to download a trailer. Its busy but it uses a BT algorithm. Everybody gets through. The average upload is only 20KBPS because they are all on cable modems? Fine, then the average download runs at 20K. But you still get through.
Why is shifting costs of a service onto the users of the service unfair in any way? If its a business distributing a product to customers, th
Re:Bah. (Score:2)
Re:Bah. (Score:3, Insightful)
Re:Bah. (Score:2)
I have no problem giving back some of my otherwise-unused upstream bandwidth to get faster downloads. I do have a problem with companies feeding me ads when I'm doing it. As far as I'm co
Tinfoil Hat (Score:2)
Sounds like an OK idea, but is it just me or does anyone else think that there is just a bit too much hype in the *media* about this. They don't usually pick up on good ideas and try to make them critical mass and the "next big thing".
Syndication is a great idea, I like RSS, (does BitTorrent even work under Linux?)--but why on earth all the *orchestrated* hype?
Enough to make me try Freenet again. Harrumph
Re:Tinfoil Hat (Score:3, Informative)
Would you like the GUI client or the command line one?
Yes, it works very nicely under Linux.
Re:Tinfoil Hat (Score:4, Informative)
Uh, yes...
Here [debian.org], here [rpmfind.net], and here [sourceforge.net].Re:Tinfoil Hat (Score:2)
Re:Tinfoil Hat (Score:2)
Considering it was written in python, anything with a complete python port can run it. Considering it was probably written and developed in a *nix environment, OF COURSE.
BitTorrent is a very sound idea, and a very nice technology. The only reason it's hyped at all is because it's new, fast, and revolutionary. It makes it 10x easier to get big pieces of data quickly, rather than waiting for hours on an FTP server choked by twenty thousand other people wanting the
Covered this previously just before christmas (Score:2, Interesting)
However, I'm a little concerned - BitTorrent has a lot of initial overhead (setting up trackers, and all the protocol stuff). I'm not sure if it would be wise for small files?
RSS + JabberD Better idea (Score:2)
Thanks for the reference. One comment made there I thought immediately too--Syndication over IM is far more interesting. Every kid I know under 16 has given up email and uses IM file transfer instead.
Email and Browsers are probably dead technology for the next generation. HTTP/HTML will survive as a backwater because it is so useful for behavioral engineering. If you write a user interface, you can't use blue underlines because users will try to click them--very Pavlovian.
We will have to go back to wh
Re:RSS + JabberD Better idea (Score:2)
Syndication over IM is far more interesting.
We're getting there. Trillian [trillian.cc] has an RSS reader plugin. Admittedly, it's not as easy as right clicking on an article and sending it to a contact (or many contacts), you have to right click select copy, then right click and paste...but considering you can't mass message people on your contact list yet in any IM that I know of, this comes as no surprise.
Re:Covered this previously just before christmas (Score:3, Interesting)
(I've noodled around trying to distribute RSS loads but it's hard to make it worthwhile.)
The
Other comments in the replies decrying corporate invol
Clarification (Score:2)
More ways for crap to flood in. (Score:3, Insightful)
However, remember when cable gained enough steam to warrant not one but many 24hr cable news networks? We are now blessed with an overabundance of crappy sensationalist "reporting". I do NOT want cnn/msnbc/fuxnews/etc. landing on my HD.
If an individual set up a feed for say, a favorite game or movie alone, I would subscribe. But most webpages I read, I gloss over quickly then am done with.
If I, and everyone else had subscribtions to all of the media content of their favorite websites delivered autonomously, the majority of it getting thrown out quickly...
think of the bandwith, the poor helpless bandwith, won't somebody please think of the child., er bandwidth!?
Good, but not perfect... yet. (Score:5, Interesting)
RSS+BitTorrent, is a step closer to a better web. It almost answers the problem of pointing your client at an actively downloaded torrent by steering users twoard a slimmer and more flexible protocol.
IMO, maybe some kind of 'standard' torrent directory/lookup that is guarnteed to be traded by all torrent clients is the right ticket; kind of like a DNS for media. The RSS+Torrent scheme is good, but all it does is displace the complexity of the matter onto a new protocol and rely on everyone hitting the same feed to begin (the problem Torrent is trying to eliminate).
It does however, make it easy to make distributing torrents a lot more dynamic. Neat stuff.
H.323 is like DNS too (Score:2)
Look at the GNUGK project (Gnu Gatekeeper). Think DNS-like routing of audio calls. BitTorrent Phone Home.
Re:Good, but not perfect... yet. (Score:2)
It enforces a share and share a like policy, no leeches.
It's impervious to legal challenges, well as much as any other protocol.
It's open source, no vendor lock in.
It's newish, let's not forget the importance of something shiny.
It works.
Already done - Konspire2b (Score:5, Interesting)
Konspire2b [sourceforge.net]
Essentially you subscribe to channels which push content instead of pulling.
Compared to Bittorrent [sourceforge.net]
This is an exhaustive analysis (with pretty charts) why under the above scenario (pushing content, as opposed to pulling), Konspire2b is much more efficient.
Re:Already done - Konspire2b (Score:5, Informative)
A scattered model gives BT as taking O(log(number of people)/(number of chunks) + 1) time for everyone to download the whole file instead of O(sqrt(number people)) as claimed in the article.
Re:Already done - Konspire2b (Score:2)
Knocking swarming, which nicely deals with users dropping out of the network.
Demoing a torrent with three chunks, which is just really small.
Assuming the second, third, fourth, fifth downloader is blocked when most clients will happily upload one chunk it has to four people at the same time.
Generally trying to hide behind block transfer sequencing, overlooking how torrent really works, and glazing over the simple issue that collective bandwidth out is the real limiting fact
Re:Already done - Konspire2b (Score:2)
Re:Already done - Konspire2b (Score:3, Interesting)
Konspire2b is suboptimal [slashdot.org].
Had an Idea like this a long time back... (Score:2)
I wanted on demand television that you could find from mirc downloads and then eventually BitTorrent. The idea would be for a really nice multimedia center attached to your TV that would download shows that you missed or if you couldn't record it (conflicts). Updates for popular programs could be downloaded and installed when the user attempts to update (as opposed to a live update). Harddrive sizes are definitely big enough to handle. The only challenge in my last im
How long before ... (Score:4, Interesting)
???
Sorry if I seem like I'm trolling but these questions will be asked at some point
Re:How long before ... (Score:4, Insightful)
(Someone correct me if I'm wrong about this)
What a great idea! (Score:3, Insightful)
The funny thing is, I ran into Andrew the other day, and he was just gushing about this new idea he had! I had no idea what he was talking about at the time. Guess I missed my chance to post a story on slashdot.
we'll use this on ampfea.org (Score:2, Informative)
but, we'll definitely use an rss-fronted bittorrent network, if and when it can actually be smoothly integrated with our existing setup.
ampfea stands for 'a meeting place for electronic artists' [ampfea.org] and its a community-supported media hosting/community service for a bunch of muso's
Anime (Score:2, Informative)
Almost Nirvana (Score:3, Funny)
RSS's "pull"/polling model is the real problem (Score:5, Informative)
This problem is easily addressed with multicasting. All a server need do is send a multicast datagram to notify all RSS syndicators that the RSS document has been updated, at which time the syndicators can fetch the new document.
Re:RSS's "pull"/polling model is the real problem (Score:2)
Multicasting (Score:2, Insightful)
P2P: The Meme War Continues (Score:3, Interesting)
In the right hand corner: Hackers Embrace P2P [yahoo.com]
This article is more Wired garbage. (Score:3, Insightful)
I don't understand where they are coming from here. If I am going to pay to download music, which consists of relatively small files, I am not going to run a BT to help out an online music store.
When they start mentioning uses so far off the base of reality, the whole article starts to smell of BS. Especially since the slowest part of the MP3 experience tends to be copying music from the PC to the player.
Typically, I load new tracks on my ipod before leaving in the morning. I'll tag the stuff, then transfer it before I hop into the shower. As far as downloading goes, I can download a whole CD of music in ~10 minutes. The only way the article's method would be worth doing would be if you invested in huge libraries of online music purchases daily.
On that note: Please quit looking to solve problems that don't exist.
Re:This article is more Wired garbage. (Score:5, Interesting)
Let's say your favorite band just went on tour and as part of a promotion they decided to post a few songs and videos "bootlegged" from each concert.
Now, it might get kinda anoying to load up their page every couple of days and click on each link to download the media. However, they could post an RSS feed with BitTorrent links that you subscribe to just once. Everytime a new bootleg goes online, the RSS feed gets updated, and the content gets downloaded to your computer automatically.
Where would we be if everytime the Internet was mentioned 50 years ago, people ranted and raved about how the postal service already solved the problem of distributing content?
This is seriously cool stuff, you are just too closed minded to realise it.
Re:This article is more Wired garbage. (Score:2)
The possibilities are always endless. Reality happens to be only one of those.
is this the same thing as konspire2b (kast)? (Score:3, Informative)
A million years ago (1998?) Wired published a whole edition on Push as the Next Big Thing. It was the first time I was really aware of them being totally wrong. Or perhaps just a bit ahead of their time.
While I think this is a neater solution, there is another product that does exactly the same thing, allow you to subscribe to channels and received pushed content via incentive compatible (you get faster speeds if you upload more) swarms.
It's called kast [sf.net].
Bittorrent kind of sucks (Score:5, Informative)
* a file is seeded, and a
* that
* clients who want to download the file download the
* the user opens the
* the client downloads various chunks of the files from both the seeds and the other downloaders
The more people download a file, the better bittorrent is able to spread the bandwidth.
The downside is that if a file isn't seeded, it's no longer available. If a
Bittorrent's main problem right now, which is a client problem, is its upstream usage can easily swamp a home connection. That's just dumb client design.
Upload limiting works, but limits your download speed. The client develoeprs have to recognize that yes, sharing is nice and leeching is bad, but disrupting the users' connection is a Very Bad Thing.
Re:Bittorrent kind of sucks (Score:4, Interesting)
I think that ideally, the most a user should see is bt://sitename.domain/file.zip, or something similar. The OS/browser should be able to handle that sort of protocol, and send it to the right application or use an integrated bit torrent client to get the file.
Correct me if I'm really wrong on this, or if it already exists. This would also be a welcome addition to Mozilla, I think.
Re:Bittorrent kind of sucks (Score:2, Interesting)
The approach that BitTorrent took of distributing the protocol information in a file is much more flexible. The option exists to name it file.zip.torrent* and as long as the proper mime types ha
You can do that already! (Score:3, Informative)
Re:Bittorrent kind of sucks (Score:4, Interesting)
A decent client, of which there are many, will let you throttle your outbound.
Note that the way torrent works, if noone uploads, noone downloads. And the faster everyone pushes, the faster everyone gets. Its not so much an artificial thing as it is an economy of bandwidth.
And again, if you don't like sending full tilt, find a better client.
Why RSS? (Score:3, Interesting)
It's a shame they're using RSS, as it's a good idea with a bad implementation. There are currently 9 different versions of RSS, and all of them incompatible with one another [diveintomark.org]. It ought to be replaced with a better technology like Atom [intertwingly.net]. However, this does look like an interesting project, nonetheless.
What Bit Torrent needs more is: (Score:2, Interesting)
Encryption. It should generate a unique key pair for each socket it opens.
Re:BitTorrent (Score:5, Informative)
It allows for people to take advantage of bandwith by downloding bits of a large file from different users hosting a 'torrent.' At the end, all these pieces are put together. Yes, it is pretty good.
Re:BitTorrent (Score:2)
Well, since the reply post WAS informative, and anyone who hadn't heard of BitTorrent just got a nice quick write-up about what it does, it was a dumb troll in the first place.
HAND, AC. Or maybe FOAD is more appropriate, take your pick.
Re:BitTorrent might be in trouble (Score:2)
Re:BitTorrent might be in trouble (Score:3, Informative)
I do know that Bram talked with the authors of Furthur [slashdot.org], an open-source JAVA P2P for legal content. A few members of the Furthur dev team also work for a company that once did buisness with Sharman Networks, so if anything, KaZaA source may be based on already GPL-ed software... but don't tell anyone that
Of course, all modern P2P is 'based' on Napster or Gnutella (take your pick), so it's all a mute point anyway
BitTorrent is not based on Kazaa (Score:4, Informative)
BitTorrent is open source (MIT license) and written in Python.
Kazaa is closed source, spyware-ridden dreck that was probably written in C++.
Re:BitTorrent might be in trouble (Score:4, Informative)
Also, Kazaa is in trouble not for it's protocol, but for running servers that allow piracy, it's just in Kazaa's case one automatically means the other, since the protocol is closed source. Of course, Bittorrent trackers that host pirated material are also susceptable to such troubles - but this has nothing to do with Bittorrent protocol itself.
BT is a bandwidth sharer not a file sharer (Score:3, Informative)
If the author of BitTorrent could be sued the authors of ftp and apache could be sued as well.
I hope this helps, your misconception of how BT functions is fairly common. It's not the same type of p2p network as Kazaa at all.