Faster Feeds Using FeedTree Peer-To-Peer

dsandler writes "Researchers at Rice University have just released version 0.7 of FeedTree, a peer-to-peer system for distributing Web feeds faster. Instead of polling feeds independently, FeedTree users cooperate to share news updates using multicast in Pastry, a scalable p2p overlay network. FeedTree reduces the update delay for existing RSS and Atom feeds to a few minutes without putting extra stress on the webserver (anyone who's ever been temporarily banned by Slashdot's RSS feed knows this is a real concern). Feed publishers can also choose to push digitally signed updates for immediate, tamper-proof delivery to subscribers. The client software (download) runs on Linux, OS X, and Windows, and works with any desktop feed reader."

Re:So...

[twiggy]

Now how do we get a plugin for something like this mass installed into browsers like that of flash, so that a web app like google reader (but independently developed, maybe specialized to other things) can take advantage of this idea?

That would make for a real nice way of creating awesome web based feed aggregation tools...

Re:Why?

[gkhan1]

This is not really the same thing. This is for feed-distribution, not cooperative downloading of large files. The files are rather small in this case

They just reinvented netnews

[Animats]

It looks like they just re-invented the netnews protocol, which works in a very similar way.

Re:Why?

[Jerf]

As already mentioned, this doesn't compete with Bittorrent, because bittorrent isn't designed for RSS feeds. Along with the file size issue idonthack mentioned (torrents are only a win when the size of the file being transferred is much much larger than the coordination overhead, generally not the case for RSS), BitTorrent is also not designed for files to change over time; it would require a complete overhaul of the protocol because the file hashes that are the foundation of the protocol would be constantly changing.

There is room for coordination with bittorrent, though; imagine a Pastry-based P2P feed that then used RSS enclosures to tie into a (trackerless?) BitTorrent feed for a fully distributed pod-/vid-/file-casting solution that anybody could run with no fear of the bandwidth involved.

Tack in some sort of P2P web system, and in theory, you could run a massively popular podcast/blog with millions of hits a day off of your cable modem. (Although something with a bit more upstreaming oomph would be good for the rarely-requested content that falls out of the P2P; anyhow, any ol' webhost could handle this kind of bandwidth.)

I think this is a worthy goal, as if nothing else, popular websites run for fun would no longer be faced with the dilemma of advertising to cover bandwidth costs or going offline.

GMail RSS

[jfengel]

I wonder: If GMail were to incorporate an RSS reader (the way Thunderbird does), it could potentially update many, many users with a single hit of each RSS site.

I'm leaning towards using RSS as a way to do announcements rather than maintain a mailing list. Rather than tell me you want me to send you updates (and deal with being potentially a spammer, deal with your unsubscribe, your email address change, etc.), just poll my site every so often (days, for the lists I'm talking about; hours, for Slashdot) and let it show up in your mail queue.

The idea isn't quite ready for prime time; too few people use RSS. But GMail could make that happen in one fell swoop. Well, two fell swoops: you'd need some sort of browser extension to make the little orange "RSS feed" button notify GMail.

I wonder if just having GMail (and hotmail, aol, and yahoo) handle that would solve the problem to the point where we no longer needed a P2P RSS distribution system.

Alternatively, if ISPs were to cache the RSS feeds the way some do with certain web pages, that might also take a lot of the load off. People will still impolitely set their RSS readers to check the feed every 10 seconds, but at least it never gets out onto the backbone if it's cached at the ISP.

Re:They just reinvented netnews

[That's Unpossible!]

That's like saying IMAP just reinvented POP3.

This is designed for USERS to help each other get the very latest RSS feeds using p2p tech.

netnews is designed to let SERVERS help each other distribute messages posted by users.

I don't really see how it is a re-invention at all.

Re:They just reinvented netnews

[John Hasler]

> This is designed for USERS to help each other get the very latest
> RSS feeds using p2p tech.

> netnews is designed to let SERVERS help each other distribute
> messages posted by users.

> I don't really see how it is a re-invention at all.

Usenet is a peer to peer network of "servers". This is a re-invention of the way articles propagate in Usenet.

Re:Rice made Pastry, too.

[xiphoris]

LOL, that was not my intention, though I guess I see how it could have come off that way. I take pride in my university, the people I have worked with, and applaud them for the things they produce.

I don't even know Dan personally, though I have researched under Drs. Wong and Wallach before. I'm just trying to point out where credit is due.

Is it wrong to want your fellow students to be praised for their hard work?

A solution without a problem

[DrHanser]

I'm afraid I don't understand what problem this is solving. It's like a solution that's still looking for an problem to solve. As an end user, why should I care? I'm not trolling; I just don't get it.