Faster Feeds Using FeedTree Peer-To-Peer 109
dsandler writes "Researchers at Rice University have just released version
0.7 of FeedTree, a peer-to-peer
system for distributing Web feeds faster. Instead of polling feeds
independently, FeedTree users cooperate to share news updates
using multicast in Pastry, a scalable p2p
overlay network. FeedTree reduces the update delay for existing RSS and Atom
feeds to a few minutes without putting extra stress on the webserver (anyone
who's ever been temporarily banned by Slashdot's RSS feed knows this is a real
concern). Feed publishers can also choose to push digitally signed updates
for immediate, tamper-proof delivery to subscribers. The client software (download) runs on Linux, OS
X, and Windows, and works with any desktop feed reader."
Re:So... (Score:4, Insightful)
That would make for a real nice way of creating awesome web based feed aggregation tools...
Re:So... (Score:3, Funny)
"We" get this as soon as you 1) write it for all known platforms that people use for the web 2) integrate it into all of the known browsers for all known platforms.
I see zero need for a plugin where a standalone app can do the same thing without destroying my web experience in
Re:So... (Score:1)
When did we stop?
Re:So... (Score:2)
With all the buzz about "web 2.0" (again, love it or hate it) - this would be the perfect application to make a plugin.
Granted, it just so happens that such a plugin would vastly improve a project I'm working on because it would take load off of my server - but I know that many others are working on projects that would
Mozpache (Score:1)
The problem is, the web is a mostly one-way architecture and certainly doesn't work in any sort of a p2p fashion.
As Tim Berners-Lee originally conceived the World Wide Web, each computer would run both a client that dials out on port 80 and a server that listens on port 80. In practice, only the pervasiveness of dial-up Internet connections and duopolistic residential broadband ISPs' terms of service have necessarily interfered with this vision.
Ok then, (Score:2)
Re:So... (Score:3, Funny)
OCILLA and caches (Score:2)
No. Anything already made available to the public on the Web is subject to the copynorms and copyright exemptions of the Web. For example, under 17 USC 512 (enacted as a rider to the DMCA), those who operate automated caches on a computer network are not liable in any United States court for damages that result from copyright infringements performed through such caches. To learn more, read about the OCILLA at Wikipedia [wikipedia.org].
Why? (Score:3, Interesting)
Re:Why? (Score:5, Informative)
Re:Why? (Score:1)
It works really well for downloading TV shows off of some sites, for example.
Naturally, because it's bittorrent, it's great at downloading b-i-g files, whereas FeedTree sounds like it's more about dist
Re:Why? (Score:2)
It is unfortunate that most people think "peer-to-peer" and "file sharing" are synonymous. They're not. Peer-to-peer has many, many users outside of file sharing/distribution.
Re:Why? (Score:1)
If they simply said it was for distributing RSS feeds across a peer-to-peer network to take load off of servers hosting RSS content, that would have been much clearer. Even if they said that it would allow you to host a very po
Re:Why? (Score:1)
Re:Why? (Score:2, Insightful)
Re:Why? (Score:5, Insightful)
There is room for coordination with bittorrent, though; imagine a Pastry-based P2P feed that then used RSS enclosures to tie into a (trackerless?) BitTorrent feed for a fully distributed pod-/vid-/file-casting solution that anybody could run with no fear of the bandwidth involved.
Tack in some sort of P2P web system, and in theory, you could run a massively popular podcast/blog with millions of hits a day off of your cable modem. (Although something with a bit more upstreaming oomph would be good for the rarely-requested content that falls out of the P2P; anyhow, any ol' webhost could handle this kind of bandwidth.)
I think this is a worthy goal, as if nothing else, popular websites run for fun would no longer be faced with the dilemma of advertising to cover bandwidth costs or going offline.
Re:Why? (Score:1, Interesting)
Mark my words: Microsoft is going to attempt to co-opt the term 'p2p', and make it their own.
Re:Why? (Score:2)
Re:Why? (Score:2)
The cure: RTFA.
Re:Why? (Score:1)
They just reinvented netnews (Score:4, Insightful)
Re:They just reinvented netnews (Score:2)
Re:They just reinvented netnews (Score:5, Insightful)
This is designed for USERS to help each other get the very latest RSS feeds using p2p tech.
netnews is designed to let SERVERS help each other distribute messages posted by users.
I don't really see how it is a re-invention at all.
Re:They just reinvented netnews (Score:3, Insightful)
> RSS feeds using p2p tech.
> netnews is designed to let SERVERS help each other distribute
> messages posted by users.
> I don't really see how it is a re-invention at all.
Usenet is a peer to peer network of "servers". This is a re-invention of the way articles propagate in Usenet.
Re:They just reinvented netnews (Score:2)
Except that Feed Tree doesn't propagate articles for usenet, it propagates entries posted to RSS feeds. I'm really trying to understand how this is a re-invention of netnews?
Re:They just reinvented netnews (Score:2)
That said, RSS/ATOM have a single source of the truth, while usenet is a web of inserters and receivers. RSS/ATOM are uniformly linear in nature, usenet is not. RSS/ATOM are by intention very very short entries; usenet posts can be much larger. RSS/ATOM are not intended
Re:They just reinvented netnews (Score:2)
USENET is a way for articles to be propogated among coordinating servers, and then users would poll those distributed servers. RSS is a way for distributing articles with only one server, and the users query it directly. What this tech does is create a way for articles to be propogated among coordinating servers, but those servers are also the users. The users then query other users, who are acting as servers, and then become se
Re:They just reinvented netnews (Score:2)
> content distribution.
Actually the decomposition of Usenet into servers and clients is a relatively recent phenomenon. Originally we read news directly from the spool using local clients. Indeed, it is still quite possible to run your own local server as a leaf node, receiving only those newsgroups you are interested in. I have been doing just that for about twenty years.
Re:They just reinvented netnews (Score:2)
That's the way USENET used to be used, and a lot of USENET software still supports that usage, including automatically locating and subscribing to newsgroups only when a user demands it. It's become more static and centralized because users preferred using it that way, not because of a
Re:They just reinvented netnews (Score:2)
USENET propagates news items with metadata in a tree-like fashion, overlayed over the Internet. FeedTree propagates news items with metadata in a tree-like fashion, overlayed over the Internet.
There are some minor differences in standards (MIME vs. XML) and usage (well-known article hierarchies vs. ad-hoc RSS feeds), but that doesn't make FeedTree new technology. I don't think it's even a "re-invention",
Reinventing it *Well* or *Badly*? (Score:2)
Re:They just reinvented netnews (Score:1)
but a quick check of the sites shows a rather different architecture.
This seems more targeted towards RSS type feeds, and looks like one of
those rather simple and clever ideas that strike one as:
"This looks like the way it should have been done from the beginning"
It addresses a very real problem with current RSS news feeds, and has what
looks like simple (that's a complement), complete, compatible, easy to install
software for a reasonable varie
No, they reinvented BBS (Score:1)
Feed Reader (Score:2)
Re:Feed Reader (Score:2)
http://mac.softpedia.com/get/Internet-Utilities/V
Re:Feed Reader (Score:1, Informative)
Does the client work on FreeBSD? (Score:2)
Re:Does the client work on FreeBSD? (Score:2)
The client and publisher both run on any system with the Sun Java runtime, 1.4.2 or newer. (The networking code in Pastry requires Sun's NIO implementation.) As for the publisher helper scripts, the configurator is Python, and the run control scripts are Bourne shell.
In other words, It Ought To Work(TM) out of the box on FBSD. If not, file a ticket [feedtree.net].
Obligitory Simpsons quote. (Score:2, Funny)
Re:Multicast?!?!?!1ONEONEONE (Score:2)
Already getting hit by Shrook (Score:4, Informative)
66.177.198.139 - Anonymous [04/Apr/2005:03:04:17 -0500] "GET
I haven't seen a hit from this in a while, perhaps that effort didn't gain much traction. Who knows if this one will... I never saw Shrook mentioned on Slashdot.
Re:Already getting hit by Shrook (Score:2)
GMail RSS (Score:5, Insightful)
I'm leaning towards using RSS as a way to do announcements rather than maintain a mailing list. Rather than tell me you want me to send you updates (and deal with being potentially a spammer, deal with your unsubscribe, your email address change, etc.), just poll my site every so often (days, for the lists I'm talking about; hours, for Slashdot) and let it show up in your mail queue.
The idea isn't quite ready for prime time; too few people use RSS. But GMail could make that happen in one fell swoop. Well, two fell swoops: you'd need some sort of browser extension to make the little orange "RSS feed" button notify GMail.
I wonder if just having GMail (and hotmail, aol, and yahoo) handle that would solve the problem to the point where we no longer needed a P2P RSS distribution system.
Alternatively, if ISPs were to cache the RSS feeds the way some do with certain web pages, that might also take a lot of the load off. People will still impolitely set their RSS readers to check the feed every 10 seconds, but at least it never gets out onto the backbone if it's cached at the ISP.
Re:GMail RSS (Score:2)
Re:GMail RSS (Score:3, Informative)
Re:GMail RSS (Score:2)
Re:GMail RSS (Score:2)
There are already several large Web-based aggregators that work this way, but for various reasons many people prefer local aggregators (just as many people prefer local mail clients instead of GMail). FeedTree solves the bandwidth and latency problem for local aggregators.
Google Reader (Score:2)
They'll likely integrate this with GMail at some point. But that's just my opinion.
Wow... (Score:3, Funny)
Re:Wow... (Score:1)
That wasn't so hard........
Rice made Pastry, too. (Score:4, Informative)
Microsoft Research became interested in the product and ported it to C#, effectively turning it into the form it is now. Many classes at Rice have now "backported" it, I guess you could say, and it's used for many of our classes that involve distributed networks, such as the current COMP 410 [rice.edu] class which has previously turned out distributed file and process system codename Voltron [rice.edu].
Here's a link to the paper [rice.edu] co-authored by Sandler and others at Rice.
Re:Rice made Pastry, too. (Score:3, Informative)
Actually, my advisor, Peter Druschel [mpi-sws.mpg.de], developed Pastry with Ant Rowstron (of Microsoft Research). Since then, a number of bright researchers from Rice and elsewhere have contributed to the project; their names and publications are listed on the official Pastry website [freepastry.org].
There are a number of implementations of the Pastry design; FeedTree uses the Java-based FreePastry [freepastry.org] package, which is under active development by Rice and the Max Planck Institute for Software Systems [mpi-sws.mpg.de] and is available under a BSD-like licen
Pastry based Squirrel seems much more exciting (Score:2)
from the site:
If everybody used this, then there'd be no need for mirrordot [mirrordot.org] and the slashdot effect would be a thing of the past and more people could afford to host pr0n on their personal websites ;)
Re:Rice made Pastry, too. (Score:2)
Dr. Wong also said, during my time in COMP 410, that Rice U. was entirely responsible for Pastry before MS took it.
So what's the real story? Did we make it or not? X_x
Congratulations on the
Disturbing trend: MS Funding kills Java App for C# (Score:2)
Re:Disturbing trend: MS Funding kills Java App for (Score:2)
So, it seems I should've dug deeper before making my previous comment. Sorry about that folks
Re:Disturbing trend: MS Funding kills Java App for (Score:3, Informative)
And yes, we use entirely Microsoft software. But I think it's a good thing. When I to
Re:Disturbing trend: MS Funding kills Java App for (Score:2)
Java is, and always has been, a proprietary technology completely specified by Sun. Sun owns the specs and decides what language features to add. Period.
The
Re:Disturbing trend: MS Funding kills Java App for (Score:2)
Microsoft is in complete control over the future of the C# language and the .Net libraries and runtime. Just because they do the standards dance doesn't mean they've given up control. Do you honestly think that C# or .Net can change in a way Microsoft doesn't approve of?
The ECMA even allows the standard to be patent-encumbered as long as Microsoft provides "reasonable and non-discriminatory" licensing fees. That makes me feel com
The patents were released. (Score:2)
Um, no.
Sorry, but that's
Re:The patents were released. (Score:2)
I didn't know about the patent grant. That's good news.
Now all that's left is your claim Sun controls Java while C# and
Re:Disturbing trend: MS Funding kills Java App for (Score:2)
Microsoft didn't co-opt anything, and in fact allowed and encouraged the open source Java version initially. These days I understan
Re:Rice made Pastry, too. (Score:3, Insightful)
I don't even know Dan personally, though I have researched under Drs. Wong and Wallach before. I'm just trying to point out where credit is due.
Is it wrong to want your fellow students to be praised for their hard work?
Re:Rice made Pastry, too. (Score:2)
I have worked under Dr. Wong for a couple of years at Rice, working on Pastry-related and other distributed systems. No, those people are not me, but I have worked with them, and am
License Terms? (Score:2)
Re:License Terms? (Score:2)
Copyright (c) 2006, Rice University
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this l
Does /. RSS push its updates to this? (Score:1)
RSS news distribution.
It's nice to be able to browse the source code.
What can we do to encourage adoption of this, before some wretched
proprietary format tries to muscle in?
Web-Based News Reader (Score:1)
The reason this is a issue (Score:2)
Micropayments would solve this. Pay 0.001 for every reload automaticlly and you wouldn't need a solution like this. Fix that and solve thousands of small problems at once.
Re:The reason this is a issue (Score:2)
Re:The reason this is a issue (Score:2)
Re:The reason this is a issue (Score:2)
Re:The reason this is a issue (Score:1)
People put their RSS-aggregator to reload every fifth minute, if you're lucky you might get a real ad impression once a day.
Then only serve one distinct ad per unique visitor per day.
How does this relate to scribe? (Score:2)
One interesting thing to note is that as a participant in scribe, you'll have to pass on notifications of feeds even if you're not interested in them, because you're a part of the tree and pretty much the only path to the guys below you. How does FeedTree deal with cheating/lying nodes that refuse to pass on message
A solution without a problem (Score:3, Insightful)
Re:A solution without a problem (Score:1)
This solution, however, would allow one source to poll the server, and use P2P to transfer the content to the many clients as want it - and as often as they want it. No additional strain is being put on the source, and the clients are all happy.
Truly beneficial? (Score:2)
I've been thinking for quite some time of utilizing this type of P2P distributed caching proxy concept with many different protocols. RSS is just one possibility amongst many that could utilize the basic technology here. Some others might include distributed file systems, distributed caching http proxies, or even a Google competitor that uses a distributed P2P implementation of the database and utilizes everyone's everyday web activity to augment the spidering (i.e. every time anyone who is part of the P2
New game in town... (Score:2)
New game in town: never use the word Java. BTW, it doesn't run on Linux and Windows. Except if you install Java of course.
Azureus also runs on the Java platform (Score:1)
it doesn't run on Linux and Windows. Except if you install Java of course.
A lot of people who are interested in peer-to-peer networking have installed a Java platform, even if only to run the Azureus client.
Why not use push? (Score:2)
I know it's a crazy suggestion, but instead of having hundreds of people polling a single RSS feed, why not have the server which hosts the RSS feed actually PUSH the updates out to the people who are interested?
We already have a nice and simple protocol (XMPP) which could be used for this, although admittedly PubSub isn't as final as it could be.
Re:Why not use push? (Score:2)
FeedTree can operate in that mode, or if the server operator is too lazy to install it, FeedTree will still provide some benefit.
We already have a nice and simple protocol (XMPP) which could be used for this, although admittedly PubSub isn't as final as it could be.
Doesn't this lead to potentially high fanout (with the attendant conce
Re:Why not use push? (Score:2)
Re:Why not use push? (Score:2)
Re:Why not use push? (Score:2)
Similar in some ways to Miski (invented in 2000) (Score:1)
More details at How I Invented a Decentralised Scaleable Push-Based Micronews System in 2000 [1729.com].
If nothing else, my documented but unimplemented invention might be good prior art, should it be needed.
Pah! (Score:1)
Distributed peer-to-peer web 2.0 rss news updates? You young whippersnappers and your fancy-schmancy names!
In my day we simply called it gossip!