Freecache 258
TonkaTown writes "Finally the solution for slashdotting, or just the poor man's Akamai? Freecache from the Internet Archive aims to bring easy to use distributed web caching to everyone. If you've a file that you think will be popular, but far too popular for your isp's bandwidth limits, you can just serve it as http://freecache.org/http://your.site/yourfile instead of the traditional http://your.site/yourfile and Freecache will do all the heavy lifting for you. Plus your users get the advantage of swiftly pulling the file from a nearby cache rather than it creeping off your overloaded webserver."
Not solution to slashdot effect, but still great! (Score:5, Insightful)
Well, it won't be the solution to Slashdotting, as you can't cache a whole site.
You can cache an HTML page (index.html) but all the images will pull from the local machine. You could cache each image separately, but the change would have to be made in the site's HTML.
On the other hand, I don't imagine it would be hard to write some kind of proxy script that grabs the page and changes the HTML to point to freecache SRCs for each image/movie... you could then point to a freecache of that page...
And of course, this all breaks the second somebody has a site that is heavily CGI based.
Still, it's a start. I'll be sure to use it if I ever submit any site of my own to Slashdot ;-) Many thanks to the guys at the Internet Archive for setting this up. You rock!
Some questions (Score:5, Interesting)
I have a few questions though, which I guess may be answered on the website:
1. Can users submit/upload files to be hosted on their website.
2. Who's responsible for ensuring that it doesn't turn into a pr0n/warez stash?
3. Can users request removal of cached content (something not possible with the Google cache).
Re:Some questions (Score:5, Informative)
Actually, you can request removal of a google cache [google.com], but you must have access to the reference source site to do so. Once you've requested removal, there is even a personalized status page where you can check the progress of the removal.
Slashdotted?! (Score:3, Funny)
Definitely not an adequate solution, given it's current condition: slashdotted to hell.
Idiots! They should've had it cache itself first before posting this to /.
Re:Not solution to slashdot effect, but still grea (Score:4, Interesting)
In other words, the important stuff, like the rest of the site and the pictures, will be resources only used on those that really care, while those that don't get to see a flash of the text for a second to get a really general idea.
After all, thats what the slashdot effect is, a whole bunch of people that don't really care that much, but want a quick, 5 second look at it.
Re:Not solution to slashdot effect, but still grea (Score:4, Insightful)
Re:Not solution to slashdot effect, but still grea (Score:3, Funny)
The other 25% is us looking at a page for 5 seconds and then replying because as everyone knows here, it's much more entertaining to reply without RTFA.
Re:Not solution to slashdot effect, but still grea (Score:5, Informative)
Re:Not solution to slashdot effect, but still grea (Score:2)
I should clarify that I mean this will not be the solution to the effect caused by "surprise-slashdotting" where the site owners are not notified ahead of time.
If a savvy site owner is notified by slashdot editors before being listed, they might be able to take some preventive action.
I don't think that currently happens very often, though.
Re:Not solution to slashdot effect (Score:5, Informative)
Re:Waiting for the resource file. (Score:3, Interesting)
Re:Not solution to slashdot effect, but still grea (Score:2)
Maybe, when the site is no longer slashdotted, people will be able to get a look at their FAQ and see that (the submitter should have done that instead of submitting stuff that he just discovered, even if it has been availlable for a long time, without even checking for it's real purpose).
Alternative solution (Score:4, Interesting)
Re:Alternative solution (Score:3, Informative)
Combining that with bittorrent should be relatively easy.
Of course, you'll probably have to view the result in IE, as the mozilla project hasn't quite worked out
Re:Alternative solution (Score:4, Informative)
What??
Re:Alternative solution (Score:5, Informative)
Re:Alternative solution (Score:2)
So instead of a URL to the image, it has the image data directly in the IMG tag.
Someone probably only has to write some JS code in Mozilla to join all the features together, the question if if JS can then pop-up the "Save As" dialog box, I think n
Re:Alternative solution (Score:3, Informative)
Re:Alternative solution (Score:2)
Re:Alternative solution (Score:4, Informative)
You'd have to come up with a scheme like:
Re:Not solution to slashdot effect, but still grea (Score:3, Insightful)
Which is unlikely. So it won't be cached. Nor will the PNG/GIFs.
Ratboy
Re:Not solution to slashdot effect, but still grea (Score:3, Funny)
<!--
<?php
for ( $i = 0 ; $1 < 5000000 ; $i++ )
{
print "a";
}
?>
-->
Hey presto. All your pages are > 5MB!
Re:Not solution to slashdot effect, but still grea (Score:4, Funny)
Only on /. could you find someone optimizing code that would be used to bloat web pages.
Re:Not solution to slashdot effect, but still grea (Score:2)
Re:Not solution to slashdot effect, but still grea (Score:2)
relative addressed image URLs would apply, as the local browser would pull them from a URL based on the URL the current page lives at.
Smart! (Score:2)
Caching is intelligent because we are interested in the content itself rather than the connection to that particular computer.
Re:Smart! (Score:3, Funny)
Let me guess.. you're posting this from soviet russia?
USHERING IN A NEW ERA OF KARMA-WHORING (Score:4, Funny)
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:5, Funny)
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:2)
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:2)
From the STATUS page linked off the main page from the story:
"This page is offline. Sorry for the inconvenience."
D'oh! Slashdotted!
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:2)
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:2)
Re:USHERING IN A NEW ERA OF KARMA-WHORING (Score:3, Informative)
Surviving Slashdotting through Freecache (Score:5, Funny)
He was apparently
Cache owner's liability (Score:4, Interesting)
But would they? I saw this on the new service's message forum [archive.org]
I was perusing the content in my cache and checking the detailed status page and I noticed illegal content containing videos in one of the caches I run. What is freecache.org doing to stop people from mirroring illegal content. I currently run 2 fairly heavily used caches and it looks like only one of them had illegal content. I cleared the cache to purge the problem, but the user just abused the service again by uploading the content again. I know freecache.org cannot be responsible for uploaded content, but there has to be some sort of content management system to make sure freecache doesn't turn into just another way to hide illegal content.
Whether you believe this guy's story [slashdot.org] or not, it seems like this could subject small ISPs to the sort of problems that P2P has brought to regular users. It's not going to matter who's right -- just the idea of having to go to court over content physically residing on your server is a risk I don't see a marginal ISP being willing to take.
So we're left with the folks with static IP addresses. They're in even more trouble if John Ashcroft decides to send his boyz over to check for "enemy combatants" at your IP address.
With the current state of affairs in the US, and the personal risk involved, I'd have to pass on this cool concept.
Why is this a problem? (Score:5, Insightful)
It may be reasonable to expect them to pull content that is illegal where they are located, but that should be a simple matter of notifying them, they pull the content, no harm done. They may even be required to disclose the identity of the uploader, after which this person can be prosecuted.
I don't think anything in this scenario is outrageous or unfeasible. What is outrageous and infeasible is holding the host responsible for what the user uploaded. Then why is this the way it happens all too often?
Putting freecache to the test (Score:5, Funny)
http://freecache.org/http://freecache.org/http://
seems to piss it off slightly. I wonder why...
Re:Putting freecache to the test (Score:3, Funny)
I'm sure he would have made a deeper recursion, but the Slashdot lameness filter was able to compress it too efficiently.
The business model is astounding... (Score:3, Insightful)
2. Host extremely popular web sites
3. ???
4. PROFIT!!!
How are they supposed to be making money on this?
Re:The business model is astounding... (Score:3, Funny)
Re:The business model is astounding... (Score:2)
Maybe by using hosting in Soviet Russia???
I know, I know, I just couldn't resist.....
Re:The business model is astounding... (Score:3, Interesting)
2. Cache extremly popular media files for your customers
3. Advertise that customers can access Freecached files from the local network instead of the Internet.
4. Get more customers and pay less bandwidth costs.
5. PROFIT!!!!
Re:The business model is astounding... (Score:4, Interesting)
It's not a way of making money, it's a way of spending them. It's run by the Internet Archive, founded and funded by Brewster Kahle [wikipedia.org]. It's there for your free enjoyment - revel in the goodness of humanity!
Re:The business model is astounding... (Score:3, Informative)
It's not a way of making money, it's a way of spending them. It's run by the Internet Archive, founded and funded by Brewster Kahle [wikipedia.org]. It's there for your free enjoyment - revel in the goodness of humanity!
Re:The business model is astounding... (Score:2)
Or use Google... (Score:4, Informative)
I trust google to be faster than these guys.
Re:Or use Google... (Score:4, Insightful)
Re:Or use Google... (Score:3, Insightful)
The cool thing here is that you can say, "Cache just these things" and still have your server supply the html but not the images (or movies).
But you still have to have a decent pipe.
Taking bets.... (Score:2, Interesting)
Re:Taking bets.... (Score:3, Insightful)
Also only works for large files unless this FAQ [archive.org] is out of date:
What files are being served by FreeCache?
FreeCache can only serve files that are on a web site. If the link to a file on that web site goes away, so will the file in the FreeCaches. Also, there is a minimum size requirement
i've used freecache before (Score:3, Interesting)
too bad the status seems to be down, its fun to see what clips/games/demo/patches are going around.
Does it Cache Immediately? (Score:2)
if it does then I propose that all posts of smaller sites hence forth should be freecached.
anyone wanna second it? not that it will do any good.
Re:Does it Cache Immediately? (Score:2)
However, it will not cache resources that are under 5M. The cache is designed to cache for large piece of content.
This has been mentioned before (Score:4, Informative)
/.ed already... (Score:3, Insightful)
Questions to the Slashdot owners (Score:4, Interesting)
Re:Questions to the Slashdot owners (Score:2, Interesting)
The working solution would be for the slashdot editors to give a site owner a heads-up so that they can prepare for the flood.
Re:Questions to the Slashdot owners (Score:2)
Reasoning basically stands as follows: "they" would most likely
Beta! (Score:5, Informative)
Slashdotted (Score:4, Funny)
Oh crap that was the wrong link - try this:
http://freecache.org/http://movies03.archive.org/
Slashdot cache (Score:2, Redundant)
It's a FAQ (Score:3, Informative)
Can I say RTFFAQ now? :)
Re:Slashdot cache (Score:2)
What if only subscribers could access the cached sites?
Also, what about using robots.txt or meta tags to tell Slashdot a page can be cached?
... execpt (Score:5, Informative)
As their status page [archive.org] explains...
Using the Wayback Machine? (Score:2)
Re:Using the Wayback Machine? (Score:2)
Re:Using the Wayback Machine? (Score:5, Funny)
You must be new here, or you would know the the news is old here.
Re:Using the Wayback Machine? (Score:2, Funny)
Bwahahahahaha (cough)(cough) bwahahahahahahhhahaaaaaaaa
This will cause problems (Score:5, Interesting)
Freeache? (Score:2, Funny)
I mean, I'm all for free stuff, but an ache...?
Me. (Score:2)
Re:Me. (Score:2)
Yes, but the way he said it was much more interesting.
OMG, we've slashdotted archive.org (Score:2, Insightful)
I do not think this is a solution to slashdotting :-)
put it under the hood (Score:5, Interesting)
A better use of Freecache is "under the hood". Make your webserver redirect accesses to your "http://whatever.com/something" to "http://freecache.org/http://whatever.com/somethi
Re:put it under the hood (Score:2)
I believe that is the idea - not for users to
Re:put it under the hood (Score:2)
I could see people using this (Score:2)
Perhaps there should be an alternative to scientific projects and OSS projects.
freecache.fsf.org perhaps?
Nothing Changes (Score:2, Funny)
Solution to slashdot? Shoot, we broke it. (Score:2, Funny)
Not really.. I can't access their servers now. All will tremble before the might of slashdotting!
Already blocked at work (Score:3, Insightful)
KevG
Ironic (Score:5, Funny)
Story is only a few minutes old and mecca of Internet caching has already been slashdotted. Maybe someone kid with an old P5 266mhz under his desk can mirror the site for us.
a dud? (Score:2)
What we really need.... (Score:5, Insightful)
www.squidserver.com/http://www.doomedsite.com
The public squid will cache a copy of it. On the first access (like when the approver looks at it) It should look at a request and see if it has a recent cache. If it does feed that, if not get the newest copy and promth the user for a refresh or automatically refresh after a set time (5 sec). It will update its cache as the site does. All without having to upload anything. After a few days when nobody is utilizing the cache, it can purge it. Waiting for the next doomed site.
DISCLAIMER: The may be how Freecache works, but I can't get to it
1) because I am at work.
2) as the comments suggest it is slashdotted.
KevG
Re:What we really need.... (Score:3, Informative)
What makes you think that www.squidserver.com always resolves to the same single squid server? Intelligent DNS resolution in conjunction with things like IP multicast and multi-homing could be used to achieve something approaching what Akamai does.
Plus, the advantage of a proper caching HTTP proxy as the "meat" of this solution means that HTTP caching rules are respected. If a site has
New way to distribute viruses in popular files? (Score:2, Insightful)
How does this system guard against doctored content coming from the cache sites? Since they allow sites to sign up to become a cache server, wouldn't it be possible for a malicious user to sign up and use some locally-modified code to add a virus to all the .exe files that get sent out from their cache? They could even customize the output of their CGI depending on what domain you are in, making it easy to target specific sites and/or hi
half baked version of p2p (Score:3, Interesting)
Freecache is really just a half-baked ("precursor") version of P2P; not in any sense a long term solution, but interesting at least.
Correct use of P2P with network based caches (i.e., your ISP installs content caching throughout the network) and improved higher level protocols (i.e. web browsing actually runs across P2P protocols) would resolve slashdot effect type problems and usher in an age of transparent, ubiquities, long-lived, replicated content.
For example,
Basically, your request (and thousands of other slashdot readers requests) would fetch "closer" copies of content rather than having to reach directly to the end server (because, the content request [i.e. HTTP GET] actually splays itself out from your local node to find local and simultaneous sources, etc]. In theory, the end server would only deliver up one copy into the local ISP's content cache for transparent world-wide replication, and each end point would gradually drag replicated copies closer - meaning that subsequent co-located requests ride upon the back of prior ones. I'm just repeating the economics of P2P here
In additional to all of this, you'd still have places like the Internet Archive, because they would be "tremendously sized" content caches that do their best to suck up and permanently retain everything, just like it does now.
Physically locality would still be important: if I were a researcher doing mass data analysis / etc, then I'd be better of walking into the British Library and co-locating myself on high speed wi-fi or local gigabit (or whatever high speed standards we have in a couple of years time) to the archive rather than relying upon relatively slower broadband + WAN connections to my house or work place.
For example, say I'm doing some research on a type of flying bird and want to extract, process and analyse audiovisual data - this might be a lot of data to analyse.
Equally, places like the British Library will also have large clusters, so when I want in there to do this data analysis, I can make use of large scale co-located computing to help me with the task.
Nothing here is now: if you think about it, these are logical extensions of existing concepts and facilities.
http://www.archive.org/web/freecache.php (Score:2, Funny)
An error occurred while loading http://www.archive.org/web/freecache.php: Timeout on server Connection was to www.archive.org at port 80
Somehow I don't think this solution will work.
Yes, it's Slashdotted. (Score:4, Insightful)
But I don't think that it really is an indicator. I happen to have read the site yesterday after reading the Petabox [slashdot.org] article, so I think I have some of the basic concepts down. As I understand it, the idea works with cooperation from ISPs (and others) to provide more localized caches of large popular files. The motivation for the ISPs is that by providing the cache, they save on their upstream bandwidth and the associated costs.
So, while it's funny that we've slashdotted the archive.org server where the Freecache website is, Freecache itself is not dependant upon archive.org's bandwidth.
It's also worth noting that the concept is still in beta and pretty new - I don't think they've got a lot of ISPs on board yet. From what I can tell, it seems a very good concept - the only thing I can think of that I would want to make sure of if I were an ISP is that my cache is only available to users on my network (the whole saving on bandwidth usage argument falls apart if you suddenly become a cache for users on other ISPs) but I would think that would be pretty easy to do.
For those who haven't yet been able to read about it, here's Google's cache [google.com] of the front page.
Censored (Score:5, Interesting)
This would be great if my employer didn't restrict access to archive.org as allegedly being in the "sex" category.
ISPs are the problem... (Score:3, Interesting)
What ISPs should really do, is sell you a 256K internet connection (or whatever speed you happen to get), but then make all local content available at maximum line speeds... In other words, if you use the caching system (which saves the ISP money on the price of bandwidth) you get your files 6Xs as fast, or better in some cases.
I don't see why ISPs don't do that. It seems like everyone would win then. It wouldn't just need to be huge files either, they could have a Squid cache too, and not force people to use it via transparent proxy (most people would actually want to use it, despite the problems with proxy caches).
Right now, users have incentive not to use it. Mainly because it's another manual step for them, and to a less extent because caching systems usually have a few bugs to work out (stale files, incomplete files, etc).
I know that it would only require minor modifications to current DSL/Cable ISP's systems to accomplish the two zones with different bandwidth.
WILL Help slashdotting (Score:4, Insightful)
It's not perfect, it will certainly not be used by everyone. Still it's something you can do defensively, especially if you're serving mpegs of your latest case mod or bear attack or whatever.
-Zipwow
Re:WILL NOT Help slashdotting (Score:2)
Re:Won't help slashdotting (Score:2)
Re:Freec ache (Score:3, Funny)
Re:lets see. (Score:2)
Nothing can survive the power of slashdot it seems!
parent is Informative? (Score:3, Informative)
Site owner+freecache==good; /. +freecache==bad. (Score:2)
Re:Caching (Score:2)
There is already an answer [slashdot.org] in the FAQs. So, sorry, but can't be done.
Re:Not so good behind a corporate proxy (Score:2)
Of course, if he was half smart, he would just block URLs containing the url he wanted to block, as that would deal with everything.