AT&T Labs Backs Publius, A Freenet-Like System

joseph writes: "This article on C|Net announces Publius, a system similar to Freenet, meant to battle censorship on the Internet. What makes this approach interesting is its backing from AT&T Labs. Of particular interest in the article are the safeguards against the common opposition to such projects, like their use for piracy. Publius features no search utility and a maximum file size of 100k."

Re:Where is the benefit?

When you can get Pink Floyd's Echoes (a good 20+ minute song) in some format (perhaps mp5) and compressed to under 100k, then I'll sit up and take notice :)

Why? Why would you want to use something like Publius for that? If you already own the track, then encode it yourself. If you haven't, then you're only interested in pirating it, right? If you had a legitimate use (e.g., evaluating a band before deciding to buy it) you wouldn't need the full 20 minute epic, and a 2 minute sampler should suffice, and may even fit in the 100K limit at low quality. If you like it, go and buy the CD...

Too bad...

Wow, I'd love to use this system to publish the Publius PDF [nyu.edu]. That way people could anonymously learn how to set up their own censorship resistant networks.

Oh wait, the PDF is 233k...
--

Inaccurate portrayal of freenet

No, it is not useless. It is designed for people with a REAL reason for being anonymous, yet wanting to spread information. For example, whistlerblowers, or people in countries with a less than perfect track record of censorship.

Except that, without the ability to do searches, no one will be able to find the material in question. Giving out the precise key is tantamount to publishing, so anonymity is preserved at one level, but possibly compromised at another.

Furthermore, whistleblowers and the like often need audio-visual proof of what has happened, such as audio recordings (ideally compressed with ogg or mp3 format for space), images, and even video footage. How is one going to reasonably publish that kind of important evidence of wrongdoing with a 100K filesize limit? By breaking up the files into 100K chunks? Then why not get rid of that limit to begin with.

It is not designed for pirates who want their MP3's (go to freenet for that sort of stuff).

This is a very unfair characterization of freenet and downright slandorous.

Freenet is intended to do precisely the same thing as publius, with the exception that freenet make no judgement whatsoever about content. Publius may make use of some better algorithms, but has also clearly made policy choices which make it less than ideal for dissidents to skirt censorship (such as the lack of searchability and the filesize limit, and worse: a philosophy of passing judgement on material and what is "fit" to be protected from censorship and what is not, with who deciding such criteria an open question). FreeNet can always adopt better encryption and storage approaches now or in the future, without making the same kinds of misguided compromises.

FreeNet remains IMHO the most promising approach to thwarting censorship of all kinds, today and in the future.

Where is the benefit?

When you can get Pink Floyd's Echoes (a good 20+ minute song) in some format (perhaps mp5) and compressed to under 100k, then I'll sit up and take notice :)

Seriously, what does this offer over freenet aside from "let's make those whining children over at the RIAA happy" type restrictions? If it's the same as freenet, but with a strict set of rules, then freenet will eventually beat it no matter how much corporate money it has backing it.

Finkployd

Are they serious?

Let's test how serious they are by publishing a list of AT&T calling card numbers :)
--

Re:Interesting cryptography

Without getting into the actual mathematics, let me say that the system of having n shares and you need k of them to reconstruct the message has been worked out. This isn't pie-in-the-sky stuff, this actually works. And it's cool.

Here's a simple example. Suppose we have three servers that store information (n=3). We want to store a number on the servers such that each server individually doesn't know the number, but any two servers taken together are enough to reconstruct the number (k=2). The system we can use is to encode the number as an angle. Draw a line at that angle to horizontal, and choose three random points on the line. Send a single point to each server. Each server knows one point, but it can't figure out the angle of the line. But if you put the information from two servers together, you get two points which lets you draw the original line (and hence figure out the angle and get the information). Of course with bigger n and k you need real cryptographic systems (and not just lines).

No search?

AT&T also started funding my BlackHoleNet project. See, what you do is you send a file (less than 100K, so break that MP3's into 100 files!) and BlackHoleNet sends it to a special device (/dev/null). Later, when you want to get a file out it is retrieved from a different special device (/dev/random). The only remaining bug in my system is that the process of traversing the wormhole from /dev/null to /dev/random is somehow scrambling the files. I just need some funding to get over this last hurdle.
--

Let's not forget...

...that the most subversive file extension in the world is not MP3, or JPG or DivX. It's TXT - the plain old written word in electronic format. Ma Bell's service may be useless for exchanging large files but it could well be a very useful service for anyone who wants to be published and can't right now for whatever reason.

It's all in the words folks. The Chinese Government doesn't give a toss about its citizens downloading MP3s. It *does* care a lot about what they read...

limits

Limiting the file size to 100kB will drastically hurt this systems ability to support the freedom of speech. Unlike the days of the original Publius and the Federalist papers, not all speech today is, or can be, in the form of text.

Next time Will Smith gets a video of the NSA killing a Senator he will be able to upload it to Freenet. Will he be able to place it on Publius?

Does it say something about the sick influence of money in our world that they are willing to tolerate the usage of the system by child pornographers, but not by people who don't feel like giving money to the RIAA?

Re:Interesting cryptography

It's called an Information Dispersal Algorithm, or IDA.

See: http://www.acm.org/pubs/citations/journals/jacm/19 89-36-2/p335-rabin/

Interesting cryptography

If you go to the actual Publius site [nyu.edu] you'll see the information on how the system works. It kind of reminds me of steanography, in that technically all the servers appear to have is random data. The difference is the key retrieval method.

The publisher takes the key, K that is used to encrypt the file and splits it into n shares, such that any k of them can reproduce the original K, but k-1 give no hints as to the key. Each server receives the encrypted Publius content and one of the shares. At this point, the server has no idea what it is hosting -- it simply stores some random looking data. To browse content, a retriever must get the encrypted Publius content from some server and k of the shares. Mechanisms are in place to detect if the content has been tampered with. The publishing process produces a special URL that is used to recover the data and the shares.

I'm wondering just how that cryptography is implemented, whereby having less than n of n shares still permits us to read the document. The pdf on their site seems to involve MD5 hashes in the process, but I was wondering if someone more cryptographically inclined could elaborate. Of mathematical note, they generate d*ln(d) shares, where d is the number of servers. This has something to do with the coupon collector problem, and that if you check d*ln(d) servers you get to every "unique" server.

All in all it seems a really good system; hopefully the common carrier concept will be better applied. Since the pages can be retrieved with special (CGI based I think) URLs, they could probably be indexed by standart search engines such as Google [google.com]. I hope this works out

Lameness filter?

Anonymity may breed distrust, but if you're not anonymous, they sue you into the ground. Piss off someone with money, or a powerful lobby, or a big corporation, and you might as well be dead. They'll make sure no one hears you. This is the age of the frivilous lawsuit, where anyone wishing to silence someone else may do so simply by making it horribly expensive to exist.

Say I write an article, saying in effect that DB2 bites in comparison to Oracle. IBM doesn't like this, and sues me for slander, libel, and false claims harming their business. None of those claims are true, and are in fact laughable, but I still have to hire a laywer and spend huge amounts of money just to get the judge to not rule by default against me for a huge sum. In the mean time, no other publisher is going to hire me or take my works for fee because of the lawsuit. And what if the judge dismisses? Well, that doesn't happen. IBM drops the suit, and refiles next week.

Pretty soon I'm 20K in the hole, the article was pulled so no one ever saw it, and IBM offers to drop the suit if I retract my statements and only write 'the truth', as their marketing dept sees fit to spin it..

You know what? I'd do it too.

Re:Where is the benefit?

I suspect that they did so for precisely that reason: To keep out mp3s. The system was designed to be a safeguard for free speech; if someone wanted to speak out against $CAUSE then he could do so without fear of retribution by (corporation, government, cia, pick one). There already is a perfectly good system (several in fact) for trading mp3s, so these guys wanted to focus on issues that, dare I suggest it, matter more in life than music.

--

Re:Where is the benefit?

You're making an assumption that all MP3s are pirate. MP3 is highly effective for any audio data. While I will concede that it currently seems to be primarily used for piracy it's still just a form of media.

Besides, as I note in an earlier post, this is hardly going to stop anybody:

url 1: list of Publius URLs for file fragments 1-19
url 2 through 20 (listed in url 1): MP3 of "They Might Be Giants - Istanbul" in fragments.

Program a client to seek and re-assemble the MP3 based on url 1. It's merely inconvenient (to the point where freenet will probably darwinize this to obscurity) but hardly a showstopper for pirates.

Re:limits

You can split a large file into pieces and insert them separately. Then insert a list of the pieces as another file. Voila - large file support. The 100k limit is almost certainly there to avoid attacks which flood the system using bottomless data streams, or use large files to diplace a disproportionate number of smaller files. A similar limit will be used on Freenet, for similar reasons. It doesn't drastically hurt the system's ability to support freedom of speech; it protects it.

Previously on Slashdot (June 30)

Slashdot Article [slashdot.org] Lots of info.

--

Embodiment of truly free speech?

This is refreshing for 2 major reasons, it is actually backed by someone with major clout, namely AT&T.

Second, this system is out to provide a safehouse for truly free speech, and not to provide a safehouse for rampant piracy.

With the right development, I wouldn't be suprised if this could be developed into a robust information repository (Like the library in SnowCrash?) However, it will be interesting to see if it becomes popular because it won't attract those who are really just in it for the warez and mp3z.

-cpytel

Don't make this into another Napster!

Yeah, this story was already posted [slashdot.org]...

Of all the comments to this story, the ones that get to me are the ones complaining about the lack of MP3s. People, we have enough Napster-like clones out there, we don't need any more! File-sharing programs are a Good Thing(tm), but because of programs like Napster, Gnutella, and Freenet, people see this programs as only being useful to MP3 traders and warez kiddies. The great thing about Publius is that it implements true free speech, i.e. the sharing of sensitive and critical information without fear of reprisal, without worrying about MP3 traders and warez kiddies. And because of that, maybe some people will get turned on to the idea of file-sharing programs, and will see them as something more than another road for piracy. It'll be interesting to see how this program will function down the road, and I hope it continues to develop.

Remember, just because you can't download a copy of AutoCAD or a Britney Spears CD from it doesn't make Publius useless. There's plenty of Napster-like programs out there, don't make Publius into another one.

--

100 K fle size

Ha-ha... well, then, it's perfect for porn!

(Trade secrets, product rumors, & illegal device compatibility descriptions - like DeCSS - will fit nicely, too.)

--

Re:Where is the benefit?

The 100k limit will keep out MP3s, but it may also stop the system being usable as a free speech tool. Sure, you can express a lot in a 100k text file, but what about photos and videos ?

Given the power and importance of images (remember the Ethiopian famine, Tiananmen square, the Bosnian prisoner camps ?) a mechanism for distributing materials that's limited to 100k just won't work.

Finally a decent use of this technology

This will probably be considered a flame but I think it needs to be said.

Finally someone has produced a product that uses the technology Napster and Gnutella are based on to do something good. Despite what many proponents of Napster and Gnutella say, the main use of those programs is to get around having to pay for music and other electronic info like videos.

Some of the posts already on this board say "What is the use of this program?" Now we just have to split the mp3s into lots of little files. The people saying this apparently totally miss the purpose of this program. It is NOT to get music without paying for it. It is to help people communicate in an anonymous and secure way. Which many of the /. readers say it key to the internet. No doubt eventually someone will make a wrapper so that mp3s and such can be put on Publius but that is not really a good thing.

I think it is great that a major corporation has helped to produce a product like this. Especially one that goes against what many government agencies want, that being anonymous, encrypted communications across the internet.

Just my $.02

Rich

So . . .?

As an Internet publisher [totk.com] myself, I'm a bit miffed as to what's going on here. Sure, I publish stuff about sports, which is not exactly Earth-shattering in terms of societal impact (though I can argue for its societal worth, mind you), but what seems weird to me is that this is a way to hide behind the computer screen.

Looking at the root of the name of Publius -- familiar with the Federalist Papers myself, because I have to soon explain why we made all those changes in the UAH SGA [uah.edu] last year anyway -- I see their point, but societal change is more often brought about by grassroots efforts led by out-in-front, standard-bearing individuals.

To demonstrate my point, could the American Civil Rights movement have progressed without someone like Rev. Dr. Martin Luther King, Jr., marching? Sure, he could have sat in Atlanta (or Memphis, or Selma, or . . .) and written beautiful works on what was wrong with the oppression of "Negroes" in American society. I dare say his impact was strengthened by his visible action.

Heck, to take it to a whole other level, Jesus Christ himself could have just written a bunch of stuff, but I guarantee fewer people would be affected by Christianity -- whether you have a positive or negative view of it -- without some decisive action in there.

Anonymity breeds a small hair of distrust. If you're going to take over the world, you've got to have people's trust.

--
<><

Publius and Freenet

Ok, here's what it looks like to me. Publius is more secure in protecting the servers from the content they host; Publius has ability to maintain a "pseudonym" without outside software; Publius has the ability to hyperlink. Here's some elaboration:

Secure servers: It is publicly known who runs the servers, but it can be kept private what server has what. In order to download something, I need to know where to get the key shares. The server doesn't know that. Also, a server can't know what it's hosting without the ability to download it. However, things are less secure in that all this means that if I know how to get something I know who is hosting it, and a govt. etc. could use an attack based on such. So servers are both more and less secure.

The last two are really just based on the document format and software architecture.

This was all written without knowledge of the code, and is jst my interpretation of the web site.

---