Wikipedia Planning a DVD Version

daria42 writes "The Wikipedia Foundation hopes to sell an English version of Wikipedia on CD-ROM and DVD before the end of the year. A boxed set of the German language version of Wikipedia has been available since last year. An updated version of the German Wikipedia was launched on Amazon.de this week, and the e-commerce site has received 8,000 pre-orders, according to Wikipedia Foundation president Jimmy Wales. Wales said it was easier to put the German version of Wikipedia onto CD as there are significantly less pages than there are for the English language version. He said that English Wikipedia would 'barely fit on 2 DVDs.'"

Another reason the German version fits on CD

Frequent mentions of David Hassellhoff compress really well.

Re:Another reason the German version fits on CD

AndbecauseGermansdonotwastebytesonspaces

Re:Another reason the German version fits on CD

I believe what you meant was "Onderspacen Der Deutschennobegewastenderbytesen."

p

Re:Another reason the German version fits on CD

You could if you studied it for a minute. The rules are actually relatively simples. For example, let's look at everyone's favorite word: "Aktiengesellschaft", meaning a public corporation, and usually abbreviated "AG".

Semantically, it can be broken into Aktien / Gesell / -schaft; taken at this decomposed level it means something a little bit like "stock fellowship".

Orthographically, well, you're in luck; as usual (always?), it breaks down along the same lines, according to the rules. What are the rules? I couldn't tell you exactly, but they're simple, and they're similar to Latin's. Anyway, it breaks down to Ak/tien/ge/sell/schaft. Breaks occur between consonants that don't form clusters, between vowels that don't form diphthongs, and otherwise before consonants.

Re:Another reason the German version fits on CD

well, we just hate redundant information :D

There is no way to turn AndbecauseGermansdonotwastebytesonspaces into one single German word btw.

It is true that the German language allows combinations of nouns of arbitrary length, but in the English language constructs like this exist as well (e.g. railway consists of two nouns). The only difference is that those speaking English are not free to make up new ones.

And as a general rule of thumb most combinations in English are limited to two words. While it wouldn't make sense to combine more than 5 words, because it would get to hard to read and understand the term, there are rare examples in German which consist of 3 or even 4 words.

Re:Another reason the German version fits on CD

in the English language constructs like this exist as well (e.g. railway consists of two nouns). The only difference is that those speaking English are not free to make up new ones.

That's right. "Railway", for example, derives from "railuswayus" and was not concatenated by English speakers. We also didn't coin "email", "Internet", or "loudspeaker" - we borrowed those from Swahili.

I'd go into more detail, but I'm off to check my voicemail and weblog (Spanish).

Love it.

I love it when /. posts an article only for me to see 'Nothing to see here.' for a minute. On the Other Hand, Only TWO DVDS? I better get to Wiki and start writing pages all about the writing Fan Fiction!

Whaaa?

Last time I checked, the current version of the English wikipedia dump [wikimedia.org], is around 585MB. It should comfortably fit on one CD. Where did this figure of two DVDs come from?

Re:Whaaa?

Where did this figure of two DVDs come from?

Well, when you add in the theatrical trailers, "making of" featurette, production stills, and commentary tracks... What I want to know is, will it be in Dolby Digital 7.1?

Re:Whaaa?

The best part will be that the 7.1 sound will be put together from contributions by users just like you and me from all over the world!

The levels will be mostly ok, except for the sections where people have entries for themselves in which the dB level will be upped by 10 +-5. Also, the encoding will be completely and totally correct, except for a very small flaw which will cause the center right speaker to output everything in Latin.

Re:Whaaa?

I followed your link:

Last dump made: 2005-03-09 (30 days ago)
Total size 50503MB (1460MB for just current revisions)

These are SQL dumps of the current and old article revision databases for each wiki. They can be read into a local database and directly used with the MediaWiki software (MySQL, PHP, Apache required).

These dumps are not suitable for viewing in a web browser or text editor unless you do a little preprocessing on them first.

Re:Whaaa?

Yeah, I know, but...

1.5GB for current revisions would still fit on one DVD.

Also, that 1.5GB is for all languages. The English version only uses 0.5GB of that.

Re:Whaaa?

Not counting images and other media, yeah.

Re:Whaaa?

Exactly right - the media take up BY FAR the largest amount of space. Being that I do a lot of work putting full length songs onto Wikipedia (and I'm pretty much the only one who does), I've put well over 2 gigabytes onto commons in the last 6 weeks alone. See the list of songs [wikipedia.org] I've put up :)

Re:Whaaa?

Nope. Wikipedia is available over HTTP in a much more up-to-date, interactive and dynamic format than DVDs. The whole purpose of the DVD sets is... I don't know. I really don't. but why BitTorrent it when you can just point your browser at wikipedia.com?

Re:Whaaa?

Wikipedia is available over HTTP in a much more up-to-date, interactive and dynamic format than DVDs.

Well, yes, if you want to read it you're probably not going to download the entire bloody encyclopedia to your local machine via bittorrent.

But some people would have valid reasons for wanting this. A lot of places resyndicate Wikipedia content, e.g. www.thefreedictionary.com [thefreedictionary.com]. or answers.com [answers.com]; I'm exactly sure why these sites do it, but I can think or many valid reasons.

Maybe data miners or researchers want to run scripts on Wikipedia and make all kinds of conclusions (such things are entirely legal and above board, since the content is free).

The whole purpose of the DVD sets is... I don't know. I really don't.

Well, not all of us are connected to the Internet 24/7. Some of us have laptops without wireless Internet, and even computers without network cards at all.

Lastly, there are many places in the world where you can't get a reliable net connection at all (e.g. various places in Africa, Asia).

Re:Whaaa?

Well, not all of us are connected to the Internet 24/7.

What the hell's wrong with you?

There are...

...no images in the dump. Just text. And not reader software.

Also, the current dump is about 800 MG, gzipped. enjoy.

Re:Whaaa?

I bet people would like to be able to read it or even search it off of the DVDs, which means storing it in bz2 format on the DVD is probably a BAD idea... So yes it's only 585 megs when bzip2'd but that isn't a very friendly format to deal with.

Why?

Not that I hope multitudes don't buy this, because any support of such and effort is good, but why would anyone buy it when you can just look it up and possibly get better updated results online?

Re:Why?

First, to "lock in" decent versions of controversial articles. But second and more importantly, to be able to produce a stable, constant "edition" that can be referenced and cited to. How do you cite Wikipedia, when the content is always changing? Now you could write a paper and cite something like Person, Random, "Wikipedia Article," Wikipedia 2d ed. (2006). Very, very, important if WP is to become a legitimate source of information.

Re:Why?

And for those who care...

An MLA/APA auto formatter for references. [sourceforge.net]

Every teacher at my school has recommended it to me. (Although I myself have not yet gotten a chance to try it.)

Free?

Why pay for what you can get free?

But...

How will the trolls deface a read-only version of it?

why this is good

Because you gotta love it when people get paid while continuing to give you their stuff for free. Everybody wins.

2 DVDs?

Surely text compresses well. Not to mention that most of the linking between different articles could be handled in such a way as to reduce wiki-markup overhead.

Then again, there are a heck of a lot of articles in the Wikipedia...

Neat idea, but...

I think it's a good idea to have wikipedia available in other formats than just online, but isn't the whole point of it that anyone can come and edit the articles to make them more correct? You couldn't do that with a DVD version. And unless someone is going to go through every article before putting it on a disc, you'd run the risk of buying an encyclopedia with some things blatantly wrong. I could envision pranksters trying to sneak in false information just before the DVD release...

humm..

I thought the whole idea behind wikipedia was that it is constantly changing. Will updated dvds be sold? And if so, will previous buyers get a discount?

But if you remove all the useless stub articles...

The footprint of the english version fits on a floppy!

Mad dash to make "corrections" before it goes gold

You know how controversial subjects in the Wikipedia get fights over entries. Back and forth it goes, with one person putting their "truth" and then the opposite side removing or replacing it with their version of the "truth." Now, just picture it: The deadline for the gold master version to be put on disc is announced, and like people pouncing on an EBay auction at the last second, the warring factions will rapidly replace each other's versions of an article, hoping that their version is the one to be immortalized on disc.

Dead-tree version coming soon?

With Wikipedia taking up so much space on DVD, I certainly hope they compress the text. It should actually compress quite nicely, I think.

I wonder... does this 2-DVD set include all articles from Wikipedia? (As opposed to some just selected somehow...) Also, I wonder if the DVD version will include all the version changes to the articles. If not, then perhaps the best version was picked out somehow?

Hmmm... This is what I think needs to happen: Wait a few more years for Wikipedia to gain even more information, and then put some kind of button on pages that allows users to "vote" for that page to be included in a dead-tree encyclopedia version of Wikipedia. The idea is to put only those articles that have the highest votes into a traditional-style encyclopedia that can rival the likes of commercially made ones. Of course, there would need to be ways to cite sources, to make the encyclopedia worthy of academic research and the like, and preferably there should also be a way for people who want to do other stuff than write articles to submit photographs or whatever kind of artwork, of their own creation and released under the free license of Wikipedia, for inclusion in the articles. For the print version, people might be able to vote for the "best" photographs and artwork for inclusion. At that point, it should be a matter of running some perl script or something to typeset the whole darn thing. This might find its way into libraries and into peoples' homes. Imagine that!

Out of date already?

I like the fact that Wikipedia is always current, so I don't know why I'd ever want this on DVD. For example, they had a great article on the pope the other day which was current right up through his death. Since I can just look this up online, why would I want stale information stored on my computer? I have a set of World Books on the shelf, and we keep them around for when you want to do research when "otherwise occupied" (i.e. sitting on the can). Of course now that I have a Zaurus with wireless networking then I don't need the hardcopy any more since I can surf from any room in the house.

How fluid is Wikipedia?

How often do existing pages change? Maybe in a case where people catch errors.

I have a spare 20GB lying around that I would install this on, if there was some way to sync it with the current state and have it download new pages and update current ones.

School usage

This would be great for schools. They could buy the DVD set and set up a local "mirror" of Wikipedia to increase access speed and decrease Wikipedia bandwidth usage.

I think I speak for all the nerds here...

...when I say, "two single-layer DVDs, or dual-layer?"

No real point for me

Say what you will about Wikipedia's well know flaws (revert wars, submitters with thinly veiled political agendas and other various sub-vices) but part of the appeal of Wikipedia is simply the form of its current incarnation.

Right now, if I wanted background information on something, I just load up the (usually speedy) bookmark, enter the search and within seconds I have my answer. Why should it then become a more laborious to use resource? 2 DVDs? No only would constantly inserting and removing discs and launching extra applications provide more hassle (not too much more, but enough to make its value as an ultra speedy information center reduced) how would the information be sorted? Imagine half the links on any page requiring you to switch discs? For me, one of the great ways to use Wikipedia is to wander from article to article following the various text links. A multi-disc setup like this would only discourage that method.

Finally, I don't really think Wikipedia is ready to be put onto physical media for distribution. I certainly wouldn't trust it for more than satiating my curiosity, for instance I'd never cite it in an essay as a source. Articles with heavy disputes in their comments abound and many have no easy answers with how to solve an article's problems. So putting it on DVDs is a two fold problem, you don't get the advantage of having your data updated constantly by other users, but you'll also be working with a lot of flawed data that will be flawed forever on the disc. Perhaps I'm exagerating the problems a little, but really, who needs this? It's not like it's a cheapr printed encyclopedia alternativ to Brittanica, and since it'll be in electronic form anyway, why NOT use the web for it, you don't have to store any of it yourself.

The only audiance I can think of that would need this are Wikipedia addicts who spend a lot of time without internet access.

Sorry for being overly negative, I really love using Wikipedia, but I think this move kind of messes up the point...

Is this legal?

In order to publish and SELL this information on CD/DVD, does the Wikipedia Foundation have to get the permission of all the article writers, or is there, perhaps, a clause on the website that says something like 'we own all the stuff put on here'. What would happen if Slashdot sold versions of article comments on DVD?

Re:Is this legal?

Have a read [gnu.org].

Re:Is this legal?

Text content contributed to Wikipedia must be GFDL, so the foundation can sell it as long as they respect the authors' copyright and the terms of the license. Although the Wikimedia Foundation is not-for-profit, even commercial distribution would have been acceptable under the terms of the GFDL. But the content copyrights still belong to those who created it.

On the other hand, it happens that people contribute material copyrighted by other people, without their consent. According to U.S. law, Wikipedia cannot be held responsible for that, as long as they act quickly to remove infringing material. When physical media is distributed, that protection is no longer valid.

Re:Is this legal?

Wow, what a karma whore.

On the bottom of every single Wikipedia page, right there in plain sight, is a link to the GNU Free Documentation License [wikipedia.org], which governs everything submitted to Wikipedia.

p

wiki is going to get sued for this

A lot of vandals copy/paste text from copyrighted websites onto Wikipedia, usually they get found and deleted but some are missed. If they sell copies of Wikipedia then they are going to get tons of copyright infringement lawsuits.

Wikipedia Magazine... I'd pay for it!

Here's an idea I just dreamed up... It shouldn't be too hard or costly to do, but it might make the Wikipedia folks quite a lot of money, if it works:

On each Wikipedia article, there should be a button where users can vote an article as being "worthy" for academic research and the like. Articles that receive high votes would actually get published in a monthly (or even by-weekly) magazine... So, for example, each month, subscribers would receive the magazine in the mail, and it would contain, in addition to paid advertising like any other magazine, something like ten or fifteen articles randomly chosen from Wikipedia. These would cover a broad range of topics. One month, you might receive a magazine with articles about Argentina, transaxles, grep, electromagnetism, George Washington, the Berlin wall, Apollo 9, goldfish, ballpoint pens, and cow manure. Some subscribers will already be familiar with some of the topics; others might not be interested in some of the topics; but chances are that if you pick up this magazine and read it, even for a few minutes a month, you'll learn some interesting new facts here and there, usually about topics that you'd never consider reading about in any serious manner, but which you're reading because the Wikipedia Magazine happens to be there.

Links at the bottom of articles would direct the reader to the article online. This would serve an additional purpose: People who find something missing or something that could be improved in an article would perhaps be more likely to find out about it and then go online and fix it, thereby improving the quality of the entire Wikipedia.

Money from subscriptions; money from advertisers in all fields (not just technical, and perhaps based on the content of that month's magazine) would finance the magazine and help finance Wikipedia. I see this as an opportunity to make quite a profit on something that is free, while mainly benefiting the community by doing so.

Unless..

Honestly, I'm wondering how the CD will be like. Will they include an option for you to have the ability to update the content at your setting? Or will it be so that you have to buy a new version everytime? I'm thinking that they will include an option to update but you can also buy newer versions so that you don't have to spend time updating (I hope)

Hmm

How much will it cost? If it is more then ten bucks then I'll probably just warez it.

Where is the Great Publishing House of Ursa Minor?

You know, Wikipedia is ripe for a Hitchhiker's Guide to the Galaxy treatment.

Put it in a little handheld, stick an Ipod hard drive in it, give it a usb port so it can grab updates, and presto.

As for Wiki itself, "At least where it is inaccurate, it is definitively inaccurate." -Douglas Adams

The fine print

Just so we're clear, the article says that the majority of the price is going towards production costs and paying amazon. But if you're cheap, and really want a DVD set, then you can just download the images off "various websites", presumably to burn at your leisure.

It's hard to get a more friendly distribution method than that!

Possible updates?

Maybe the whole encyclopedia can be installed onto the hard drive, so the user can just choose to update topics as they view them, and also updates when they search for the topic. And such a system can be used online and offline. Oh, and they can also throw in local editing and uploading your updates.

Vandals

Although vandals are rare, it's not inconceivable that across their entire page set there would be at least one vandalised page. Kind of unfortunate if that gets included in the DVDs!

Anyone know if they have any way of stopping this?

Another good thing about this...

... maybe the zealots who use Wikipedia as their ideological battleground (e.g. this, [wikipedia.org] this, [wikipedia.org] or this [wikipedia.org]) can host their own wikipediae, with their own versions of The Truth, and thus the revision wars on the original Wikipedia will stop.

Or not.

BONUS FEATURES!

Like any good DVD, expect the Wikipedia to be chock-full of bonus features. Among the highlights:

• Dolby Digital Surround Sound
• 1337 5u8717135
• EBG13 Fhogvgyrf
• TourettesMOTHERFUCKINGBITCHCOCK subSHITMONGERtitles
• A short music video in which Jimbo breaks into his famous freestyle diss of Brittannica
• Re-enactments of some of the best NPOV disputes... with puppets!
• And BJAODN [wikipedia.org], of course

And expect a 25-disc Collector's Edition in about 6 months, with the addition of:

• The complete article talk pages of articles on touchy subjects like abortion, Iraq, circumcision, George W. Bush, homosexuality, and Why Canada Sucks. Invariably, the amount of text generated by the endless bickering over article content is an order of magnitude greater than the article itself... hence the 23 extra DVDs

Guide

They should publish this in the form of a mobile device with the cover saying "DONT PANIC". It would be as cool as hell, and quite useful on certain ocasions when a laptop is too large, or you dont want to risk it being stolen. Anyone thought of making their iPod (mabey its too small) into The Guide?

waiting at the door

I'm *still* waiting for the encyclopedia guy to come around to my house.... he hasnt been around for a good 15 years!

A bit of history on this

I first heard about this back in July of 2004. The people at Mandrake had already approached some of our people, and told us they wanted to put Wikipedia on DVD. The stumbling block was, of course, copyright issues. We launched a copyright tagging project in August - basically, they did an sql dump of the list of all uploaded files that had no copyright tag and tagged them. In January, Angela sent them an email, telling them it was done, and that's when the DVD project actually started.

Stupid Idea

What's the point? Wikipedia is an inherently online medium. The articles change daily, new ones are created, etc. This cannot be reasonably placed on a static medium.

The German version is smaller because...

Deutche has an amazing built-in fractal encoding scheme. For example, the German version may say:

Gerflugenichterschweitzenbaggen.

whereas the English version has to write out:

Shortly after September 11, 2001, the United States attempted to rally its allies for a strike against the presumed Al-Queda stronghold in Afghanistan.

Unfortunately, the RAR algorithm averages a 3% compression ratio on German text, in comparison to 82% for English and 94% for French - it's like bzipping a .gz file. On the other hand, there are significant savings due to the lack of entries on "sweet nothings", "pillow talk", and "Bavarian romantic verse".

Make it an appliance

self upgrading... and of course, based on GNU/Linux :)

Are you all really *that* connected?

I think you're all missing the point. I'd love to kick back with Wikipedia during a long plane trip and depth-first traverse myself into oblivion. I have friends who live in rural areas with bad phone lines who might still like to read a few articles. A lot of restaurants in my town still don't have wifi, so I can't browse during lunch.

I can think of a million and one reasons why having a fixed version that is instantly available would be a very handy thing indeed. I have all the Internet connectivity I could want (short of a neural interface), but I'd still probably shell out a few bucks for a copy.

How to help out

The main problem facing Wikipedia at the moment is lack of development resource. At FOSDEM last month Wales urged the assembled audience of open source developers to get involved with the work of the foundation.

The article doesn't mention how you can help out. Maybe might want to check this out:
http://meta.wikimedia.org/wiki/Help:Contents%23For _MediaWiki_hackers [wikimedia.org]

Good, Bad, it is what it is.

Althought I think actually USING the DVD set for normal use when you have broadband kinda defeats the purpose, I can think of a few reasons why it could be a good thing.

A) Archival. Average users will be able to get a working, usable snapshot of Wikipedia, with media.

B) Preservation. If Wikipedia were to shut down, you'd have a copy of it.

C) Faster access. If you have a slow connection, you can still access Wikipedia at fast speeds. This benefit dwindles over time as articles are updated.

D) Offline access. If you're on the road with no net connection, you can still access Wikipedia. This benefit also dwindles over time as articles are updated.

E) Although backed by Google now which helps with the financials, if it brings in some cash to help support itself it's likely to stay around for much longer.

Here's a question:

How are they going to get a Snapshot of Wikipedia in which there is no vandalism in any of the articles? It's ok to let people download Wikipedia when a few articles have been been vandalized, but to sell it in stores like that? They might have to work out a system where they copy the whole thing, put it on another page, and allow access only to admins, who will revert any vandalism they see, before having the whole thing ready for sale.

Re:Here's a question:

How are they going to get a Snapshot of Wikipedia in which there is no vandalism in any of the articles?

It's called "editing".
You've been reading slashdot too much.

Wiki* in Plucker handheld formats

I've been working on the Wikipedia [wikipedia.org], Wikiquote [wikiquote.org], Wiktionary [wiktionary.org] and other similar works to convert them to Palm handheld formats (primarily Plucker [plkr.org] format, but now iSilo for those users as well, with less functionality in iSilo, of course). I did a lot of work to the core Mediawiki [sourceforge.net] software that drives it, to make it more usable on handheld devices.

You can see my work so far at the following links:

Wikipedia in Plucker format [plkr.org]
Wikiquote in Plucker format [plkr.org]
Wikitionary in Plucker format [plkr.org]

..and of course, my beautiful anti-alias fonts for Plucker [plkr.org], made with PalmFontConv [sourceforge.net] by Alexander Pruss [palmgear.com].

I've also converted the Creating XPCOM Components [mozilla.org] book by Doug Turner and Ian Oeschger to Plucker format [plkr.org] as well as the FreeBSD Handbook [plkr.org].

I have literally hundreds of similar-quality works I'll be releasing over the next few months to the community on an ongoing basis.

If there's something you'd like to see, just let me know [mailto]

I hope they are careful about rights

I hope the take the history of Mathworld [wolfram.com] as a warning as what can happen in the publishing world.

DVD? Pffft...

Recursive wget's are soooo much easier.

partioning for two DVDs

If the content would only fit on 2 DVDs (mostly due to media other than text I suppose) I wonder how will they solve the problem of minimizing the number of times users have to change DVDs? Do algorithms exist that will solve this, in reasonable time on reasonable hardware, for something the size of en.wikipedia?

I know I for one would find it incredibly annoying if it turns out I would need to play DJ a lot to go from "Gautama" to "India" to "Bhopal", say, which is precisely the kind of loosely coupled chain I find myself browsing for the fun of it. The links go everywhere, and I would argue the topology is at least as valuable as the content itself..

I expect you could isolate some "clouds" of articles mostly referring to each other, but anyone want to guess as to the percentage of "cross-DVD" links?

Wikipedia is one of the best human 'inventions'

More and more I caught myself finding the answer I need in Wikipedia. I think that all human knowledge can be finally encoded in the medium called 'internet', in Wikipedia form...thus allowing unlimited access from schools and other institutions. It would really be a very helpful tool for teaching, along with the classic ways.

Wikimedia Foundation

This is about the Wikimedia Foundation [wikimedia.org], not Wikipedia Foundation which doesn't exist.
Both the article and the /. post are wrong.

Re:What's their point?

German people dont necessarily speak english and vice versa. And two DVD's is a lot of space. And 4 cd's isnt even a DVD. And just because other people use multiple disks doesnt mean its a good idea. I remember playing riven and having to switch disks way way to often. And for a Encyclopedia there would be a 50% chance that you would have to switch disks everytime you looked something up. That would be rather annoying when trying to do any research.

Re:What's their point?

Is German smaller in code than English, it not, then what is causing the extra space???

The English version is more comprehensive with more articles, largely because it has more people working on it.

Re:See wikioncd

Indeed; this is what I mentioned up above. It can be done, and it can be done reasonably well in a cross-platform app.

Re:Nice one

Hey, you implicitly agreed to this when you decided to contribute to Wikipedia, fully agreeing that your work was licensed under the GNU FDL.

Re:What's their point?

1. The code is the same size, but it doesn't matter since the space the software would take up on a CD/DVD/... is dwarfed by the size of the data.

2. The English Wikipedia has (roughly) about 2,5 times as many articles as the German Wikipedia. Mean article size (in bytes) is roughly equivalent.

3. The English Wikipedia uses many more images etc. than the German Wikipedia.