The Knol Hypothesis

Frequent Slashdot contributor Bennett Haselton sends in his latest, which begins like this and continues behind the link. "When Google's VP of Engineering announced their proposed Knol project, where users can submit articles on different subjects and share in the AdSense revenue from the article pages, he didn't mention "Wikipedia," but practically everyone else did who blogged about it. Here's what I think will happen, if Knol is implemented according to the plan: Even though it won't technically be a "Wikipedia fork," it will quickly become equivalent to one, with a "gold rush" of users copying content from Wikipedia to Knol articles hoping for a piece of the AdSense dollars. But I submit this will be a good thing, especially if bona fide experts in different fields join the gold rush as well and start signing their names to articles that they've vetted."

First, I've been saying for a while that someone should fork Wikipedia and start assigning "ownership" of articles to credentialed experts where possible, so that an article can be cited as a source that has been vetted by a recognized individual, and to guard against vandalism. Citizendium does something like this, but started from the ground up rather than fork Wikipedia. I argued that they should fork as much as possible from Wikipedia (having experts "bless" the content in the process); the project's official reason for not doing this was that authors are more motivated when starting with a clean slate than when taking over someone else's article. True, we all know the energizing feeling of a clean slate compared to the sluggish feeling of taking over a 50%-completed project with all of its flaws and compromises, but the "energizing feeling" often doesn't make up for the advantages of having 50% of the work already done for you (which is, in a nutshell, the only reason people ever finish 50%-completed projects instead of starting over!).

So could some other Wikipedia fork achieve the same thing? Programmer/blogger and Guardian columnist Seth Finkelstein, a frequent Wikipedia critic, has pointed out that other sites such as http://veropedia.com/ have tried to build a "verified" version of Wikipedia. "But," he writes, "it doesn't work for many reasons:

1) Maintenance
2) Nobody knows the site exists, or uses it.
3) Google will kill the site's ranking, because of "duplicate content"
4) Roughly 99% of Wikipedia's value is the Google-rank it has, and sites trying to copy its content don't have — or get — that Google-rank.

All true. But Knol has a shot at solving all of these problems. #1 should be mitigated if users earn money for maintaining articles — and besides, many articles like "Abraham Lincoln" won't need much maintenance anyway. #2 should not be an issue since it's a Google project. #3 and #4 depend on how Google lists Knol pages in its search results. The VP's blog post says only,

"Our job in Search Quality will be to rank the knols appropriately when they appear in Google search results. We are quite experienced with ranking web pages, and we feel confident that we will be up to the challenge."

Of course the question on everyone's minds, not answered directly by those sentences, is whether Knol pages will get any special treatment in search rankings. Google would probably be criticized if they manipulated the results outright. But they might achieve the same result indirectly — for example, having a tab across the top of their search results page for "Knol results," along with the tabs for Web, images, and news. Or if Knol results get killed for "duplicate content," Google might (legitimately) consider this a bug and tweak their duplicate-detection algorithm. Thus Knol would have the same advantage in Google that Microsoft's Media Player has on Windows: The operating system doesn't favor Media Player directly, but compatibility problems with Media Player will always get fixed first (while the RealPlayer people have to watch their programs get broken by Windows upgrades). One way or another, it's pretty certain that Knol results are not going to be "unfindable" on Google.

Now, I'm sure Knol will not formally fork Wikipedia. I wouldn't see any problem with them doing that, but it would be too controversial, after the VP announced it without ever mentioning "Wikipedia," and with Google already dealing with speculation that they're only creating Knol to complete with Wikipedia in their own search results. But with users having cash incentives to copy content from Wikipedia, probably most of the content would get replicated very quickly, and I would be surprised if many users didn't start writing scripts to robo-copy as much content from Wikipedia as possible.

Then you get to the point where experts start improving it. If the first couple of entries on "Physics" are just the robo-copied Wikipedia version, "signed" by users that nobody has ever heard of, this is barely an improvement over the unsigned article on Wikipedia itself. But then only one Physics professor in the entire world has to think it's worth their while to read the standard Wikipedia article, make any necessary corrections, and sign their name to it on Knol — and now you have a version that has been vetted by a credentialed expert, increasing its value many times to people who want to cite it as a source, or who want a higher degree of confidence that it's accurate. (Hopefully Knol will allow authors to confirm their e-mail addresses and display them — in an image, presumably, to stop them being scraped by spammers. This will allow professors to prove that they really have faculty .edu addresses and enhance the credibility of their articles, something I suggested for Citizendium.)

So, some criticisms of Wikipedia would not apply to Knol. Author Nick Carr has written of Wikipedia,

"Certainly, it's useful - I regularly consult it to get a quick gloss on a subject. But at a factual level it's unreliable, and the writing is often appalling. I wouldn't depend on it as a source, and I certainly wouldn't recommend it to a student writing a research paper."

When I asked if he would recommend Knol for the same purpose, he was more optimistic:

"Probably. Since a Knol would be written by an identifiable person at an identifiable point in time, I don't see why you wouldn't treat it, in doing research, in a similar way that you'd treat, say, an article by that person. Obviously, you'd need to judge the writer's expertise and authority when deciding whether or not to draw on his or her work, which becomes somewhat more problematic where no editorial or peer-review system applies, as in Knol."

This is where I think the value of a professor's .edu e-mail address comes in, which can at least establish a writer's authority in their subject. I asked Seth Finkelstein whether he would recommend Knol in those same circumstances (verified professor's .edu address, etc.), and his take was, "Of course I would, but you loaded the question in a way so as to remove any problem from it." Well, yeah. I just happen to think Knol actually could remove those problems.

Then there was a little-noticed phrase in Google's blog post that suggests another area where Knol could improve over Wikipedia: the inclusion of "how-to-fix-it instructions." Given that people often need how-to instructions a lot more badly than they need encyclopedia articles, it's surprising that there hasn't been an attempt to standardize around a "Wikipedia of how-to's." Perhaps it's because the Web itself actually does pretty well for that — type in the text of some error message, and you'll usually get some hits on support forums where people ran into the same problem. The trouble is that the ranking of search results depends on the popularity of the site, not on whether the thread ended with someone posting a solution to the problem, so you might have to read through a lot of search results to find an answer. And if you're an expert who happens to know how to do or fix something, there's not much incentive for you to post a page about it (even with AdSense ads), because your page will get buried in the search results beneath all the support forums discussing the same question, even if your post is more concise and useful. Some gurus like Dave Taylor and Leo Notenboom have written so many how-to articles that their own sites have risen through the Google rankings, so if they write a how-to article about something, it will get read (which, of course, creates an incentive for them to write more of them). But for a new expert just starting to write how-to articles, it would take a long time to reach that critical mass.

Knol, however, creates an incentive for experts to start posting how-to-fix-it advice and start reaping the rewards right away, since your how-to articles are just as easy to find under a given subject as anyone else's. Your earnings would start out small as you began to write articles, but they would rise in proportion to the number of articles you wrote, and you wouldn't have to slog along writing for no reward like a typical blogger or site creator, hoping to hit "critical mass" some day. You'd find out early on what the reward would be (financial and otherwise) for the work you were doing, and could decide if you wanted to continue.

Actually, the possibility of "instant rewards" does depend on how Knol articles are ranked against each other within a given topic. The Google blog post says, "For many topics, there will likely be competing knols on the same subject," and, "Knols will include strong community tools. People will be able to submit comments, questions, edits, additional content, and so on. Anyone will be able to rate a knol or write a review of it." Presumably the top-rated articles on a given topic will be displayed first by default, so I'm making the assumption that good articles really will get sorted to the top. But if there are already 50 articles on some topic, even if you know you can write a better one, how do you know it would rise to the top of the pile? If 10 people rate your article a 9, and all the other articles have been rated by 100 people each and got an average rating of a 7, then yours should still be listed as the highest in terms of average rating. But how do you get even those first 10 people to see your article? You could invite your friends, but then how do you stop anyone from gaming the system by inviting their friends and asking them all to rate their article a 10?

I'd written about this in the context of whether Wikia Search might try to solve these problems by allowing users to vote on search results, if they could prevent people from gaming the system. That was basically just me thinking out loud that something like that would be cool, before Wikia Search announced any specifics, and I haven't heard that Wikia is trying anything like that. But now that Google has stated that they will use voting and ranking systems in Knol, the question is how to reward new authors while preventing cheating. I suggested some ideas in an article about how to stop cheating and vote-buying on digg. One idea was that you could have a section on the page that showed people links to different articles at random, so that users couldn't self-select on what articles were shown to them, and if those randomly selected users followed the links and voted on the articles, count only those votes in determining the true "rating" of an article. (This is what HotOrNot does for people's picture ratings.) Even if few people rated those articles, the ratings that were collected, would be representative of real users, and not the horde of friends that you'd sent to rate your article. If that did not prove popular enough, then you could give authors the option to pay random users to rate their articles — as long as there was no way for authors to tie payment to higher ratings, the ratings would average out to reflect the article quality, and could be used to sort articles based on their actual merits.

So, I'd like to think that someone in the Googleplex is reading everything I write, but it's probably just a case where great nerds think alike. I wrote in Feb 07 that I thought Citizendium should allow authors to put their name next to articles, both for the "name up in lights" incentives factor and to enhance the article's credibility, and now Knol is going to do that (not to mention throwing in money as well). The same month I wrote that someone should build a search engine that groups together user-submitted articles under different topics, and provides a means for newly submitted articles to rise through the ranks as a result of user votes, and it sounds like Knol will attempt that too. Then in April 07 I wrote about the ways that you could prevent cheating in such a system, and even though Knol hasn't talked about what they will do to address that problem, they're almost certainly thinking about it, and have probably come up with some of the same ideas.

So let's do a test to find out if Google is reading these articles. There's one area where Wikipedia would beat Knol, and that is that everything on Wikipedia can be redistributed for free. That's something really special, and it's the one part of the Wikipedia hype that I actually buy into. I don't really care that Wikipedia articles were created as part of a "worldwide collaborative effort" unless that helps to achieve the goal of being useful. But Wikipedia, for all its flaws, represents the first time in human history that we have a compendium of a huge amount of human knowledge that can be copied freely, that literally belongs to the world, and because it's duplicated in so many places, it can literally never be taken away. That part of the hype really is true, and is quite heady when you think about it.

Google Knol has not declared this as one of their goals; a Knol article might not be freely distributable. When a proprietary project is hosted on a private site, there's always the risk that the company will pull the plug on it. They probably won't pull off the content offline, but they might shut the service down to stop new content from being added, the way Google did with Google Answers. Yes, Knol authors will retain ownership of their writings, so they could try to regroup and continue the project somewhere else, but it would be a huge mess to try and contact all of the authors and get their permission to copy all of their articles to the new location. As currently planned, Knol doesn't "belong the world," and Google never promised not to take it away.

So, I think that Google Knol should include a feature whereby authors can flag their articles as being freely distributable under the same terms as Wikipedia articles. (Any author who copied an article from Wikipedia and submitted it, would be required to set this flag, because under the terms of "copyleft", you can't copy something that's freely copyable and then try to stop others from copying it!) Then a user who wanted a copylefted, freely distributable article, could limit their search to articles that have this flag set. This would give Knol the best of both worlds: if the author of the top-ranked article did not wish for it to be freely redistributable, then they wouldn't put it on Wikipedia, but they could make it available on Knol, and users could choose either the top-ranked copylefted article or the top-ranked article overall, depending on what they wanted. If the best article on a given subject also happened to be flagged freely distributable, then so much the better.

Maybe the Knol people have had this idea already. But even so, if they end up implementing it, then I'm starting right away on articles about how Google should implement Google Anti-Censorware, Google Site Hijack Prevention, Google Security Compensation, and Google Sergey And Larry Give Bennett Their Airplane.

missed something:

The license for wikipedia content is similar to the BSD code license in that you can't add any new restrictions on the content. What do you think the odds are of google letting that fly?

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Regards,

Re:

They can't keep people from modifying the content and redistributing it. However, they are in no way obligated to keep people from modifying the local copy of the content on their own servers. You can publish Wikipedia content on something that's not a wiki, in other words.

Violates the GFDL

Anyone can copy Wikipedia, but like the GPL, the result remains open. So anything copied from Wikipedia into Knol, and anything derived therefrom, remains freely copyable, regardless of any terms Google may seek to impose..

Possibility for abuse

People will start edit the content for it being very attractive in some sense. Sensational claims, spicy personal details about personalities, etc...

The whole Web is now such a Wikipedia, where people give content and try to earn money for people visiting. The only thing left is to mark certain pages on the Internet as "definition" page on particular subject and voila.

Re:

If the money is worth it enough, you can definitely see Knol being the new "domain buying" where people snatch up ALL the variations of the possible directory names so that any competitor will have to think of a longer directory name or one not quite as specifically on topic in order for it to rank (think Squidoo).

Sure, plant the seen of the idea.

and I would be surprised if many users didn't start writing scripts to robo-copy as much content from Wikipedia as possible.

They will now!

Re:

This is actually a perfect opportunity for this to be fair. If anyone can write scripts to mass-post Wikipedia to Knol, presumably Wikipedia itself could. Let them survive, and get compensation, but posting their own articles-- either the main maintainers of those articles, or Wikimedia themselves.

big holes in your idea.

all college students, faculty and staff had .edu addresses, as do many other indidivuals that work for places like Hospitals or Doctors offices that are actually owned by a university. Just in the town I work for there are 3 colleges, and two hospitals that are owned by one of those colleges. That's literally thousands of people with .edu email addresses that are not professors.

Re:

Just about every university not in the US doesn't use .edu top-level domain, but rather the local country code top-level domain.

I submit that this is a bad thing

Not in the concept of having another competitor (good), but that Google's search results will increasing become Google products and NOT actually what's around the web.

This is already slowly happening and I find the increasing number of spots taken up by Google in the search results to be particularly bad for their brand, search results, and image.

I would definitely not like to see a search result on "jumping beans" filled with 1x Google Image Search, 2x Youtube/Google Videos, 2x Google Knol, 1x Google Products, 2x Google News, 2x Google Base.

Re:

Sure beats pages on pages of link farms and ebay referrers, like it's often the case.

Yawn

There are way too many news items on this (and every other) news site about fascinating, exciting or terrible things that MIGHT happen. Maybe. If the stars are right.

Isn't news supposed to be about what DID happen, or inevitably WILL happen?

And if you distill this significantly smaller portion of the news down to that which is actually relevant to the reader...that is, things that may truly affect them that they am not already keenly aware of (no point in telling someone about the rain if they're already wet)...then I think the news could be neatly summed up over dinner on one of those little slips of paper found in a fortune cookie.

In fact, that would be a good place for this news.

Wrong Approach?

The problem of article citation in Wikipedia isn't due to the fact that anyone can edit, more so the fact that it's hard to trace [virgil.gr] who is saying what and why. Attributing editorial control of a 'knol' to an expert introduces a bottleneck which, as the article states, may be fine for those wanting to learn about Abe Lincoln but which cannot scale as dynamically as the free-for-almost-all Wikipedia approach.

I think a better approach would be atomicize knowledge (e.g. "Hugh Hefner shaved his beard on 02/10/08"), make that source a verifiable resource attributable to individual users and then attribute adsense payouts dependent on page counts for ($num_verified_references - $num_unsubstantiated_rumours). You could then retain the successful wiki model for article construction but with greater trust for the facts contained. But then I, sadly, don't work for Google and I'm probably missing something significant.

Snow Crash

For those who have read the book Snow Crash, which i would recommend, this sounds a lot like the information brokerage system there. Everyone uploads useful data and then people who use it pay money, except the people paying money are the advertisers targetting the people using the data.

Who has the copyright to the work on Knol?

So for original work, who would have the copyright for Knol? What would stop Google from taking the work and start charging access fees etc. I'm not saying they would, but it would be good to understand what kind of licence(s) would exist on the material, and who maintains the copyright on work.

Vulnerable because of adsense

I don't think it will fly because it provides incentive for abuse. Some people will want to make a business off the adsense revenue. At best Google can only move reactively to maximize customer revenues through providing maximal value. However for people interested in profit, the system will be explored to give maximum customer revenue through providing minimal value. Bring on the arms race with the spam and hired guns to rate it up. Wikipedia isn't that bad an idea.

Recent developments regarding Abraham Lincoln

articles like "Abraham Lincoln" won't need much maintenance anyway

He just died [wcbstv.com] last year!

Brand dilution continues

The only interesting thing about Knol is that it is a new low in Google's continuing brand dilution. Although Google has already spewed out an incoherent product mix, they had previously still retained at least this minor degree of focus: we index and monetize content; we do not create content With Knol, the final clear line in making Google a distinct brand is fuzzed. They don't create content, well, sorta, maybe, they "manage" the creation of content. Of course, you could argue this line was already crossed when they went into the blogging business.

Google now has a full-blown case of the Microsoft Business Disaster Model. This model goes like this:

• Get a highly profitable monopoly.
• Watch gigantic sums of cash accumulate.
• Panic at the thought of actually distributing that cash to shareholders, as the law requires.
• Start throwing money at any additional product line you can think of, believing that because you got that first profitable monopoly (largely by luck), you are Really Smart, and therefore you can make money at anything.
• Watch with relief as stockholders don't notice how much of their money you are shoveling into the fire, because your core monopoly is still making huge profits.
• Spend years telling yourself that having divisions that lose gigantic sums of money for years means you are now a "long term" strategist.
• Drift slowly into decay like the Soviet Union, still powerful, still important, but internally depressing, wasteful, and decrepit.

The most profitable company this year was Exxon-Mobil. A company that has to get its hands dirty and actually move a physical product had higher profits than Microsoft, a company that just thinks up bits that it then distributes, largely electronically. Imagine the profits if Microsoft were to sell off all its huge money losers, retain only enough employees to maintain Windows and Office, and pay out all the profits as dividends. It would be the most incredible stock the market had ever seen.

Re:

That's an interesting concept and I hope you get some mod points, even if I'm not 100% convinced.

Google is scary right now; maybe even scarier than mid-90s Microsoft. I remember my more-business-side oriented coworker reading some article along the lines of "Microsoft Wins", that they could do or buy anything any smaller company started, so it was Game Over Man Game Over.

But I see Google being a lot more flexible in what it tries. While Microsoft became a hegemony of the desktop, Google keeps most of its in

Re:

Panic at the thought of actually distributing that cash to shareholders, as the law requires.

I'm not sure what law you think requires this -- presumably investors are paying Google's exorbitant P/E because they expect the company to plow their cash into some Magical Free Lunch-Powered GScheme, not because they expect it back in cash tomorrow. Otherwise they'd be buying a utility stock, or T-bills.

The most profitable company this year...Imagine the profits...It would be the most incredible stock the market

double efforts

There's already a Wikipedia fork which focuses on expert opinion and reliability - Citizendium [citizendium.org].

Doesn't have the big "Google" name behind it, and not very many articles, but for all I care, it's Wikipedia done right. Ever since the deletionism and notability nazis have taken over Wikipedia, I'm kind of disillusioned about it.

Re:

Different. A one-dimensional assessment such as "laxer" doesn't do it justice.

One, it does have an intermediate step, "Cold Storage", where articles are moved instead of deleted. The problem with deletion is that it removes everything about an article, including the metadata (history, etc.). That's the exact opposite of what a Wiki is all about.

The notability criterium is a pretty stupid rule anyways, as the massive discussions around most notability deletions show clearly. When the subject is discussion co

Wikipedia copyright violations

The official policy of Wikipedia is that copying material from copyrighted sources is copyright infringement, and the offending material should be quickly removed. I've seen an entire article deleted outright because of such an infraction. If you see copyright infringement in Wikipedia, follow the directions about dealing with copyright violations [wikipedia.org]. If you've seen copyright violations on Wikipedia, and you have not followed those instructions, it would be more appropriate to say that you have let it slide. Don't blame it on someone else, as you don't know if anyone else has noticed the violation.

Re:Wikipedia plagiarism

Don't know how parent got modded as insightful.

A lot of WP articles have been copied from websites (or textbooks). In traditional publishing, this would be a serious transgression - but in WP lets it slide.

Show us the money. List the pages here - better yet, go to the pages, remove the content, and post on the talk page showing the original source to indicate it is copyright. If it is a whole article then stick the {{copyvio}} tag on it and delete the content - the page will get deleted. Your comments are not just wrong - they are plain wrong - wikipedia doesn't have a tolerance for copyright violations, even to the extent that "fair use" claims on images are very tight and open to challenge at any moment.

The advantage that WP has in this regard is that its authors are anonymous - they can not be prosecuted for copyright infringement.

That would be also a lot of junk. There have been a number of cases where people have posted (what prosecutors argued was libellous) material on wikipedia - the authors, whether IP addresses or registered users were identified through the correct legal means and cases were brought to court. You are anonymous virtually nowhere on the net - and indeed even less so on a high profile site like wikipedia. If I post libellous comments on some backwater website about someone it is unlikely to ever be drawn to the attention of the person - it I do it on a biography page on wikipedia it will not only be noticed, but will be openly logged. Copyvio's are exactly the same - if the copyright owners wish to identify any material then they are not only free to do so, but wikipedia actively encourages it by making the methods clear. In reality, such as with the book on Oil resources published in the summer (full of entire passages lifted from wikipedia that was not referenced) the reveres is more likely true these days.

Wikipedia is no saint, and there are plenty of problems with regard to inherent bias, POV campaigns, unreliable information and cabal editing - but the one thing that can't be levelled at it as you have done is that it is some "nobody cares about copyright" site, quite the opposite is the case.

Re:

At the risk of committing a self-referential act:

The thing that hath been, it is that which shall be; and that which is done is that which shall be done: and there is no new thing under the sun. (Eccl 1:9)

This was never more true than in the case of software patents.