Wikia Search Engine to be Launched on January 7th

cagnol writes "The Washington Post reports that Jimmy Wales, the founder of online encyclopedia Wikipedia, has announced the launch of a new open-source search engine, Wikia Search, on January 7th, 2008. The project will allow the community to help rank search results, in a model close to Wikipedia. However the company is a for-profit organization. This new search is supposed to challenge Google and Yahoo."

Re:Easily Abused?

[jwales]

The question of abuse is obviously one that we are taking very seriously in thinking about design issues. My belief is that the key to solving this thorny question is hinted at by the success of wikis and the wiki model: the key is to put tools in the hands of the community that allow for broad oversight and control by the community in a process of open dialogue and discussion. This is very different from approaches that allow only for atomistic participation by a "community" which is never allowed to really become a community due to excessive reliance on algorithmic voting systems and similar.

One of the first lines of defense in the early days will be use of a community (wiki) generated whitelist [wikia.com] of sites to crawl. We will want to work outward from there, but basically the first thing is for us to assess "look, what are the most important must-have sites on the net" and crawl them. One thing that the mainstream media never seems to report very well, mostly because I think they don't get why it is important, is that we are doing everything here under free licenses. The software GPL, the data we generate under free licenses, etc. The aim here is not just to create a good search engine, but to create it and *give it all away* in a way that I think has a chance to restructure the entire search industry. Well, maybe not, maybe so, but what the hell, it'll be fun to see. :-)

Re:Challenging Google?

[jwales]

No, it is no response to Knol. I have been working on this for a year. The press has talked about it endlessly. :-)

It'd be sort of cool if we could create a search engine in a week or two to respond to Knol, but actually it takes a bit longer. :)

I see Larry and Sergei socially from time to time. I spoke about the search project at Google Zeigeist a few months ago. Going to a google party next month. The media loves a "fight" but really, that's just a nice story arc the press makes up. (Notice: google is not in the search business, google is in the advertising matching business. This search engine doesn't hurt that business at all, indeed it probably makes it marginally less likely we will see the emergence of a proprietary competitor to topple them.)

It is actually possible for people to just enjoy doing cool stuff without being bastards about it. People forget this sometimes, maybe due to the reputation of a certain dominant software provider. :)

Re:What a joke...

[jwales]

Again, it would be hard for this to be a response to Knol, since I announced it and have been working on it for a year. :-)

And, if you read the linked article, you would know that *zero* donations from Wikipedia have anything at all to do with this: Wikia is a completely separate organization.

Also don't make the classic mistake of thinking that "open source" automatically means "volunteer coders". It generally does not, and the classic FUD from the proprietary world fails to describe reality for precisely this reason.

And finally, one of the most important concepts here is that of a broad deep whitelist, which is something that I think can be done realiably and well with appropriate tools in the hands of the end users. The entire problem of bot-driven spam comes from a lack of reliable quantities of human oversight in the process. All you have to do to massively spam google is fool a computer. (Well, even then, google does a pretty damned good job of preventing massive spam though of course there are always some problems.) Pretty hard to get that nonsense by a properly organized community effort.

(But of course, the design of a community which can move things forward quickly without a lot of useless work is nontrivial.)

Bingo

[Anonymous Coward]

Exactly. I have *always* used google to find wikipedia articles. You can't beat google with a 'site:' prefix. As a matter of fact, I have the following firefox bookmark stashed under the "wik" keyword:

http://www.google.ca/search?complete=1&q=site:en.wikipedia.org+%25s [google.ca]

Which means of course, that simply typing "wik integer" into my address bar provides me with a list of wikipedia articles relating to integers. No need (or desire) for wikipedia's own search.

Re:Sooo....

[jwales]

Completely different. :) For one thing, we are doing everything completely freely licensed. Mahalo is proprietary.

For another thing, Mahalo is "human edited" search results for the top queries, which is not a bad idea of course, but it is not intended to be a full search engine. Mahalo have indicated an interest in replacing their google search backup with our open source alternative, if we get to be good enough, which is obviously a far from foregone conclusion.

Re:Your track record says otherwise

[mdwh2]

Come to your wikipedia page?

you mean the one that you have been documented (and here) not only editing, but wiping clean the edit history on, trying to bury your tracks?

No - he said his talk page, not his Wikipedia article.