Slashdot Log In
Wikia Search Engine to be Launched on January 7th
Posted by
timothy
on Tue Jan 01, 2008 09:01 PM
from the wisdom-of-crowds dept.
from the wisdom-of-crowds dept.
cagnol writes "The Washington Post reports that Jimmy Wales, the founder of online encyclopedia Wikipedia, has announced the launch of a new open-source search engine, Wikia Search, on January 7th, 2008. The project will allow the community to help rank search results, in a model close to Wikipedia. However the company is a for-profit organization. This new search is supposed to challenge Google and Yahoo."
Related Stories
[+]
Technology: Wikia Search Launches Alpha, Not Ready Yet 107 comments
babooo404 writes "Jimmy Wales' latest project, Search Wikia has launched into alpha this morning. Most reviews have been negative. The system is a 'social search' and uses the Nutch search algorithm. You can friend people along with creating profiles, and the system uses a Wikipedia-style format for 'mini articles.'"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Challenging Google? (Score:2, Interesting)
Re: (Score:2, Insightful)
Re:Challenging Google? (Score:5, Informative)
It'd be sort of cool if we could create a search engine in a week or two to respond to Knol, but actually it takes a bit longer.
I see Larry and Sergei socially from time to time. I spoke about the search project at Google Zeigeist a few months ago. Going to a google party next month. The media loves a "fight" but really, that's just a nice story arc the press makes up. (Notice: google is not in the search business, google is in the advertising matching business. This search engine doesn't hurt that business at all, indeed it probably makes it marginally less likely we will see the emergence of a proprietary competitor to topple them.)
It is actually possible for people to just enjoy doing cool stuff without being bastards about it. People forget this sometimes, maybe due to the reputation of a certain dominant software provider.
Parent
Re:Challenging Google? (Score:4, Interesting)
If so, kudos... Grub's been languishing in not-ready-for-primetime land for far too long, and the ability to crawl your own site to keep results current is a bonus, too.
Parent
Re:Challenging Google? (Score:5, Funny)
Parent
Re:Challenging Google? (Score:5, Funny)
Oh, come on. The people who matter already know that most Linux users aren't elitist snobs.
Parent
Re:Challenging Google's Revenue Model (Score:3, Interesting)
I don't know anyone who would "pay to search" casual queries. There are some professional databases which do operate on this principle for high powered content.
From the RIAA threads we learn people don't want to pay as endusers for their content. The post above asks about the ad
Re:Challenging Google's Revenue Model (Score:5, Insightful)
Is content going to ever be totally free? It will be if people understand the inherent rewards of an open society. Information's negligible cost of duplication is the revolutionary model is the thing that is shattering the old models (c.f. http://homes.eff.org/~barlow/EconomyOfIdeas.html [eff.org]). Wikipedia is already doing that. As much as I'm a critic of Jimmy Wales, citizendium, etc. (with their NPOV lunacy), the system he's helped build is saving people's lives and improving quality of life in ways the old world just doesn't understand yet.
Personally, I'm hopeful that as long as we still have the Right to Read (c.f. http://www.gnu.org/philosophy/right-to-read.html [gnu.org]), we're on the path to freedom and salvation. A corporation who makes up a new "model" to take advantage of content producers isn't going to take hold anymore. There's just not a point anymore. The price of content is already quite low for common knowledge. Even if the arbiters of knowledge try to keep it from common knowledge, we can paraphrase it. The greatest risk to real productive use of our knowledge still remains Patents. Information may finally be free, but the freedom to tinker is not.
Parent
Easily Abused? (Score:5, Insightful)
Re:Easily Abused? (Score:5, Funny)
Parent
Well that's lame. (Score:3, Funny)
What a crappy search engine. When I search for something, I want the top 100 results to be 100 different porn sites! I can find two porn sites without help.
Re: (Score:2)
Re:Easily Abused? (Score:5, Insightful)
Parent
Re: (Score:3, Interesting)
Re:Easily Abused? (Score:5, Interesting)
Point well made - while spam attacks may be pretty obvious, they could be spread out over time to make them less obvious.
Additionally, I can see this search engine being very much affected by public mood. For example, say there was a royal death and a certain right-wing 'upmarket' tabloid newspaper [dailyexpress.co.uk] decided to claim that it was a conspiracy by the Government to kill the royal off. This is linked to from said newspaper's web site, and this people improve its ranking. Therefore it floats to the top of the results pile, thus giving it more exposure and setting off a vicious cycle.
Just a hypothetical situation, but certainly possible. Such a model would also make it possible to carry out smear attacks and to ruin the rankings of competing companies, parties, organisations, whatever - a practice that IMHO should be left to search engine admins.
Parent
Re: (Score:3, Funny)
Oh yeah. Let's give the highly underpaid, highly overworked admins yet more unrelated tasks to carry out! Can't you people do your own company smear attacks? Why do you want to bother the admins with that? Besides, smearing competitors is technically new content, and content creation i
Re:Easily Abused? (Score:5, Informative)
One of the first lines of defense in the early days will be use of a community (wiki) generated whitelist [wikia.com] of sites to crawl. We will want to work outward from there, but basically the first thing is for us to assess "look, what are the most important must-have sites on the net" and crawl them. One thing that the mainstream media never seems to report very well, mostly because I think they don't get why it is important, is that we are doing everything here under free licenses. The software GPL, the data we generate under free licenses, etc. The aim here is not just to create a good search engine, but to create it and *give it all away* in a way that I think has a chance to restructure the entire search industry. Well, maybe not, maybe so, but what the hell, it'll be fun to see. :-)
Parent
Your track record says otherwise (Score:4, Insightful)
The thought that Jimmy Wales, cofounder of Wikipedia could have an open site without abuse is laughable. You operate under the sham of an open community, yet exclude those outside a very narrow political agenda. Your a fraud, using open source principals as a smokescreen that presents your personal world-view set as fact to the world. I don't buy what your selling, and I'm calling your bluff. The sad thing is that this will probably make you a fair amount of money if more people don't start to see through you.
But then the wonderful thing about leading revisionist history is you can substitute your own revisions for reality....
Parent
Mod Parent (Score:3, Interesting)
Speaking explicitly as a reader of slashdot, with all the group-think biases a site like this introduces, wikipedia is floundering in a mire of their own arrogance, and the dissatisfaction with this needs to be heard.
Re: (Score:3, Interesting)
Actually, no. Wikipedia can be criticized on a lot of grounds, some of them even valid
Perhaps you'd like to come to my talk page at Wiki
Re:Your track record says otherwise (Score:5, Insightful)
you mean the one that you have been documented [wikipedia.org] (and here [wikitruth.info]) not only editing, but wiping clean the edit history on, trying to bury your tracks?
The game you're playing is dirty and how dare you come here unwilling to meet us on equal ground.
Parent
Re:Easily Abused? (Score:5, Insightful)
I thought so. Your solution is already broken.
Parent
Re: (Score:3, Insightful)
I've been running a (small, nothing compared to what you're doing) community powered search engine for a while now (little less than one year), it's been a neat little project and I've learned a lot.
I think the combined power of having your name and wikipedia as a launchpad and quite probably the capital to see this through may give you a chance worth taking.
Re: (Score:3, Insightful)
So basically...they're asking for people to abuse the ranking system. To patrol something like this would require a company with resources like Google, and most likely the reason Google doesn't have such functionality. Just my two cents.
And when you think about it, Google's pagerank algorithm already returns search results based on what the community thinks. This new venture is simply a means to take other peoples' sweat equity and turn it into profit for good old Jimmy while giving the people who did all the work little more than warm fuzzies inside, if that.
yeah (Score:4, Funny)
Not only that, Wikipedia is reporting that its marketshare has tripled in the last six months.
Re: (Score:2)
Market share of... (Score:5, Funny)
Parent
Re:Market share of... (Score:4, Funny)
Parent
I don't care how they arrive at a rank! (Score:5, Insightful)
Personally, I don't care how search engines rank the websites they return as long as what is returned is proper, relevant and useful.
Re: (Score:2)
Vandalism (Score:2)
My prediction: killed by nonprofit competition (Score:3, Interesting)
Re: (Score:2, Insightful)
What I always wanted (Score:2)
first things first (Score:5, Insightful)
Re:first things first (Score:5, Insightful)
As long as I need to use google to search Wikipedia, I don't see Wikipedia creating a google killer.
Parent
Re:first things first (Score:5, Interesting)
Parent
Re: (Score:3, Interesting)
They also do a variety of statistical analysis on a ridiculously large data set. For example, if a particular phrase appears over and over again, and all of the words in the query match the phrase save one, it may be more likely that the non-matching
What a joke... (Score:3, Insightful)
If you think wikpedia gets vandalized, wait until there's money involved. Wikpedia for all it's trappings, doesn't directly influence spam. But a search engine... IF, and this is a big IF, this thing becomes mainstream, having the code public will make it very easy for the bot herders to control it. The idea is simply flawed. Google is currently dealing with bot herders attempting to manipulate it's page ranks - while the idea of it being open source sounds great (well, ok it doesn't to me - I don't have the love affair with open source that most slashdotters do - I've never bought into the security myth that there's GOOD coders out there with so much free time on their hands that they are walking OTHER peoples code. I don't like doing that when I'm PAID to do it. Not too mention there just aren't that many good coders out there....but I digress) it's simply going to work right into the hands of the malware crowd - especially now that it's more organized crime than it is vandalism.
EK
Re: (Score:2)
having the code public will make it very easy for the bot herders to control it
This assumes that it's impossible to devise a web search algorithm that can automatically and reliably avoid poisoning. I don't think there's any particular reason to believe that this is true. Humans are quite capable of identifying with a fairly high amount of accuracy what sites are just linkfarms, or useful sites which are the target of a de-ranking campaign, so it should be possible to have a computer automatically do it.
I'm not saying they will succeed, just that it isn't necessarily flawed at a f
Re:What a joke... (Score:5, Informative)
And, if you read the linked article, you would know that *zero* donations from Wikipedia have anything at all to do with this: Wikia is a completely separate organization.
Also don't make the classic mistake of thinking that "open source" automatically means "volunteer coders". It generally does not, and the classic FUD from the proprietary world fails to describe reality for precisely this reason.
And finally, one of the most important concepts here is that of a broad deep whitelist, which is something that I think can be done realiably and well with appropriate tools in the hands of the end users. The entire problem of bot-driven spam comes from a lack of reliable quantities of human oversight in the process. All you have to do to massively spam google is fool a computer. (Well, even then, google does a pretty damned good job of preventing massive spam though of course there are always some problems.) Pretty hard to get that nonsense by a properly organized community effort.
(But of course, the design of a community which can move things forward quickly without a lot of useless work is nontrivial.)
Parent
Re: (Score:3, Interesting)
There are a huge number of sites in the interwiki linnk map:
http://meta.wikimedia.org/wiki/Interwiki_map [wikimedia.org]
Including for example, uhm, slashdot. And Citizendium. And Merriam-Webster.
And finally, I have nothing to do with the list. I've never edited it, never asked anyone to edit it, and I have no input into what goes on it.
I am sure you will apologize for spreading this information. Right?
I can see... (Score:3, Funny)
Just in Time for the Election (Score:2)
Hope it works better than wikipedia's search (Score:5, Insightful)
Don't get me wrong, I like wikipedia, but their search on the site is next to worthless.
Wikia, the place to go for furry fan fiction (Score:4, Insightful)
Wikia has been something of a dud. What Wikia really does is monetize fancruft. Their big wikis are for Star [Trek|Wars|Gate|Craft], Everquest, Marvel comics, Yu-Gi-Oh, and similar subjects. They're the resting place for fan articles thrown out of Wikipedia. [wikia.com]
Wikia's search engine, based on the user demographic they have now, is going to have great coverage of furry fan fiction. [wikia.com]
There's already a good manually-updated search engine. It's called Open Directory [dmoz.org]. It's quite useful as a data source for answering the question "what is this web site about"? It tends to run months behind changes to the web, since it's manually updated. While not many people query DMOZ manually, it's used by Yahoo, Google, etc. to get some basic information about a web site.
As an example of how great Wikia search is going to be, Wales suggested searching for "Tampa hotels". [techcrunch.com] The major search engines return too many bottom-feeder reseller and directory sites for searches like that. As I point out occasionally, we've already solved that problem over at SiteTruth [sitetruth.com], which looks for business legitimacy. Type in "Tampa hotels" there and watch it push the marginal sites to the bottom of the search results. We have that one handled.
Wikipedia works because people are willing to do substantial work for free for a non-profit organization. That doesn't work for a commercial business. You can get people to write about themselves (Myspace, Facebook, etc.) but beyond that, "crowdsourcing" doesn't go very far.
The problem here is... (Score:3, Interesting)
How is Wikipedia to draw traffic to their search engine? Obviously not via Google, as search engines are content free on their own. Integrating it with Wikipedia? But again, Wikipedia is the end target, not a start point, so how could this work.
I don't think Wikipedia has the strategy or money for this to reach critical mass and show its potential, but it'll be interesting as an experiment.
Scary Implications (Score:4, Funny)
Re: (Score:3, Funny)
And Furries.
Re:Sooo.... (Score:5, Informative)
For another thing, Mahalo is "human edited" search results for the top queries, which is not a bad idea of course, but it is not intended to be a full search engine. Mahalo have indicated an interest in replacing their google search backup with our open source alternative, if we get to be good enough, which is obviously a far from foregone conclusion.
Parent