Wikia Search Engine to be Launched on January 7th 189
cagnol writes "The Washington Post reports that Jimmy Wales, the founder of online encyclopedia Wikipedia, has announced the launch of a new open-source search engine, Wikia Search, on January 7th, 2008. The project will allow the community to help rank search results, in a model close to Wikipedia. However the company is a for-profit organization. This new search is supposed to challenge Google and Yahoo."
Challenging Google? (Score:2, Interesting)
Re: (Score:2, Insightful)
Re:Challenging Google? (Score:5, Informative)
It'd be sort of cool if we could create a search engine in a week or two to respond to Knol, but actually it takes a bit longer.
I see Larry and Sergei socially from time to time. I spoke about the search project at Google Zeigeist a few months ago. Going to a google party next month. The media loves a "fight" but really, that's just a nice story arc the press makes up. (Notice: google is not in the search business, google is in the advertising matching business. This search engine doesn't hurt that business at all, indeed it probably makes it marginally less likely we will see the emergence of a proprietary competitor to topple them.)
It is actually possible for people to just enjoy doing cool stuff without being bastards about it. People forget this sometimes, maybe due to the reputation of a certain dominant software provider.
Re:Challenging Google? (Score:4, Interesting)
If so, kudos... Grub's been languishing in not-ready-for-primetime land for far too long, and the ability to crawl your own site to keep results current is a bonus, too.
Re: (Score:2, Insightful)
Sincerely,
The Rest of the Internet
Re:Challenging Google? (Score:5, Funny)
Re:Challenging Google? (Score:5, Funny)
Oh, come on. The people who matter already know that most Linux users aren't elitist snobs.
Alternatives to Wikia: Wikipedia Search (Score:2)
Compare the test query 'nobel prizes 1987' [linguit.co.uk] on Google [google.com], Wikipedia's original search [wikipedia.org] and this new search engine [linguit.co.uk], for example... they take more navigation time and user effort to get to the results.
Re:Challenging Google's Revenue Model (Score:3, Interesting)
I don't know anyone who would "pay to search" casual queries. There are some professional databases which do operate on this principle for high powered content.
From the RIAA threads we learn people don't want to pay as endusers for their content. The post above asks about the ad
Re:Challenging Google's Revenue Model (Score:5, Insightful)
Is content going to ever be totally free? It will be if people understand the inherent rewards of an open society. Information's negligible cost of duplication is the revolutionary model is the thing that is shattering the old models (c.f. http://homes.eff.org/~barlow/EconomyOfIdeas.html [eff.org]). Wikipedia is already doing that. As much as I'm a critic of Jimmy Wales, citizendium, etc. (with their NPOV lunacy), the system he's helped build is saving people's lives and improving quality of life in ways the old world just doesn't understand yet.
Personally, I'm hopeful that as long as we still have the Right to Read (c.f. http://www.gnu.org/philosophy/right-to-read.html [gnu.org]), we're on the path to freedom and salvation. A corporation who makes up a new "model" to take advantage of content producers isn't going to take hold anymore. There's just not a point anymore. The price of content is already quite low for common knowledge. Even if the arbiters of knowledge try to keep it from common knowledge, we can paraphrase it. The greatest risk to real productive use of our knowledge still remains Patents. Information may finally be free, but the freedom to tinker is not.
Re: (Score:2)
The old world was based on personal greed and individualism, even ideology cannot trump efficiency and human community, it's nice to know that inside there is a helper/altruistic tendency to human beings.
Re: (Score:2)
I was a little fuzzy with my set theory. "A certain segment of people" doesn't want to pay as an enduser. Those are the people who originated the new battlefront. If you pay the classical way, great for you, great for them, everyone goes home to dinner.
Content is becoming "free as in beer" to the enduser for the next phase, financed by ads.
someone can figure out how I can earn back the check I just dropped off to my landlord today.
Re: (Score:2)
Pity to see things heat up between the 'good guys'.
There is a difference between healthy competition and trying to drive your competition into the ground in order to squeeze every last penny from the market.
Competition should not be confused with the anticompetitive mafia like behavior that we all too often see from some other big business. For example, if Google acted like Microsoft (or the old AT&T and IBM) they would use their market position and simply require websites to exclude other search engines from indexing their web pages or else be exclud
Re: (Score:2)
More like supplant altavista. Yahoo search never really operated in the same 'space' as googles search, that's why they're still around.
Easily Abused? (Score:5, Insightful)
Re:Easily Abused? (Score:5, Funny)
Well that's lame. (Score:3, Funny)
What a crappy search engine. When I search for something, I want the top 100 results to be 100 different porn sites! I can find two porn sites without help.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re:Easily Abused? (Score:5, Insightful)
Re: (Score:2)
Having an open algorithm is good, as non-disclosure isn't security, but the issue is allowing people to rank searches and such. Having that public is asking for people to abuse the system, and as noted before, a lot of malicious parties could seemingly legitimately rank their sites (porn sites, etc) higher, leading to ranking battles by bots. Of course, the issue of vandalism occurs with Wikipedia, however when people are looking to make money off of it they'll likely be more persistent.
With the criteria publicly accessible, yes, porn sites may be able to try to game it, but all the somewhat large producers will also be able to game it as well. Any engine with a flaw that allows it to be "gamed" is not a robust engine at all. If anybody spends a few weeks on the problem they can investigate any system and reverse-engineer it. Google is no different in this regard. A lot of what you don't realize is that a lot of google's processes are actually very manual. What do you think they are
Needs to be more open than that (Score:2)
Re: (Score:3, Interesting)
Re:Easily Abused? (Score:5, Interesting)
Point well made - while spam attacks may be pretty obvious, they could be spread out over time to make them less obvious.
Additionally, I can see this search engine being very much affected by public mood. For example, say there was a royal death and a certain right-wing 'upmarket' tabloid newspaper [dailyexpress.co.uk] decided to claim that it was a conspiracy by the Government to kill the royal off. This is linked to from said newspaper's web site, and this people improve its ranking. Therefore it floats to the top of the results pile, thus giving it more exposure and setting off a vicious cycle.
Just a hypothetical situation, but certainly possible. Such a model would also make it possible to carry out smear attacks and to ruin the rankings of competing companies, parties, organisations, whatever - a practice that IMHO should be left to search engine admins.
Re: (Score:3, Funny)
Oh yeah. Let's give the highly underpaid, highly overworked admins yet more unrelated tasks to carry out! Can't you people do your own company smear attacks? Why do you want to bother the admins with that? Besides, smearing competitors is technically new content, and content creation i
Re: (Score:2)
Re: (Score:2)
A "community" model is unlikely to scale. The early search engines like Yahoo were community models, and a good example is still dmoz. These models started getting beaten around 96 or so by fully automated systems, because such models don't have depth. You get a tip of the iceberg effect, because there is only so much editorial
Re: (Score:2)
Errr, what? If they had the resources to do it, why wouldn't they? Especially considering their overall support of open source and Wikis in particular.
Re:Easily Abused? (Score:5, Informative)
One of the first lines of defense in the early days will be use of a community (wiki) generated whitelist [wikia.com] of sites to crawl. We will want to work outward from there, but basically the first thing is for us to assess "look, what are the most important must-have sites on the net" and crawl them. One thing that the mainstream media never seems to report very well, mostly because I think they don't get why it is important, is that we are doing everything here under free licenses. The software GPL, the data we generate under free licenses, etc. The aim here is not just to create a good search engine, but to create it and *give it all away* in a way that I think has a chance to restructure the entire search industry. Well, maybe not, maybe so, but what the hell, it'll be fun to see. :-)
Your track record says otherwise (Score:4, Insightful)
The thought that Jimmy Wales, cofounder of Wikipedia could have an open site without abuse is laughable. You operate under the sham of an open community, yet exclude those outside a very narrow political agenda. Your a fraud, using open source principals as a smokescreen that presents your personal world-view set as fact to the world. I don't buy what your selling, and I'm calling your bluff. The sad thing is that this will probably make you a fair amount of money if more people don't start to see through you.
But then the wonderful thing about leading revisionist history is you can substitute your own revisions for reality....
Mod Parent (Score:3, Interesting)
Speaking explicitly as a reader of slashdot, with all the group-think biases a site like this introduces, wikipedia is floundering in a mire of their own arrogance, and the dissatisfaction with this needs to be heard.
Re: (Score:3, Interesting)
Actually, no. Wikipedia can be criticized on a lot of grounds, some of them even valid
Perhaps you'd like to come to my talk page at Wiki
Re:Your track record says otherwise (Score:5, Insightful)
you mean the one that you have been documented [wikipedia.org] (and here [wikitruth.info]) not only editing, but wiping clean the edit history on, trying to bury your tracks?
The game you're playing is dirty and how dare you come here unwilling to meet us on equal ground.
Re: (Score:2, Informative)
you mean the one that you have been documented (and here) not only editing, but wiping clean the edit history on, trying to bury your tracks?
No - he said his talk page, not his Wikipedia article.
Re: (Score:2)
Re: (Score:2)
Re:Easily Abused? (Score:5, Insightful)
I thought so. Your solution is already broken.
Re: (Score:2, Interesting)
timothy
Re: (Score:3, Insightful)
I've been running a (small, nothing compared to what you're doing) community powered search engine for a while now (little less than one year), it's been a neat little project and I've learned a lot.
I think the combined power of having your name and wikipedia as a launchpad and quite probably the capital to see this through may give you a chance worth taking.
Re: (Score:2)
Re: (Score:3, Insightful)
So basically...they're asking for people to abuse the ranking system. To patrol something like this would require a company with resources like Google, and most likely the reason Google doesn't have such functionality. Just my two cents.
And when you think about it, Google's pagerank algorithm already returns search results based on what the community thinks. This new venture is simply a means to take other peoples' sweat equity and turn it into profit for good old Jimmy while giving the people who did all the work little more than warm fuzzies inside, if that.
Re: (Score:2)
Google is coming off more and more as a smarmy, slimy, privacy invasive, viral marketer.
Overall it looks to be a very interesting project and of worth while value to the community and is sure to create many lively debates as it grows.
There is also likely to considerable support from commercial companies looking to gain a piece of the search engine market
Re: (Score:2)
Google results are based upon what the community think?
Pagerank is based on others linking to websites. So in Google's case, the community == the entire internet instead of some limited subset who get to decide what's relevant and what isn't.
is that anything like addwords are democratic (as long as you have the money to pay for them)
You mean AdWords? No, I wasn't referring to that -- see my comment above. Also if this new venture is to be for-profit, you think there won't be paid ads of some kind? Perhaps money exchanging hands for certain benefits as long as you have the money to pay for it?
Re: (Score:2)
It is community driven yet difficult to manipulate.
If I link to your site from my site then its a vote for your site.
The vote quality depends on the number of votes for my site.
Paradox? Yes it is but you can calculate it to a certain extent and it gives the perceived quality of a site by the community.
The rest of your post is off topic and your just trolling.
Re: (Score:2)
Re: (Score:2)
He's a hero of the public domain movement to me, he has enabled an unparalleled wealth of free content (perhaps Sourceforge is a parallel), anyone who finds a way to make open collaboration profitable and therefore competitive in this capitalist world should be heralded a champion of the open source movement.
And Wikipedia is profitable? I thought it was sustained by the donations of corporations (bandwidth, money, etc.), individuals (such as the latest fundraiser), and the efforts of its volunteer army. Where's the capitalistic profit? If he manages to make this one profitable then start the hero worship at your leisure, but so far I don't see much in the way of success.
Re: (Score:2)
4. Profit!
Out of the equation. Open source is fine, but it will end up making itself useless as it will be profitable to get the code and figure out how to artificially UP your rankings, or sell that info, or pretend you know how, and just take people's money with the promise that you can help them. We don't need more companies doing that, god knows.
yeah (Score:4, Funny)
Not only that, Wikipedia is reporting that its marketshare has tripled in the last six months.
Re: (Score:2)
Market share of... (Score:5, Funny)
Re:Market share of... (Score:4, Funny)
I don't care how they arrive at a rank! (Score:5, Insightful)
Personally, I don't care how search engines rank the websites they return as long as what is returned is proper, relevant and useful.
Re: (Score:2)
Vandalism (Score:2)
My prediction: killed by nonprofit competition (Score:3, Interesting)
Re: (Score:2, Insightful)
What I always wanted (Score:2)
first things first (Score:5, Insightful)
Re:first things first (Score:5, Insightful)
As long as I need to use google to search Wikipedia, I don't see Wikipedia creating a google killer.
Re:first things first (Score:5, Interesting)
Re: (Score:2)
Re: (Score:2)
It's not hard to make a list of all the search phrases people enter into the search bar, and if you have millions of people searching, you'll get a lot of redundancy on even fairly esoteric typos. Since many people also correct their own typos, if you combine and remember all consecutive search phrases by the originating IP address, you'll get a frequency bump for when a common typo is immediately fixed.
With the volume of search phrases going through their servers, there's no need for
Re: (Score:3, Interesting)
They also do a variety of statistical analysis on a ridiculously large data set. For example, if a particular phrase appears over and over again, and all of the words in the query match the phrase save one, it may be more likely that the non-matching
Re: (Score:2)
Slashdot broke your URL. For anyone attempting to create a similar keyword, simply replace "%25s" with %s".
What a joke... (Score:3, Insightful)
If you think wikpedia gets vandalized, wait until there's money involved. Wikpedia for all it's trappings, doesn't directly influence spam. But a search engine... IF, and this is a big IF, this thing becomes mainstream, having the code public will make it very easy for the bot herders to control it. The idea is simply flawed. Google is currently dealing with bot herders attempting to manipulate it's page ranks - while the idea of it being open source sounds great (well, ok it doesn't to me - I don't have the love affair with open source that most slashdotters do - I've never bought into the security myth that there's GOOD coders out there with so much free time on their hands that they are walking OTHER peoples code. I don't like doing that when I'm PAID to do it. Not too mention there just aren't that many good coders out there....but I digress) it's simply going to work right into the hands of the malware crowd - especially now that it's more organized crime than it is vandalism.
EK
Re: (Score:2)
having the code public will make it very easy for the bot herders to control it
This assumes that it's impossible to devise a web search algorithm that can automatically and reliably avoid poisoning. I don't think there's any particular reason to believe that this is true. Humans are quite capable of identifying with a fairly high amount of accuracy what sites are just linkfarms, or useful sites which are the target of a de-ranking campaign, so it should be possible to have a computer automatically do it.
I'm not saying they will succeed, just that it isn't necessarily flawed at a f
Re: (Score:2)
Re: (Score:2)
Just because we don't currently have an algorithm that can defeat spammers, it doesn't mean it's impossible to ever create it. The fact that "even Google" are constantly tweaking their search methods suggests that they don't believe it's as good as it can possibly get, and theirs is just one possible way of indexing the internet. The answer may well in a completely different approach.
My hunch is that the "end game" will be a massively distributed network where individual users index some subset of the int
Re:What a joke... (Score:5, Informative)
And, if you read the linked article, you would know that *zero* donations from Wikipedia have anything at all to do with this: Wikia is a completely separate organization.
Also don't make the classic mistake of thinking that "open source" automatically means "volunteer coders". It generally does not, and the classic FUD from the proprietary world fails to describe reality for precisely this reason.
And finally, one of the most important concepts here is that of a broad deep whitelist, which is something that I think can be done realiably and well with appropriate tools in the hands of the end users. The entire problem of bot-driven spam comes from a lack of reliable quantities of human oversight in the process. All you have to do to massively spam google is fool a computer. (Well, even then, google does a pretty damned good job of preventing massive spam though of course there are always some problems.) Pretty hard to get that nonsense by a properly organized community effort.
(But of course, the design of a community which can move things forward quickly without a lot of useless work is nontrivial.)
Re: (Score:2)
What I think is a joke is the advertising model. Wikipedia wouldn't be Wikipedia is there were sponsored ads. One of the first things I saw on the Wikia Search link was an ad for Netflix and a promise of the site's four Open Principles (which includes Transparency).
Can I ask a question I think is important... whose pockets does the advertising revenue go into? Because my desire to support the site is incredibly contingent on the answer to that question. And my inquisitiveness about the answer is what
Re: (Score:2, Insightful)
It seems odd to me that a completely separate organization would get this very special treatment. This ensures Wikia gets higher search engine rankings, and by extension more exposure and ad revenue.
What is your explanation for this?
Re: (Score:3, Interesting)
There are a huge number of sites in the interwiki linnk map:
http://meta.wikimedia.org/wiki/Interwiki_map [wikimedia.org]
Including for example, uhm, slashdot. And Citizendium. And Merriam-Webster.
And finally, I have nothing to do with the list. I've never edited it, never asked anyone to edit it, and I have no input into what goes on it.
I am sure you will apologize for spreading this information. Right?
Re: (Score:2)
No, Jimmy Wales had this idea for a long long time.
Re: (Score:2)
I can see... (Score:3, Funny)
Just in Time for the Election (Score:2)
What a great idea! (Score:2)
Hope it works better than wikipedia's search (Score:5, Insightful)
Don't get me wrong, I like wikipedia, but their search on the site is next to worthless.
Re: (Score:2)
Re: (Score:2)
citation needed (Score:2)
Wikia, the place to go for furry fan fiction (Score:4, Insightful)
Wikia has been something of a dud. What Wikia really does is monetize fancruft. Their big wikis are for Star [Trek|Wars|Gate|Craft], Everquest, Marvel comics, Yu-Gi-Oh, and similar subjects. They're the resting place for fan articles thrown out of Wikipedia. [wikia.com]
Wikia's search engine, based on the user demographic they have now, is going to have great coverage of furry fan fiction. [wikia.com]
There's already a good manually-updated search engine. It's called Open Directory [dmoz.org]. It's quite useful as a data source for answering the question "what is this web site about"? It tends to run months behind changes to the web, since it's manually updated. While not many people query DMOZ manually, it's used by Yahoo, Google, etc. to get some basic information about a web site.
As an example of how great Wikia search is going to be, Wales suggested searching for "Tampa hotels". [techcrunch.com] The major search engines return too many bottom-feeder reseller and directory sites for searches like that. As I point out occasionally, we've already solved that problem over at SiteTruth [sitetruth.com], which looks for business legitimacy. Type in "Tampa hotels" there and watch it push the marginal sites to the bottom of the search results. We have that one handled.
Wikipedia works because people are willing to do substantial work for free for a non-profit organization. That doesn't work for a commercial business. You can get people to write about themselves (Myspace, Facebook, etc.) but beyond that, "crowdsourcing" doesn't go very far.
Re: (Score:2)
Funny thing is (Score:2)
The problem here is... (Score:3, Interesting)
How is Wikipedia to draw traffic to their search engine? Obviously not via Google, as search engines are content free on their own. Integrating it with Wikipedia? But again, Wikipedia is the end target, not a start point, so how could this work.
I don't think Wikipedia has the strategy or money for this to reach critical mass and show its potential, but it'll be interesting as an experiment.
Re: (Score:2)
It's called relevance feedback (Score:2)
And since people are bringing up Google as competition: Google Search has an estimated retrieval accuracy* of around 10%. Not very hard to beat, except that the Internet is a rather large document set. Have you ever browse
Scary Implications (Score:4, Funny)
Wikipedia fails (Score:2)
Why would the search website work?
I doubt it works.
Re: (Score:2)
so.... (Score:2)
Promoting stuff isn't their primary business I guess.
Hope they don't do ads....
Google already does this? (Score:2)
Because of Google's anti-spam techniques this method is very hard to game. Better still, the defences are automatic. On Wikipedia, a lot of vand
Personal preference ranking... (Score:2)
What I want is to get rid of things outside my daily context. I work in web development so when I do a search for "CSS 3 column hack" I don't want to get results related to some sport team abbreviated CSS where they have a coach whose a hack but likes to use a 3 column lineup.... stretching a but but you
One-Upping the Least Bad (Score:2, Insightful)
Funny to read this today, after I spent a couple of hours yesterday searching Google for something that doesn't exist -- a Plucker [plkr.org] type app for the iPod Classic. What struck me was just how badly Google performed. Any search containing the word "iPod" seems to return pages upon pages of blog entries about the (long since released) iPhone. What one tends to find with a Google search are a lot of loud, content-light blog entries, popping with ads, with short dashed-off articles broken across several pages.
What's gonna happen to Wikia? (Score:2)
So what's going to happen to those Wikis now that Wikia is turning from a MediaWiki host site to a search site?
Re: (Score:3, Funny)
And Furries.
Re: (Score:2)
Re: (Score:2)
Re:Sooo.... (Score:5, Informative)
For another thing, Mahalo is "human edited" search results for the top queries, which is not a bad idea of course, but it is not intended to be a full search engine. Mahalo have indicated an interest in replacing their google search backup with our open source alternative, if we get to be good enough, which is obviously a far from foregone conclusion.