Netflix Prize Competitor Already Beats Netflix

Baldrson writes "Within the first week of the announcement of The Netflix Prize a team has already beaten Netflix's own movie recommendation algorithm. This is pretty impressive given the previously quoted researcher who said: 'You're competing with 15 years of really smart people banging away at the problem.' The team is WXYZConsulting.com apparently registered by a data mining professor named Yi Zhang. Congratulations are in order for Netflix and Prof. Zhang's team who are demonstrating, yet again, the power of prizes to accelerate progress."

Upon further consideration...

the power of prizes to accelerate progress

Hmm...In that case, I'm offering $1000 USD to the person or group that can find me the perfect girlfriend!

Re:

Your girlfriend's name contains 5 letters. Just count them with your left hand.

Send me my check thanks.

Re:Upon further consideration...

Well, would a temporary girlfriend work? I'm pretty sure you could find one for that amount of money. Otherwise, I think you would need to pay more.

Re:Upon further consideration...

Comb your hair, permanently delete all your porn and all those DVDs of backups, comb your hair and shave the beard. Buy some new, clean, clothes without words or logos. Ask out pretty girls. Avoid the use of the words woot, pwnd, and 'leet' in any casual conversation. Do not admit to your unhealthy infatuation of a sci-fi or fantasy series of books or movies.

Re:Upon further consideration...

So your advice is to be somebody else entirely in an attempt to please someone? That's the foundation of a healthy relationship if I ever saw one.

If you follow his advice...

...you might just win yourself a relationship so 'special' that 90% of all couples in America share. You'll buy her jewelry and allow her to spend your money on frivolous trifles, and she in turn will allow you to stick your penis in her vagina. When all else fails, resort to mediocrity!

Assuming this isn't a hoax...

I think this demonstrates how important "many eyeballs" are in problem solving. Intelligent people "who have been attacking the problem for 15 years" can still fail to see an "obvious" solution. I shudder at how many scientific fields probably have obvious solutions that aren't being found because only a small cadre of people have been exposed to the problem. I also shudder at people who artificially set up barriers to understanding their own fields, in order to protect their own egos. The attitude of "journal articles need to be cryptic or they must not be important" needs to go.

Sometimes one person with a different perspective

Sometimes one person with a different perspective on a problem can see something that a groups of "experts" had never thought of, or had discounted because they assumed it wouldn't work.

That's why a fresh perspective on a problem can be quite enlightening, and why I tend to go ask other programmers for their ideas/comments when I get stuck. I don't know everything, and I sometimes make stupid assumptions or forget to consider certain technquies. No group is immune from this.

Re:

I did work for the USDA horticulture lab around here awhile back. I didn't think security of the data was a huge deal, just as long as no one outside could get to it.

Boy was I wrong. Within the same building, it was a big deal to show other scientists y

Hmm, it also demonstrates...

I think this demonstrates how important "many eyeballs" are in problem solving.

I think it also demonstrates how the oft-used mantra of "if it needs to be done, it will be done" doesn't always work without some incentive. One of the hurdles of OSS is that the only things that get worked on are the things that people want to work on. The love of developing software can only get you so far (and wow, has it gotten us far). But for some things to advance, it will need financial backing. It's a prickly problem for the OSS community.

banned in Quebec

Looking in the competition rules, I was surpised to see that:

Residents of the province of Quebec in Canada are ineligible to participate. Residents of Cuba, Iran, Syria, North Korea, Myanmar (formerly Burma) and Sudan are also ineligible to participate.

Is Quebec the next target for regime change?

Re:banned in Quebec

Quebec outlaws most contests by requiring companies offering contests to have a head office or place of business in Quebec. No need to resort to conspiracy theories, it's just good business to make them ineligible due to their laws.

Re:banned in Quebec

From the FAQ [netflixprize.com]: "Most of those countries appear are on the U.S. Treasury Office of Foreign Assets Control's list of embargoed counties for which we cannot provide economic assistance. If this list changes, we'll post a change to the rules and let you know. Quebec has other reasons." Here's why [about.com] Quebec is on the list.

Re:

iirc Quebec has very stringent rules [free-news-release.com] on contests and lotteries and is usually excluded from international contests because of the prohibitive effort required to comply.

Umm... Duh

Quite a few teams have beaten the Cinematch engine, but not by the required 10% for the prize. The submission is in error. They also haven't won the 1% Progress prize yet, but they're very very close.

to note --

although yes they have "beatten netflix", they haven't won the prize yet.
they have about a 1% improvement on the netflix algorithm, but the prize is for 10%. they are the frontrunner for the progress prize, though, being the people who are the closest to the mark after a year (i think).

on top of that, netflix has been doing improvements on their own code in the meantime, and its been looking like around a 1% improvement, also.

I'll do you one better.

I have perfected the perfect movie recommendation mechanism. It's called a "friend."

I hold a patent on the idea, and I've copyrighted the statement "hey, I saw this movie you'd like."

Re:I'll do you one better.

Actually, I find that friends have a less than 60% chance of making a recomendation that I'll like. People like vastly different things, and for different reasons.

However, recommendations from multiple friends raises the accuracy to close to 100%.

Recommendation Software

You know, I've never seen recommendation applications worth much of anything. Ringo was okay, until M$ turned it into Firefly, which died in 1999(?). It will be interesting if this turns out well, or if it turns out like TiVO, which in Patton Oswalt's words, is like "working with a retarded kid." "No, TiVo, NO! Westerns aren't cartoons! / But you like horsies! Liar!"

IMDB

IMDB's recommendation system for movies is so bad it's funny in it's own right. You could look up a science fiction movie, and it would recommend a head cleaner cassette for a Betamax VCR.

Congratulations

Congratulations on your solution to the Netflix problem. You might also find the following problem(s) interesting:

• Millenium Math Problems [claymath.org]
• International Campaign to Ban Land Mines [icbl.org]
• World Peace [worldpeace.org]
• Where are my socks? [laundry-alternative.com]
• Where's Waldo? [amazon.com]

Just another proof that IT depts aren't that smart

I've been selling technology for almost two decades and one thing that I see over and over is that internal IT departments either a.) vastly overestimate their abilities b.) prevent introduction of outside techology providers for political reasons or c.) both. There are several companies where the CIO told me "oh, we're already building that in-house. it will be live next quarter" and years later they still have not successfully implemented that technology. Kudos to Netflix for acknowledging that somebody outside their company might be able to do it better. At most companies, the CIO would have never let this happen and/or the CEO wouldn't have the business sense to challenge internal assumptions.

It's not called Prizes. It's Patronage.

I'm tired of people not realizing that "Prizes" are really just Patronage in desguise. I'm not saying Patronage is a bad thing... far from it. But the idea that Prizes are somehow working shouldn't come as a surprise to anyone with knowledge of 15th century aristocracy.

Pay the people who do the work, don't get people to work for pay.

Re:It's not called Prizes. It's Patronage.

An AC already pointed this out, but I'm without mod points: This is not patronage. A patron supports the worker until the work is completed. These people are supporting themselves while they do the work, and collecting payment only if and when the work is done to the payer's satisfaction.

It's a pretty clear distinction. This is a prize.

Re:Is the algorithm available?

it takes all of a few seconds to "register a team" I did it a few days ago just to have a peek at the test data.