Build a Better Netflix, Win a Million Dollars?

An anonymous reader writes "In a quest to better movie recommendations, Netflix is opening their database (nytimes, registration and first child required) to users to try to craft a better recommendation technology. The problem is not easy. Says one researcher: 'You're competing with 15 years of really smart people banging away at the problem.'" Recommender systems are really an interesting problem, and that is likely very interesting data to play with.

I had a thought like this a while back...

[AceCaseOR]

..except, instead of making it open to the community (which is not a bad idea, I must say) I thought of having Google do it. This is, perhaps, IMHO, a much better idea. Now, what we really need is a Movie Genome Project, much like the Music Genome project that lead to Pandora.

go see porn sites

[LiquidCoooled]

They have decent tech for building similar/recommended alternative pages.
Especially the newer blogish type pages where theres a gallery and a small selection underneath.

Not that I would know of course.

Privacy issues?

[Vultan]

How will they handle privacy issues? Don't the same issues appear here that appeared with the AOL data this summer? With enough ratings you can narrow down to a specific person, and then find out about all the pr0n that this person has been getting as well.

Copy the Music Genome Project

[Zaphod-AVA]

The problem with recommendation systems is that they use too little information to catagorize their subject.

What they need to do is copy the methods of the Music Genome Project (www.pandora.com), and list a larger set of attributes for the films. This way it can recommend films by checking many more characteristics, such as director, tone, writer, or subject.

only a million?

[StandardDeviant]

If you can beat "15 years of really smart people", then your work product probably has more than a million dollars in value if you were to license it out to places like Amazon, eBay, Netflix, etc. Even a 1% improvement in revenues from a 1% improvement in recommendation accuracy is probably worth more than 50K, if sold to the major e-tailers. On the other hand, if you just want an interesting problem to screw around with in your spare time and don't want to go through the bother of forming a company in order to monetize that work, this is a pretty cool opportunity.

Fix the problems with what they send me first

[Jimmy King]

I wish they'd fix the problems in the logic determining what they actually send me from my queue before fixing problems with what they recommend to me. If I've got season 1 of a show in my queue prior to season 2, don't start sending me season 2 because some disc of season 1 is unavailable (which has happened to me multiple with both netflix and blockbuster online), send me something else completely. They've got the tech to keep one season of a tv show in order, it can't possibly be that difficult to extend that to keeping multiple seasons of a show in order.

On top of that, don't show me that it's available in my queue but send me something else instead. While I haven't asked netflix about this, I have asked blockbuster online, and I imagine they are both doing the same thing. The disc is "available" just not at the warehouse used to ship to me personally. Instead of basing one piece of information off of total stock and one off of local stock, base them both on the stock at the warehouse shipping to me.

Difficulties on the data-gathering end

[jfengel]

Any marketer will tell you that what people tell you they want and what people actually want are very different things. Even if people answer honestly, the data you gather is often unreliable: people simply don't have as good a handle on what they want as they think they do.

Not that marketers have a better handle, but simply that people will swear up and down that they would buy a peanut-butter-filled hot dog, that they loved the one they tried, and then don't actually buy any.

Don't believe me? Go see Snakes on a Plane. Nobody else did. (Sure, $33 million seems like a lot, but that's chump change for a major studio release these days.)

The best improvements will come from insights gained between the lines. You may have rated The English Patient eleventeen stars, but if your next seven rentals were all episodes of The Girls Next Door, which you only rated 3 stars, it certainly looks like you want more Hugh Hefner and less Ralph Fiennes.

The best data is the data that the subject doesn't realize he's giving you. Once you start imposing conscious choice on the ratings, you get only what they say they like, not what they really like.

Intractable problem - liking the movie, not genre

[OakDragon]

I stopped rating movies after I found that I got recommended a lot of crap. Say I rent a slasher movie that, for its genre, is artfully done. I rate it high. Now I have recommendations for a bunch of worthless, straight-to-video stuff that I really don't want to see.

This is the real nut to crack, IMO. How do come up with an algorithm that rates 'quality,' an elusive concept that means different things to different people?

Not to mention, I'm fickle.

Here's a problem to solve with much larger impact

[Yogs]

Disclaimer: I subscribe to the same sort of service, except through blockbuster... maybe Netflix does have this feature. My wife and I share a queue... I imagine many, many of these queues are shared. We have very, very different tastes in movies. Instead of getting recommendations that suit us both (which is next to impossible), the recommendations just get very, very confused. If I could just keep my and her recommendations from tangling, we would both have an easier time.

I hope Amazon...

[GigG]

I certainly hope Amazon chooses to license anything that comes out of this. I've been a Amazon customer for about 10 years and have bought a couple of hundred books from them. For the first 2 or 3 years they gave me pretty good recomendations and I found a number of new authors that I probably wouldn't have started reading. Over the last few years they never catch a new author and suggest them.