New Algorithms Improve Image Search

bc90021 writes "Electrical engineers from UC San Diego are making progress on an image search engine that analyzes the images themselves. At the core of this Supervised Multiclass Labeling system is a set of simple yet powerful algorithms developed at UCSD. Once you train the system (the 'supervised' part), you can set it loose on a database of unlabeled images. The system calculates the probability that various objects it has been trained to recognize are present, and labels the images accordingly. After labeling, images can be retrieved via keyword searches. Accuracy of the UCSD system has outpaced that of other content-based image labeling and retrieval systems in the literature. One of the co-authors works at Google, where the researchers have access to image collections at the largest of scales."

when I was your age

I remember when we had to go to a gas station and *buy* porn. Now you have computers out there finding porn for you. You kids today have it too easy!

Parent not just funny

Since a huge % (perhaps most) image searches are for porn, it is probably a worthwhile thing for a search server to quickly classify likely porn as a way to reduce search server loading.

Cool!

If this doesn't revolutionize the searching of online porn galleries, I don't know what will. :)

Snarkiness aside, this is pretty cool stuff. I hope to see usable OSS code in a few years. Imagine how cool it would be to query "show me all pics with my daughter and her rabbits" and have it week through the 1000's of digital family photos.

Re:Cool!

I find it disturbing that you combine porn, your daughter, and rabbits all in your post.

You have issues.

Re:Cool!

> If this doesn't revolutionize the searching of online porn galleries, I don't know what will. :)
>
> Snarkiness aside, this is pretty cool stuff. I hope to see usable OSS code in a few years. Imagine how cool it would be to query "show me all pics with my daughter and her rabbits" and have it week through the 1000's of digital family photos.

...the coolness of which is directly proportional to hotness of your daughter, the hotness of whom must then be further weighted by multiplying her hotness by some function of her age. The age-multiplier curve features an abrupt discontinuity that jumps 0.00 to 1.00 at age 18, and some sort of exponential backoff function that starts decreasing the multiplier at around age 35-45.

But apart from the fact that it's almost Easter, what's with the rabbits? *clickity clic*-hey, I didn't know you could do that with Cadbury easter creme eggs!

(Rule #34: There is porn of it. No exceptions.)

Re:Cool!

> But, sadly this is Slashdot, so images that contain girls but aren't pr0n are apparently incomprehensible.

Fortunately, this is Slashdot, so discussions of pr0n that don't feature square-waves, multipliers, and exponential backoff functions are apparently incomprehensible too!

(What are these "girls" of which you speak? I only remember Millie Amp... she was imaginary, skinny as a wire, but when her insulation got stripped, she stopped resisting, got really hot, and started to moan "ohm, ohm, ohm"?)

Re:Cool!

By 'girls', I mean the limiting reagent in human reproduction. As a class of compounds, 'girls' are extremely common but somewhat volatile, so creating bonds with them is sometimes difficult. They are attracted to other similarly elusive compounds. Examples of these attracting compounds include 'Time', 'emotional vulnerability', and 'financial stability'.

so how does this ...

change the way I search for Natalie Portman p0rn?

Probability

The system calculates the probability that various objects it has been trained to recognize are present,

The probability is either zero or one, because whether or not the feature being sought is present is a state of nature. It would be more helpful to call this number the confidence that the feature is present.

Re:Probability

Not if it is a Bayesian probability [wikipedia.org].

Re:Probability

Or a fuzzy set, as (virtually) all set in the real world are.

For instance, the set of pictures for which the statement "is this a picture of a chair" is true. There is no objective criteria for this. So imagine you have a bunch of pictures and show each one to a thousand people. Sometimes you might get 0 or 1000 "yes" responses, but often you'll get some number in between (because there are chairs, but barely visible, the picture includes a kids booster seat, or a rock big enough to sit on). This could be interpreted as a probability that somebody will consider a picture to be of a chair.

Re:Probability

You're not big on quantum superpositioning I take it.

I can take it or leave it.

Re:Probability

You're not big on quantum superpositioning I take it.

I can take it or leave it.

I can take it and leave it!

A military system I saw on a TV program ...

... was similarly trained to recognise tanks in landscapes. I was doing really well - getting a great score on the fresh images it was presented with.

Then they introduced it to a new batch of images and it fell apart.

Turns out that the initial set of images had all the tanks shot on a sunny day and all the tankless images shot on a cloudy day (or vice versa). It had learned to tell a sunny day from a cloudy day.

Ha ha.

Re:A military system I saw on a TV program ...

The system used neural nets. Generally you try NN's when you don't really understand the problem well enough to try a conventional approach. The problem with NN's is you really don't know what they are actually "learning".

Why is it better?

I wish the article would mention more about why it is better than similar techniques that have been proposed in the past. (For example, http://luthuli.cs.uiuc.edu/~daf/papers/WAP-fin.pdf [uiuc.edu] seems similar) For instance, where do they get their labels for the training data? A lot of people have tried using contextual words drawn from surrounding web text to limited success due to noise. It's also questionable how well their techniques can do if they need to pre-build a separate classification for each keyword. Finally, there are words that it seems impossible that they could ever distinguish. For example, 'man' vs. 'woman,' would be incredibly complicated for anything but a human. Where are the details? Oh yeah, it's a news story! Here's a link to the paper http://www.svcl.ucsd.edu/publications/journal/2007 /pami/pami07-semantics.pdf [ucsd.edu]

Re:Why is it better?

I find it interesting which ones of the object-recognition and scene categorization algorithms make it to Slashdot.
Why does this one make it?
This is a very hot research topic at the moment.
to name a couple of groups:

http://www.robots.ox.ac.uk/~vgg/ [ox.ac.uk]
http://lear.inrialpes.fr/ [inrialpes.fr]
http://www.vision.caltech.edu/ [caltech.edu]
http://www.science.uva.nl/research/isla/ [science.uva.nl]
http://www.cdvp.dcu.ie/ [cdvp.dcu.ie]
http://www.informedia.cs.cmu.edu/ [cmu.edu]
http://www.research.ibm.com/slam/ [ibm.com]
http://www.ee.columbia.edu/ln/dvmm/newResearch.htm [columbia.edu]

oh, and people should not stare themselves blind on the claimed results.
Research papers *always* have to present good results, or else you do not get published.
Furthermore, these images are of a very high quality, make by professional photographers.
Many algorithms perform very well on these ('corel'-like) sets, while utterly failing if applied on real-world data:
http://www-nlpir.nist.gov/projects/trecvid/ [nist.gov]

Does it accually work...

...or is it something like this [slashdot.org]?

pr0n

Yes, personal searches were bound to be the first thing to be mentioned, but what about when others (ISPs, bosses, co-workers) are performing these searches on computers you use? I'm sure most people are smart enough not to do such things at work, but what about pop-ups (you couldn't help getting those kinds of popups while searching for a 'fix' to an app), false matches (boss doesn't view, only flags you if the keyword search comes back positive), etc?

Anyone tell Jeff Hawkins yet?

I wonder if these students are using this [slashdot.org] software library?

The problem is...

The problem is we all know what's gonna be the first result when searching "Caves on uranus"!!!

--
Great hosting [dreamhost.com] 200GB Storage, 2_TB_ bandwidth, php, mysql, ssh, $7.95

The tech isn't mature enough yet

Run this story again when the system can tell the difference between D, DD, and DDD. Bonus points if it can handle "higher" criteria.

The first real success for a long sought after AI

An old AI joke was to call a limited domain version of this a "cat box". The idea a camera with a light on it that comes on whenever it's pointing at a cat.

Image SPAM

It would be nice if we could humor these engineers to put as much effort into image spam filters :-P

Not exactly new

Their work seems to be based on Gaussian Mixture Models which have been around for two decades or so. It's not a very advanced method either and there are a bunch of better adaptive systems for image recognition, wavelet neural networks being an obvious example (and they've been around more than a decade).

There is absolutely nothing newsworthy about this. On the contrary, you'll find tons of similar works - mostly as senior year student projects in CS/AI.

Semantic Robot Vision Challenge at AAAI's

a robot challenge that will test robots' vision and language understanding [cmu.edu].
the robots/sobots must be able to recognise objects automatically and perform tasks like: get the "star trek" poster or get the blue dry erase marker. the final event will be held at the twenty-second AAAI conference on artificial intelligence [aaai.org] in vancouver, canada july 22-26 '07 [taken from ofpblog [skynet.be]]

sounds like tagging , not image search

I keep waiting for a real image search to be created without the intermediarry step of tagging it with text.
I'll be happy when I can tell the search page "find images like this" and give it an existing picture or a sketch. Tagging is too reliant on the consistant metadata to be useful in a general way. Humans can easily find all pictures of, say, fluffy the cat in a pile of photos from all different sources. Can we teach a computer how to do that without having to wait for it to re-tag images from different sources before it can search?
Still, the better methods we find for tagging, the closer we get to that I guess...

A combined solution might be useful

I recall seeing an image search a little while back where people tagged images manually, building up a weighted list of tags. It might be a good idea to use a system like that, to train a system like this. Like the spam filters we all know and love.

Using games to get lots of tags

One complaint about this work is that it requires tagging an initial set of images that are needed to train the system. Vasconcelos' work uses the academic standard "Corel" dataset of labeled images but also uses tagged images from Flickr to train the system. Using human computation games like the ESP game for images and ListenGame www.listengame.org [listengame.org]for audio, collecting data is not as tough as it once was...

if...

a picture's worth a thousand words, where are we going to store all the words for the useless myspace photo's that get archived?

How Many Non-techies Think This Isn't New?

It alwasy cracks me up on TV how they show a fingerprint or face recognition system searching for matches. The camera cuts to a computer that's looping away through a bunch of pictures until a match is found. Admittedly, I don't know all of the details, but obviously current systems must have some sort of indexed data points that are entered in a database and then you run queries against the database for potential matches.

It's a little more plausible now that broadband is readily available but this has been portrayed on TV for years. Can you imagine some podunk field office connecting to an FBI database through a dialup and downloading high resolution images until they found just the right one? Then again, that would make for some good entertainment. Detective walks in..."I've got good news and bad news. The good news is we found the killer. The bad news is, he died of old age."

I feel validated now...

...because my master's thesis, back in 1996, was using neural nets with a fuzzy logic component to identify surface features on Landsat satellite imagery. The algorithm I came up with was even scale invariant.

Guess I should have published and patented...damn...there goes any feelings of validation...

Haar wavelets

Nobody's mentioned Haar wavelets yet? Weird.

Look them up - they're part of OpenCV, and I'm pretty sure it's the same basic principles in action.

Useful application

This would be great for deviantArt, as one of the problems is mis-categorising their submissions. If a computer was able to help with that would make finding art of specific subjects/styles much easier.

Please bring it on!

demonoid.com, mininova.org or btjunkie.org?

I missed the link, where can I torrent this?

Finally!

Now I can search for porn stars that look like that girl in my Englsh class!

Cool! A Minnie Driver/Anne Hathaway love scene.

> The system calculates the probability that various objects it has been
> trained to recognize are present, and labels the images accordingly

"Ok, Joe. Let 'er rip on this new test database."

Cock
Cock
Cock
Vagina
Cock
Cock
Hairy armpit

"Oh, cool! The upgrade works and can distinguish it!"
"Nah, wait until you see this!"

Cock
Cock
Cock
Midget with banana split in hairy ass crack with guy eating the banana split without using his hands on the Howard Stern show
Cock
Vagina
etc.

can it help me sort p0rn faster?

So many boobs; so little time.

Re:Organizing my archive

tags

Re:Electrical engineers?

C'mon...get with the times.

The keyword here is 'footprint', and since the new everything is now software, electrical engineers, as an example, would have a need to reduce the heat and energy 'footprint' of a given piece of equipment used to run the software that accomplishes the hit.

If a hardware engineer can come up with a better search method via software that works quicker, reduces errors and further searching, uses a smaller processor, less heat, etc. then he/she has just reduced the energy footprint, regardless if he/she had to make the software that drove the smaller footprint or not.