Identifying Manipulated Images

Jamie found a cool story at MIT Tech Review. (As an aside, it sits behind an interstitial ad AND on 2 pages: normally I reject websites that do that, but it's a slow news day, so I'm letting it through.) Essentially, software is used to analyze light patterns in still photographs. Once you can figure out where the light sources are, it becomes a lot easier to determine if an image has been photoshopped.

Steganography

Does it also apply to steganography? Would sort of suck if it did.

Re:Steganography

*Somewhere in the middle of the NSA / MI6 buildings, a check mark is put next to an IP address.*

Detector == Quality Control

People who manipulate images will use these tools for quality control: When the fabrication passes all tests, it is ready to be released.

Re:Detector == Quality Control

Detecting discrepencies between light sources is something that can be written as an algorithm and should increase in complexity linearly with the product of the number of light sources and number of pixels. Determining the correct value for pixels is something that must be written as a herustic and will increase in complexity exponentially with the number of objects (including light sources and non-visible objects) in the system. A herustic is not guaranteed to finish. Ever. Even if it always did, because the problem is exponential, far more images can be checked than can be fixed up.

Everything is photo-shopped!

Duh! [xkcd.com]

No ads, all on one page

The printer-friendly version:

http://www.technologyreview.com/printer_friendly_article.aspx?id=20423 [technologyreview.com]

Re:No ads, all on one page

The link works fine if, instead of clicking on it, you cut-n-paste it into a new browser tab. Here's what you get, if you can't be arsed to go to the trouble:

Monday, March 17, 2008
Identifying Manipulated Images
New tools that analyze the lighting in images help spot tampering.
By Erica Naone

Photo-editing software gets more sophisticated all the time, allowing users to alter pictures in ways both fun and fraudulent. Last month, for example, a photo of Tibetan antelope roaming alongside a high-speed train was revealed to be a fake, according to the Wall Street Journal, after having been published by China's state-run news agency. Researchers are working on a variety of digital forensics tools, including those that analyze the lighting in an image, in hopes of making it easier to catch such manipulations.

Tools that analyze lighting are particularly useful because "lighting is hard to fake" without leaving a trace, says Micah Kimo Johnson, a researcher in the brain- and cognitive-sciences department at MIT, whose work includes designing tools for digital forensics. As a result, even frauds that look good to the naked eye are likely to contain inconsistencies that can be picked up by software.

Many fraudulent images are created by combining parts of two or more photographs into a single image. When the parts are combined, the combination can sometimes be spotted by variations in the lighting conditions within the image. An observant person might notice such variations, Johnson says; however, "people are pretty insensitive to lighting." Software tools are useful, he says, because they can help quantify lighting irregularities--they can give solid information during evaluations of images submitted as evidence in court, for example--and because they can analyze more complicated lighting conditions than the human eye can. Johnson notes that in many indoor environments, there are dozens of light sources, including lightbulbs and windows. Each light source contributes to the complexity of the overall lighting in the image.

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

Hany Farid, a professor of computer science at Dartmouth College, who collaborated with Johnson in designing the tool and is a leader in the field of digital forensics, says that "for tampering, there's no silver button." Different manipulations will be spotted by different tools, he points out. As a result, Farid says, there's a need for a variety of tools that can help experts detect manipulated images and can give a solid rationale for why those images have been flagged.

Neal Krawetz, who owns a computer consulting firm called Hacker Factor, presented his own image-analysis tools last month at the Black Hat 2008 conference in Washington, DC. Among his tools was one that looks for the light direction in an image. The tool focuses on an individual pixel and finds the lightest of the surrounding pixels. It assumes that light is coming from that direction, and it processes the image according to that assumption, color-coding it based on light sources. While the results are noisy, Krawetz says, they can be used to spot disparities in lighting. He says that his tool, which has not been peer-reviewed, is meant as an aid for average people who want to consider whether an image has been manipulated--for example, people curious about content that they find online.

Cynthia Baron, associate director of digital media programs at N

Colombo did this on his 1970's TV show

Someone wore a photo mask and tripped a speed camera to give their partner proof that they were across town (LA) at the time of the murder. He noticed the shadow under the nose was wrong by comparing previous and following pictures from the same camera.
I am not sure which episode it was. Peter Falk as Det. Lt. Colombo

Found the episode

Columbo and the Murder of a Rock Star, 1991 and yes Dabney Coleman was the bad guy.

Uh Oh

This bodes ill for all those geeks out there with "out-of-state" girlfriends!!

Goes both ways

...and then the photoshoppers will write evolutionary algorithms to modify their photographs until they pass evaluation by this tool.

Re:Goes both ways

Yeah, but do they blend?

weak

this method is way better

Forensic Analysis Reveals Al-Qaeda's Image Doctoring [slashdot.org]

Limited utillity

In a studio or other arranged settings it's pretty standard to use multiple lighting sources. So this tool will mainly be usefull for outdoor settings. If it's up-close and personal then it's also very common to use lights or other tools outside. Sooo this tool should be used with moderation.

Adds a step for the photoshoppers

One would think that it would be simple enough, after finishing whatever touch-ups that you want to perform, that you use this technique to calculate where the light sources should be, and then correct the minute details that would give it away as an altered image. Sounds like the kind of thing that would be a simple photoshop plugin actually, once you are all done you just run the "make undetectable from light source detection analysis" tool and call it a day.

Finding Photoshopped Pics for Fun

Does anyone else have a habit at looking at pictures and trying to see how they've been manipulated? These types of pictures are rampant in advertising. Pick up any magazine and start looking, and the poorly edited pictures will jump out quickly. The more professionally edited pictures have much more subtle problems, and can take a bit of poring over to find. Many product images (on packaging and in catalogs) are the same way, and are usually the worst edited of the bunch. Some things I look for:

• An object rubber-stamped in multiple places. Each copy is identical, which gives it away. They are often scaled, rotated or mirrored to make them look more unique.
  
• Lighting and shadows, which is what the algorithm in this story deals with specifically.
  
• Focus. Often multiple objects will be in focus at varying distances impossible with a single shot.
  
• The same image of a person is used in multiple shots. This is most prevalent in product images in catalogs.
  
• Poor masking, where edges of objects are over or under processed, either clipping part of the object (hair can be particularly tough to do), or showing some color edges from the original background.
  

Anyway, that's just the geek in me I guess, because I really do enjoy finding flaws in images. What I hate is an image that has a sort of surreal perfection to it that I know must be composited, but I can't find any smoking gun.

Apollo

I love how the first comment is asking if the apollo landing photos were photoshopped.

Re:Apollo

I love how the first comment is asking if the apollo landing photos were photoshopped.

Photoshop was HUGE in 69. Huge.

speaking of which

did anyone else catch the blog in the new york times about the fenton photographs [wikipedia.org]?

apparently this guy took some photos of some cannonballs in the crimean war that became famous as a poetic commentary on war. this documentary filmmaker, errol morris [wikipedia.org], has gone completely unhinged obsessive compulsive over whether or not the photos are fake and/ or manipulated. it's utterly fascinating, and a little weird, to see so much time and effort devoted to these photos. specifically, cannons and shadows. utterly esoteric and thorough. he also expands into the larger topic of the history of manipulated politically sensitive photos. makes for a good read, especially if you are interested in pre-photoshop image manipulation

check it out, talk about thorough [nytimes.com]

Re:Expert User Required

TFA says an "expert user" is required. This expert user inputs coefficients that drive the equations that analyze the picture.

So basically, if you want an image to be doctored, you use one set of values. If you want an image to be genuine, you use another set of values. Maybe somebody else's requirements differ from mine, but this is not the kind of flexibility I want in a tool that is supposed to tell me if an image has been altered or not.

Ummm, what? FTA:

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

I mean, that's not even close to what you posted - "running the same analysis on different parts of the image and then comparing the results" is not the same as "you pick the results".

Re:Good

It just sounds wrong to say that an image has been "GIMPed".

Actually, now that I think about it, I kinda like it...

Re:Good

That's why I say "to gimp a photo" rather than say "to photoshop a photo". It spreads awareness, breaks the Adobe monopoly, and sounds more natural. Even Adobe discourages the use of "photoshop" as a verb.

Re:Good

Even Adobe discourages the use of "photoshop" as a verb.

You know why, right? If they let "photoshop" be corrupted in the language as a verb, they would eventually lose the trademark rights to the name. Eventually, companies would be able to get away with naming their software "MS Photoshopping Program" or "Gimp Photoshop Utility" or whatever, and Adobe wouldn't be able to do anything about it.

Are you sure now, that you want to discourage people from using "photoshop" as a verb?

Re:That should help

It's not needed and won't help. Most of the UFO photos are pre-Photoshop and were done with different methods:

• Have a small model of the UFO and fling it into the air high enough that there's no context. Although those CAN be detected, they can't by this software.
  
• The objects are secret military aircraft, not alien craft. The hoax of alien craft is started by the government (pick one) to mask the true meaning of the object photoed. This software won't help with that, either
  
• It's something else flying around up there. Is it a bird? Is it a plane? Is it a weather balloon? Is it ball lightning? Who knows? If it's a flying thing and you don't know what it is, then it's an Unidentified Flying Object. This tool won't help here, either.

This tool can't do anything someone trained in art can't do. The first thing you learn in art school is how to see. You can't draw if you can't see, and that's usually the biggest reason most people can't draw.

As one of my instructors used to say, "I don't know what I like but I know what art is."

-mcgrew

Re:"photoshopped"?

You could probably find out if you googled it.