New Photo Fraud Detection Software

An anonymous reader writes "CNet is reporting that Hany Farid, Professor of Computer Science and applied mathematics at Darthmouth College, has developed a new version of his Image Science Group's photo fraud software now in use by the FBI and large media organizations. The current software is written in Matlab, but the new version will be written in Java making it much more readily available to local police and smaller media organizations. From the article: 'I hope to have a beta out in the next six months,' Farid said. 'Right now, you need someone who is reasonably well-trained to use it.'"

but...

How do we know the fraud detection software isn't fraudulent? :D

smash.

Very good idea, but...

Can we adapt it to detect Slashdot article fraud?

open source?

"Will you be able to get a copy of the Java-based version of the Image Science Group's applications? Probably not. One of the dilemmas of this type of software is that the more widespread the distribution, the more chance forgers will exploit it to their advantage."

followed by -

"...the software will be made freely available under an open-source license.
--
"Taxpayers," he said, "are paying me to do this research and it needs to go back out." "

Which is it?

Re:open source?

Those aren't necessarily mutually exclusive. He said access will be limited, most likely to police, FBI and other government agencies. But if those agencies want to enhance it, or make it better, they'll be able to do so under the open source license. The license may have a clause that limits who can get access to the software, but for those who are able to access it, typical OS rights are given to them.

Or he might not know what he's talking about, and/or wanted to use the term "open source" for good free publicity.

Re:open source?

Negative. The GPL doesn't have such a clause, but it is easy enough to modify the GPL (or any other license) to allow distribution of the source with the binary, and then restrict distribution of both. Granted, you may be using a different definition of the word 'open' than the software owner is. This scheme is more open than a closed-source solution where I sell you a binary with rights only to use the software and absolutely no rights to the source. While it may offend your OSS senses, it is, I think, a valid use of the term open source.

If you really want to take the term 'open source' to the extreme, I could argue that even the GPL fails to meet some level of openness. The GPL restricts use of its source code on several accounts.

Re:open source?

Or he might not know what he's talking about, and/or wanted to use the term "open source" for good free publicity.

Odds are something got lost in translation. I met the guy a few years back and he's quite sharp and very nice and unpretentious. He gave me a copy of the paper this work is based on. I thought at the time he should commercialize it. Open source would be even better.

Anyway, the paper was published and an algorithm should be able to be implemented by anybody with the appropriate skills. So, somebody could do a GPL version even if he doesn't.

The company I was with at the time wasn't smart enough to accept his offer to collaborate on some research. Just as well for him, I say.

This article likes to contradict itself

As pointed out earlier, apparently the source code won't be released but it is open-source. Interesting.

Anyways, also FTFA:

Still, around 1 percent of accepted articles contain manipulated images that do significantly affect the results, said executive editor Mike Rossner. Those papers get rejected.

So do they get accepted or rejected?

Same shot from a different angle...

Note that the submitter identifies the product as "photo fraud software" not "photo fraud detection software." This is quite apt, since the application will obviously cut both ways. Someone cooking a photo could simply run each version through this software, making minor tweaks until their "improvements" pass its inspection. If the software works the way it appears to, it would be the image manipulator's equivalent of a spell checker.

Outsource..

Now the CIA can outsource 1/2 their work... :P If we hack it into a video fraud detection package we can effectively get rid of the CIA all together...

Why Soviet Russia would love to go digital

In capitalist west scientist detects your airbrush fraud.

In Soviet Union you airbrush scientist out of photo.

http://en.wikipedia.org/wiki/Forced_disappearance# Soviet_Union [wikipedia.org]

The fun a government can now have with this package will be great.
False positives to kill a story?
Could real digital "abusing prisoners" images now be spun as a hoax?
Just a few well placed reports as to the authenticity of any new digital images could kill a story?
Or lure a leaker out to 'prove' the reality of the images only to face character assassination?

Fallibility

From the article:

"Right now, you need someone who is reasonably well-trained to use it."

I would like to hope that if this software is going to be used for anything of consequence, that someone reasonably well-trained will always be using it. A system is only as good as its operator, ultimately.

It's a fraud, because my blackbox software says so

As other commenters have already pointed out, the confusion over its open-source nature(is it or is it not?) is critical. Without the source code/algorithm being open-source and freely accessible to the public how can one trust its "judgement"? In a legal situation, an accused will always question the accuracy of the algorithm and the software.

On a different angle, I wonder how soon before such detection capabilities will be available to consumers either as an installable plugin or web-based feature. Imagine being able to verify the authenticity of any picture on the web, ranging from that nude shot of your ex- to that impossibly perfect low-light picture taken by your photography class buddy ;-)

Re:It's a fraud, because my blackbox software says

What you're saying is a rehash of the argument that "Information security isn't secure when the algorithm's security depends on its secrecy".

Thing is, does the same thing hold true when you're talking about detecting fakes (say), as opposed to building strong encryption? If I announce "Well, we can tell this photo isn't genuine because this part which shouldn't be in focus is", I've effectively announced to any potential fraudsters who might be listening "OK, folks, you need to learn to get your focusing correct".

Realistically, the only way such an algorithm remains secure is if it cannot be beaten even with a full understanding of how it works - and I would ask if such an algorithm even exists yet. If the algorithm is anything less than 100% effective, chances are it doesn't.

It works!!

It determined that Pamela Anderson's boobs are FAKE!!
=D

what could go wrong?

Rewrite software written in powerful mathematical simulation software in java?
What could possibly go wrong?

And now, rather than processing an image in 30 minutes, it takes 30 hours, yay!

Re:what could go wrong?

"...What could possibly go wrong?..."

Well, memory leaks and array bounds probably won't go wrong ;-)

Looking at some benchmarks [idiom.com] for numerical processing using Java, it appears to stack up quite well agains C++ at least.

Yeah I know, what exactly is being measured, are the benchmarks relevant, are any benchmarks relevant, blah blah blah. Just pointing out that the parent's postulated x60 slowdown is a trifle pessimistic.

T&K

In related news...

In related news NASA announced today that it would close its public picture archives ;)

A Java Version

So - when can I expect to see this as a Firefox Extension?

"Warning: This nude of Britinet Spears has been photoshopped"

Pug

It'll only really get fun ...

... when it's come full circle: Photo manipulation SW gets so good that it can fool the photo manipulation detection SW every time, so you're going to need human specialists to detect the kinds of manipulation that SW can't. Just hope the jury hasn't been replaced by a 12-member P2P system by then.

Images used for testing

Apparently they consulted with FARK.com's [fark.com] talented staff of professional photo retouchers while testing the software.

Moon landing (you knew it was coming!)

I wonder what would happen if the software was used to process photos from the original Apollo moon landing. I am of the belief that there is a great possibility that at least the original Apollo landing was faked. Subsequent missions I am not too sure about, but I believe at least the first one was a fraud. I wrote this page [mrnaz.com] up many years ago when I was in 10th grade at school. After reading it again just now, however, I really think I need to update it for new facts I've discovered and general maturity of presentation :)

$MATLAB/bin/mcc

If it's a matlab program, a C version is available now.

A matter of time

Just a matter of time before this application gets it's brother: "the simulator".

It uses the same algorithms in a slightly different way: instead of checking for the signs of forgery it finds the tell-tale signs of modification and then reverse-modifies them to "what-should-be-there" to make an "original" modified image.

The result will be an image that is ofcourse different only from mathematical standpoint - visual information will be the same. If that wouldn't be true I would love to have an application that "unblurs" or "unblackouts" the censored parts of some pictures.

Image will have after processing the properties of an "original" because the signs of "not-original" will be detected and "fixed". Way to go... :D

how effective would this be?

Photos have gradual changes in color (even when there are abrupt changes in the scene being photographed) and edits don't. But if people are trying to produce edited photos that need to be undetectable, it should be possible to write a filter that fixes edited photos. To do the equivalent of taking a photo of the edited photo. It's easy to seek for someone who isn't hiding. When they try to hide, it's a bit tougher.

Government document redaction

I wonder if the software could be modified to easily uncover the blacked out parts prevelant in many government documents. I suspect they would put up a big stink about this though.

When will this software be released

on a Warez site?

Republicans would never do that!

The light in the famous doctored photo that puts Sen. John Kerry next to actress Jane Fonda at a protest rally actually comes from two different directions.

"The lighting is off by 40 degrees," Farid said. "We are insensitive to it, but computers detect it."

Well even if that one is fake, at least we know that the one of John Kerry french-kissing Joseph Stalin is real.

Dupe!

I feel like an old curmudgeon here, shaking my fist and shouting "dupe! [slashdot.org]".

Sigh.

How long 'til used on Oswald-with-gun photo?

I wonder if/when it will be used on the Oswald-with-gun photo that many (apparently starting with Oswald) claim was faked.

The photo appears at the start of this wikipedia article on Lee Harvey Oswald.

(Of course the article is the subject to disputes of its own. B-) )

MMMM. My first test to beat this would be.

Once you have the comp in done in your image editor with as little pixel and grain distortions as possible. Try the following. Note this does require a $5000 set of plugins http://www.thefoundry.co.uk/ [thefoundry.co.uk] Furnace (no not the After deffects set) and the $5000 program them run in. http://www.eyeonline.com/ [eyeonline.com] or http://www.apple.com/shake [apple.com] http://www.d2software.com/ [d2software.com] (and yes a few others in the $100k+ range) In fact I might even do the whole comp in my film compositor with the use of some other tools. Anyway. Comp elements use historgram matching to match elements in the shot that should have the same color ranges. (You could do a color overlay in PS.) Ok now you have a good comp with good edges or edge blending and light wraps. You completely degrain the shot with furnace (each element seperatly degrained) You then regrain the shot as a whole. degraining and regraining work on all three color plates seperatly so when you regrain the shot it should be adding basically another blending of the of the colors making them uniform to what the original piece of film would have been. Now you take that file and do a film out of that. That is a very simlified break down of the technique. Many of the steps for each step are left out. My basic assumption is that the software looks for irregularities in the pixels deformations, areas of transition, and color offsets in the comp. I would love to go up against this software. (not that I would win but it would be fun to try as long as someone else is paying for the film outs and scans :-).) OH and if your source and destination or supposed to be film you probably want a drum scan not a CCD scan.

If you actually read the article

It appears that the program also detects software bloat -- it found unnecessary manipulations with some
Big_Foot_Prints.

surprised it's not mentioned: OJ covers

Another example not mentioned is the OJ Simpson mugshot photos that were on the cover of Time and Newsweek at the same time, where they were the same photo but one adjusted the contrast/gamma so it was darker, more sinister. Not exactly a touchup/photoshopping but still a modification done on purpose to skew interpretations.

I seem to recall some US guvmint propaganda as well where the picture showed a large group at a speech, denoting good turnout and agreement, yet when you looked closely you saw the same block of people three times. That's why they call it propaganda, right?

Re:please please please

So there's some big birds in the pics....
Now what?

You may like to read this

"Basically the famous Playboy article said that the feds deliverred UNSOLICITED kiddie porn to peoples homes and if they did not IMMEDIATELY report it to the feds (as if they open their mail that hour or minute) the gestapo feds kicked down your door and arrested the duped person"

http://www.eff.org/legal/cases/AABBS_Thomases_Memp his/ [eff.org]

He ran a BBS that was legal in California. A prosectutor in Tennessee sent him child porn by post in order to use Federal law to have him arrested (for receiving child porn through the post). He was then arrested and taken to Tennessee, charged with selling porn to Tennessee residents and the child porn charge dropped.