Image Recognition on Mobile Phones

mysticalgremlin writes "In a recent presentation, Semacode founder Simon Woodside presents his company's bar code scanning technology that is used in mobile phones. Simon also discusses many places where bar code scanning powered phones are being used. Not bad for an 'image recognizer for a 100 MHz mobile phone processor with 1 MB heap, 320x240 image, on a poorly-optimized Java stack'"

Lookup Required

[hey]

Once a barcode is read you just get the product code. What good is that?
You need then to lookup that code up in a database for real info.

Re:Other uses

[mehu]

If your mobile phone can read barcodes, we could print them anywhere - in papers, on billboards, TV adverts - and all you'd need to do is take a photo and your phone automatically loads the webpage in its built-in browser.

This is already standard in Japan- barcode readers come on pretty much every cell phone here. They read special 2d-matrix barcodes that look like this [wikipedia.org], which generally encode a URL or email address. You don't even need to take a picture of it in the usual sense- you run a little app called "barcode scanner" and just hold your phone over it, and as soon as it recognizes the barcode, it instantly launches the web browser or opens a new email with a specified To: address & possibly a predetermined Subject: line. They're often used on posters & product ads as a "get more info by scanning here" thing, or even to sign up for store memberships & things- hold your phone over the little square, and you instantly get a web page w/ a form to enter your info. Much faster than typing a URL on your phone.

Yet another area where Japanese cell phones are WAY ahead of the US...

Business Card Scanner

[Anonymous Coward]

The LG V phone has a business card scanner built in. If you take a picture of a business card, it can then use OCR to identify the text and add it to your address book. It's not perfect, but it's still pretty impressive (and you can edit out the mistakes). That's got to be harder than merely interpreting a bar code.

Not unique

[csirac]

I've seen some Japanese phones that have apparently had this ability for quite some time now, I was absolutely amazed when a friend showed me one that even OCR'd english text out of a snapshot!

And there's a company called Grabba [grabba.com] that makes commercial bar-code scanning solutions out of PDAs and PDA-phones (among other things). A friend of mine works there... interesting stuff; they also sell a dock thing that a PDA can clip into, which gives it a camera so you don't need to use a mobile phone. Popular with inventory/warehouse type applications, it also does 2D barcodes as well.

Re:Other uses

[JanneM]

However, I wonder whether this idea may have some re-interest. If your mobile phone can read barcodes, we could print them anywhere - in papers, on billboards, TV adverts - and all you'd need to do is take a photo and your phone automatically loads the webpage in its built-in browser.

Exactly this has been available and used everywhere in Japan for a few years already.

Re:Lookup Required

[TubeSteak]

They aren't trying to recognize 1-D barcodes (ya know, normal barcodes).

"It needs to locate and read two-dimensional barcodes"

Nowadays, PDF417 [wikipedia.org] is the standard for 2d barcodes.
http://www.barcodeman.com/faq/2dbarcode.gif [barcodeman.com]

It can store between 10 and a crapload of characters

A 320x240 image gives you plenty of characters, depending on how much redundancy you want to throw in.

Re:Not bad...

[RileyCR]

This is impressive. However this has been going on for a while. And not just barcodes. ABBYY for example has a complete SDK for barcode and full OCR. They use it to extract all bacodes and text on a cell phone, i believe currently support symbian and windows mobile.

Re:Not bad...

[Anonymous Coward]

Err ... I've been using my mobile phone's camera as a QR code (2D bar code) scanner on my SonyEriccsson from 3 years back, using the mobile phone's bundled app. Japan has this for years and it got to slashdot today?

Re:Not bad...

[phoebe]

Although Japan and Russia got there a bit quicker ... Intelcom provide a encoder and decoder Java toolkit [intelcom.ru] for mobile phones [intelcom.ru]. Japan have a Sourceforge Java project [sourceforge.jp] for encoding and decoding too.

Impressive? These guys here do it real-time.

[Anonymous Coward]

http://www.beetagg.com./ [www.beetagg.com] They got REAL-TIME 2d code detection, on wm5 and symbian (and treo650 seems to be coming soon) phones. Works on my Java phone (non-RT). Shame the Moto-Q is not on the list. Anybody care to test?

In Japan ... not only bar codes ... OCR as well

[mxpengin]

My phone ( which I have had for more than half a year ) besides the bar code reader, has OCR of roman and japanese characters. And the most impresive use of this in the telephone is the ability to input some japanese word (yes in Kanji) directly into the dictionary. Really impresive for us non native japanese speakers. My phone is a sanyo w32SA [sanyo-keitai.com] , in the link you can read about in the part OCR kino.

Re:Other uses

[zeropaper]

and there's also Semapedia.org who's using semacode to load content from wikipedia (related to a place) really nice idea

Re:Other uses

[mykdavies]

Actually I think that the Japanese system is QR-Code http://en.wikipedia.org/wiki/QR_Code [wikipedia.org]

Re:Bar code scanning powered phones?

[plover]

My favorite use for this would be to conduct instant price comparisons.

Be prepared to be ushered out of the store. Chain stores frequently use "secret shoppers" for price comparisons in their areas, and they used to use small handheld scanners for data entry. I bet cell phones are high on the list of inconspicuous tools now, though. Either way, if they're spotted they're shown the door.

Home Depot (and others) also have "No Cameras or Recording Devices" signs posted, so I'm sure they think they reserve the right to toss you out. That doesn't stop them from prominently featuring a shopper with a camera phone showing a ceiling fan to the waiting spouse at home on their TV commercials, though. I know I'd be calling a lawyer if they said "boo" to me about it.

Some envelope-back calculations

[Kadin2048]

Your comment got me thinking about how much information you could squeeze into one of those barcodes.

At most, a 320x240 tag would give you 76,800 bits of information, or slightly less than 11,000 7-bit ASCII characters. That's assuming you could match the pixels of the tag to the camera's sensor exactly.

I assume you probably wouldn't want to use any more than half of the camera's vertical and horizontal resolution though, which leaves you with 160x120 (for 2,700 characters), and I assume you'd need to have a few rows blacked-out on at least two sides to identify the border of the tag (so subtract (160+120)*2 pixels for bordering...that leaves 2662 characters) and you'd probably want to have a hash or checksum (lose 128 bits).

Still, that leaves about 2,643 characters in an image, which is about a page and a half of typed text using the old guideline of approximately 1,500 characters per page.

That's pretty impressive; provided you could make your reader focus on objects near to the lens, so that you could make the tag suitably small (less than an inch or so across), that's a lot more efficent way to convey textual information than actually writing it out. Instead of just embedding a URL link, you could put written information on there; maybe stuff that would clutter up the packaging / display / poster if you wrote it all down. If these things became ubiquitious, I could see whole advertising campaigns in urban areas (e.g. subways) where the "ad" got you interested, and then you could get more information via the tag.

They say a picture's worth a thousand words, and it sounds like it may not be far off from that.

Re:When barcodes came out ...

[ashirusnw]

Bear in mind the "virtual machine" on most phones is in fact simply a slow interpreter - it makes BASIC look souped-up.

Presumably you're referring to the KVM (the J2ME JVM) which is slow. I think you're out of date.

AFAIK modern phones have Sun's CLDC HotSpot VM (the "CLDC HI VM") which has speeds equivilent in relative terms to a JVM on a desktop PC. The Blackberry phones in particular have a great JVM. When more phones have decent ARM-based gigahertz processors speed Java speed will stop being an issue much like the desktop space.