PNG Group Unconcerned About Apple's Patent

melquiades writes: "A recent story raised concerns that Apple's patent on some forms of alpha compositing was blocking the development of PNG, MNG and SVG. Not so, says Greg Roelofs, a member of the PNG group: 'The PNG group did discuss the Apple patent several weeks ago, and we decided it was completely irrelevant to PNG itself, almost certainly irrelevant to the pnmtopng utility and to PNGs animated extension, MNG, and probably irrelevant to SVG as well.' Here's the article on OS Opinion. So if it's not a big deal, why was there a general call for prior art to overturn Apple's patent? It looks like some PNG developers got worried, but the core team thinks there's no problem. Is this just a case of the right hand not knowing that the left hand is paranoid?" Once bitten, twice shy?

If Apple cared...

...I'm sure they would have said *something* by now.

Leftover Paranoia

Its just leftover paranoia from the problem Unisys had with people using the GIF format. The PNG group seems to have a good understanding of the Apple patent, from what I read in the article.

PNG's

Does anyone really use that format? I haven't heard too much about it lately after it was supposed to be the next greatest format for pictures ever. Anyone out there using them?

Re:PNG's

Uh, sorry, what? Is anyone out there still *not* using PNG?

Re:PNG's

Sites that don't use PNG:
www.salon.com, www.userfriendly.org, yahoo.com, www.redhat.com, www.debian.org, www.gnu.org, and of course, www.slashdot.org.

So, who is using PNG?

Re:PNG's

I use PNG's everyday, as it is the file formate that firework's (macromedia) uses. It contains a "web layer" and when created in firworks you can create "slice guides" and rules. Similair to how a PSD contains layer. The only difference is, I can open a PNG in IE, though I wont see the layers, and it will be a REALLY big file, compared to what it would be if it was exported out as a gif or jpg or I guess a flat PNG.

Re:PNG's

Well, the main reasons why PNG hasn't replaced GIFs:

• People are slow in changing their habits; they are used to GIFs, they (still) use GIFs. If it ain't broken don't fix it (that's of course assuming you haven't been harassed by Unisys. :-))
• Some old browsers did not support PNGs at all. Shouldn't be a big problem nowadays, though (esp. since much of Javascript stuff in use would break them anyway)
• PNG doesn't have animation, for cheap-ass simple animation GIF is the choice (MNG would replace it, but it's bit of obscurity, being rather ambitious format project)
• Although PNG has really cool features (full alpha channel is _really_ nice for blending image borders, neat translucency effects etc), many browsers (NS 4.x series, IE on Windows at least until 5.5) still only implement 'GIF-level' features, ie. on/off transparency, no gamma correction etc. Mozilla has good support, and IE on Mac too (no idea why; usually windows version is more advanced).

Prior Art

Now, the call for prior art is a totally separate issue. Even though the PNG format is not threatened by the Apple patent, it would still be a good idea to come up with prior art and destroy the patent.

As I understand it...

It looks to me like the patent covers alpha-masking, not alpha-blending.

As I recall, PNG and SVG both use alpha channels. That is, transparency data is stored as a part of the image's color encoding.

Alpha-masking is different. In this case, an image is defined in the "normal" RGB fashion. Along with it, a second image is stored, which has only alpha information in it, not color. You could say that an alpha channel is inline, whereas an alpha mask is not.

Is the patent ridiculous? Of course it is. But I don't think it would affect PNG and SVG anyway, because they use a different method of storing transparency information.

Re:As I understand it...

I thought so too, after first glance at the patent. However, after reading it through second time, I understood what the patent really is about.

What it is is doing 'compontent-sensitive' image blending. Instead of using single alpha mask or channel for all colour components, it uses a full mask image, that is then used for blending images component by component. When mask image is a gray-scale image, this effectively degenerates to 'normal' alpha blending. However, when using non-gray colours, blending is not linear (as with alpha blending) for the components. In RGB, for example, you could take red value from source, green from destination, and blend blue 50-50 from both, and get... um... probably interesting results?

What I would like to know is if this is useful? What kind of effects can be achieved by using non-grayscale mask images? Potentially it might produce interesting effects... But are those just curiosities?

... and no, obviously this doesn't cause any problems for PNG, which 'only' uses embedded alpha channels.

The actual forum post

The OSOpinion article refers to Greg Roelofs'
forum post [linuxtoday.com]. My favorite part of the Roelofs' post is this classic line:

" So I nominate this for the 2001 tempest-in-a-teapot award... But since the /. crowd lives for this stuff, I wouldn't presume to imagine that my comments will have any discernable effect on the (flame)festivities. :-) "

How many posts did that article get the other day? :)

<SIGH> another one for the courts...

Unfortunately, this, like many patent issues, can't be decided except by a court. Just because someone says "we don't infringe on Apple's patent", that doesn't stop Apple from suing or a judge from ruling that it does infringe.

That's why the call for prior art goes on. Deny that Apple has a case, but build a strong defense against their case. It's the cover-your-bases approach, and, sadly, the only wise one available in the current intellectual property age.

That's an utterly ridiculous attitude.

While we're at it, let's spend all our time and effort preparing to defend PNG against patents on automated shoe manufacturing and quack magnetic therapy devices. After all, "only a court can decide" whether PNG software infringes on those, too.

The patent itself specifically contrasts what it covers with the use of an alpha channel, such as is used by PNG. It is obviously not a threat to anyone who bothers to read the patent.

Aside from that, PNG was designed to avoid patent issues by people who knew what they were doing, in an drawn-out open process where anyone could have pointed out weaknesses. It's based on well-established techniques with a long history of prior art. Your first assumption should be that any patent either doesn't affect PNG, or is so blatantly invalid that nobody would ever dare take it to court, not that there's trouble ahead because a one-line description of a patent sounds vaguely like something PNG does.

You can't live your life shrieking in terror every time someone whispers "patent."

PNG groups opinion isn't particularly relevent

What's actually relevent is what Apple's lawyers think. It's going to be expensive to fight a legal battle, so expensive that anyone who's been harassed by Unisys has folded, even though there is a fair chance that their patent would be declared invalid if it came up in court.

Apple Unconcerned too?

A while back Apple posted this [apple.com] to their web site. It seems they are against RAND licensing fees. I assume that would also include their own technology, but I am really not sure how all this fits together.

Article on Linux Today

Here [linuxtoday.com] is an article about this issue on LinuxToday [linuxtoday.com].

Web browsers may be at risk

Apparently the patent only covers compositing of images that start from several sources. This is all very well for PNG images, as all the data is stored in one file. However, once the PNG is in a web browser, it may get composited with other images from another source. If the page has a background image and an alpha'd PNG on top of it, surely that would then infringe the patent?

Probably not a problem for Microsoft with IE - they would probably just have a little patent fight with Apple - but for free browsers this could be a problem.

There's a few [google.com] interesting [google.com] threads [google.com] on [google.com]
usenet [google.com] about it as well.

it's relevant all right

Unfortunately, the article referenced does not seem to give any details for why people think this isn't "relevant"; there is no analysis of the claims.

Apple itself identified the patent as relevant to SVG and at some point asked for non-discriminatory royalty payments. They apparently have backed off from those requirements by now, but Apple still hasn't dedicated the patent to the public domain, leaving some legal uncertainty hanging over open source projects.

If the Apple patent were valid, I think it could be a problem and might have to be overturned. In that sense, it is "relevant", and it makes a lot of sense to collect prior art.

It's also relevant in a different sense: it tells us about what Apple is up to, what their attitude is towards free and open source software, and the intellectual and research standards at the company.

is a call to prior art a bad thing?

So if it's not a big deal, why was there a general call for prior art to overturn Apple's patent?

is there a problem with this? with the average IQ of a patent lawyer at about 35 these days, you can't be too careful. mark my words, the future of Open Source depends upon it.

Why, you ask?

From the the article on OS Opinion: (my emphasis)

Earlier this year, the World Wide Web Consortium (W3C) asked interested parties to submit information about patents they own that

might potentially infringe upon the development of scalable vector graphics (SVG).

So Apple reported one of their patents that could in the future conflict with the development of SVG. Sombody read that, combined the two evil buzz-words "Apple" and "patent", and paranoia kicked in. He then scanned through the patent, found "alpha channel", and thought his worst fears had come true. So he called up some buddies and started the usual anti "evil/stupid patent" actions.

All in all, simply self induced FUD.

Noninfringement Opinion?

I know this is Open Source, but I generally thought that most Open Source software was better engineered than much of the commercial stuff out there. One of the key components in a feasibility study BEFORE other development work begins is legal feasibility . If you don't believe me, dust off your copies of Pressman's Software Engineering and see what HE says. I'm not just making this up.

Part of a legal feasibility study has to be a patent infringement analysis. Whether you agree kthat software should be patentable or not, the fact it that in the US it IS patentable. If someone else made it first and was granted the exclusive right to make that product, then you will infringe on their rights if you make the product.

There is simply no excuse for not doing this analysis. Yes, there is some up-front cost to a patent attorney. However, with a large enough development group, this cost can easily be absorbed by the group at a nominal level per member. The potential consequence is that every developer can find themselves defending a patent infringement suit. The last estimate of which I am aware placed the cost of litigating an "average" infringement suit at just around $500,000.00. If the case is deemed "exceptional," the infringers could potentially have to pay the patentee's legal costs as well as their own, on top of paying damages for the patentee's lost profits.

Bottom line: all this can be avoided by getting a noninfringement opinion by a reputable and competent patent attorney. This is a basic software engineering step and should be performed at the very beginning of the development process. The attorney will even be able to suggest ways you can still go forward with the project, but do so in a way that does not infringe on someone else's rights.

keep cool

Don't worry, everything is fine.
Nothing's wrong with the patent

...

At least not for several years until the PNG-format is used all over the internet and apple can make a fortune by taking it out of the drawer.

It's a false patent, if there was prior existance.

The idea of a patent, is for the inventor of a technology, idea, whatever to have rights over it for 20 years.

We all know Apple wasn't the first, and therefor they shouldn't have the patent. I'm not worried about it either, but I would like another stupid patent overturned.

Most relevant I say

This shows what the slashdot community thinks of software patents. I agree that they need to go, kinda, but for slightly different reasons. But what they do today is only to hinder development, forcing new names and similar techniques just to stay out of trouble for things that I think should be free for anyone to use. Isn't there a better way to pay inventors (because they should be paid I think) instead of this horrible system.

Socialistic capitalism ;)

An Anti-Software-Patent Database

One way that the Open Source community could help fight software patents is to establish a database of prior art. When issues like this come up, relavant prior art would be hyperlinked to the supposed patent.

Prior Art

So if it's not a big deal, why was there a general call for prior art to overturn Apple's patent?

The big deal with this patent is not its implications for PNG, MNG, or SVG - it's the abundance of *published* prior art, from way earlier than the patent was applied for. It's a blatantly obvious case of the USPTO not doing its homework.

People just don't get patents

Patents aren't what you think they are. All patents make broad claims that can not be defended in court. The patent writer knows this -- they are really trying to patent something that is specific and unique that other people aren't doing.

There are lots of patents that discuss alpha blending. They don't affect PNG.

The important thing to remember about patents is "don't go looking for trouble". Don't worry about new patents that come out -- wait until the patent owner comes looking for you. If you look far enough in the patent database, you will certianly find patents that can be interpreted to cover your idea. Unless you bring it to their attention, or are a direct competitor, most patent owners won't care about you. Most companies view patents as a defensive play -- they don't really care about them, they just grab the patents so that if a competitor sues them for patent infringement, they can sue back. E.g. Via countersuing Intel on patents.

Typical obvious patent

If you read the patent, it's an almost textbook example of a painfully obvious 'invention'. The patent clearly admits that the use of a special transparent 'color' in one image is an old technique for blending two images. Next it admits that using a 1-bit alpha mask is a known technique for combining two images. Then it admits that an 8-bit alpha mask has been used to smoothly blend two images in the compositing process! Finally, it admits the 'invention': Let's use a full color image instead of a grayscale image for the alpha mask! Wow! How do they think of these things! What a genius it takes to think that maybe after first using black and white masks, then greyscale, you might try color!

The real clincher is that they go on to list all of these wonderous things you can do with full color mask. But most of their examples only require a good old fasioned greyscale map! Somebody at Apple pulled a fast one at the patent office. The entire 'Fig 1.' shows an example which doesn't even use Apple's 'invention'! It just uses good old fashioned greyscale alpha masks which Apple lists as prior art! Then two of the claims list anti-aliased text! Does anybody do this with color alpha masks? No font renderer spits out multi-color images! They spit out greyscale which is used to merge the text color/pattern with the background! Once again, they claim their invention is needed to do something which everyone does with the prior art method!

On the plus side, it's such a silly invention it's trivial to work around. Here's an idea: use three different alpha masks! One per channel! I wonder if Apple managed to get a patent on that. Hey, here's another nifty idea! Generate the three alpha masks from the three channels of a color image! I bet Apple never thought of that! Only a genius like me could think of that. What an invention! I'll make millions!

Man, this patent is the worst. Check out Claim 3: A method as in claim 1 further comprising displaying the result image. Whoh Nellie! Your going to display the image! Man, that's adds a whole new level of complexity. Usually, I composite images and then just free the memory. But actually displaying an image you've generated! Man, what incredible insight!

Nothing to fear but fear itself

From the article:

"Fear Itself

Misinformation on this matter has caused many open-source advocates to incorrectly accuse Apple of holding back Open Web technologies.

Let's hope that cooler heads prevail and that fear-debunking articles such as this one are more widely read."

It is amazing how worked up everyone got in November when they learned that a company that supports web standards and has a public policy of royalty-free licensing for them (as of October) had responded in June to a call to disclose all patents to the PNG group that they should be aware of for infringement issues in the future.

No one answered melquiades's question

If Apple's patent is valid, nothing put out by the PNG group infringes. However, when an application, say, Mozilla, displays a PNG in front of some other graphic and uses the PNG's alpha mask to do partial transparency, that would infringe. Similarly, when a Gnome or KDE desktop displays a PNG icon and uses the partial transparency to have a smooth and not a jagged edge, that would infringe. But just storing the alpha mask in the file doesn't collide with anything in the patent.

In practice, there's so much prior art that the patent will be dead meat once it hits a court. Traditionally, the patent holder deals with this by licensing for a fee small enough that it's cheaper to pay than fight, and in this case, perhaps giving an out to the free software folks, while possibly trying to get a bit of cash from others. But in this case, I suspect that even that strategy is not going to work.

GIMP influenced?

Well, the patent most certainly looks like a standard mask-like image manipulation. So I understand people to get paranoid about this (given that PNG uses a mask for its alpha transparancy). I still doubt if the GIMP wouldn't be influenced by this patent.

Re:DO NOT TRUST APPLE- you cant predict the future

IANAL, and my knowledge of patent law is spotty, but I'm pretty sure that it's legal alright. What you're thinking of is perhaps if the patent holder actively tried to encourage infringement (e.g. by implying that the patent wouldn't be enforced, or that licenses would be granted) which could result in the holder being required to grant a license to the infringer.

I'm fairly sure that passively waiting for someone to infringe is A-OK, however. Do you have some links or such that you could post to support your position? I'd be interested to see if my understanding is correct or not.