Calculation of the illumination of a point for PNG image, taking into account transparency

[lisichkin_alexander 0]

Hello!

It is required to calculate the illumination of a point in a TPNGImage object, taking into account transparency. I wrote the following algorithm:

function GetBrightness(const AImage: TPNGImage; AX, AY: Integer): Integer;
var
  LColor, LTransparentColor: TColor;
  r, g, b, alpha: Byte;
  tr, tg, tb: Byte;
  LDstAlpha: pByteArray;
begin
  LColor := AImage.Pixels[AX, AY];
  r := Byte(LColor);
  g := Byte(LColor shr 8);
  b := Byte(LColor shr 16);
  if AImage.Transparent and (LColor <> 0) then
  begin
    if (AImage.Header.ColorType in [COLOR_RGBALPHA, COLOR_GRAYSCALEALPHA]) then
    begin
      LDstAlpha := AImage.AlphaScanline[AY];
      if Assigned(LDstAlpha) then
      begin
        alpha := LDstAlpha[AX];
        r := Byte(Round(r * alpha/255));
        g := Byte(Round(g * alpha/255));
        b := Byte(Round(b * alpha/255));
      end;
    end else
    if AImage.Header.ColorType = COLOR_PALETTE then
    begin
      LTransparentColor := AImage.TransparentColor;
      tr := Byte(LTransparentColor);
      tg := Byte(LTransparentColor shr 8);
      tb := Byte(LTransparentColor shr 16);
      r := Byte(Round(r * tr/255));
      g := Byte(Round(g * tg/255));
      b := Byte(Round(b * tb/255));
    end;
  end;
  Result := Round(0.3*r + 0.59*g + 0.11*b);
end;

Please analyze my algorithm for errors.

P.S. Unfortunately, in my test base there are no images with TransparentColor - only with alpha channel.

Alexander.

 

[Bill Meyer 333]

Suggest learning the terms used in dealing with video images. Illumination refers to light striking a surface. You are dealing with the light a user will perceive from the screen.
See: https://en.wikipedia.org/wiki/Luminance

It won't affect your mathematics, but will help people understand properly what you want to achieve.

 

Edited December 17, 2022 by Bill Meyer

[lisichkin_alexander 0]

l have to explain why I started all this 🙂

 

There is a Web service, written in Java, that we use to convert and compress images to Png.
It was noticed that this service does not correctly convert colors for some Png images.
I was given the task of building an algorithm that would determine that two pictures are identical.
Moreover, with the condition, that colors are identical - the images themselves are considered the same.
I came up with the idea of comparing the illumination histograms of two pictures.
And this algorithm worked well, until I find some images with transparency - for which I did not correctly determine the color of some points.

The question can be reformulated: how to get the R G B color of a point, taking into account transparency.

Edited December 17, 2022 by lisichkin_alexander

[PeterBelow 218]

1 hour ago, lisichkin_alexander said:

l have to explain why I started all this 🙂

 

There is a Web service, written in Java, that we use to convert and compress images to Png.
It was noticed that this service does not correctly convert colors for some Png images.
I was given the task of building an algorithm that would determine that two pictures are identical.
Moreover, with the condition, that colors are identical - the images themselves are considered the same.
I came up with the idea of comparing the illumination histograms of two pictures.
And this algorithm worked well, until I find some images with transparency - for which I did not correctly determine the color of some points.

The question can be reformulated: how to get the R G B color of a point, taking into account transparency.

You cannot, since the pixel color in question depends on the background the image is rendered on. Just replace transparent pixels with a default color for the comparison, e.g. black.

[Anders Melander 1639]

Cross post from: https://stackoverflow.com/questions/74759895/calculation-of-the-illumination-of-a-point-for-png-image-taking-into-account-tr

 

I don't think your handling of COLOR_PALETTE is correct. AImage.TransparentColor specifies a color that should be considered fully transparent (i.e. alpha = 0). It does not contain the alpha of the RGB channels.

 

For performance, I would do the Alpha/255 once instead of 3 times:

alpha := LDstAlpha[AX] / 255;
r := Byte(Round(r * alpha));
g := Byte(Round(g * alpha));
b := Byte(Round(b * alpha));

or for much better precision and even better performance:

alpha := LDstAlpha[AX];
...
Result := Round(0.3 / 255 * r * alpha + 0.59 / 255 * g * alpha + 0.11 / 255 * b * alpha);

 

Apart from that, I don't see how comparing the YUV luminance of colors would help you determine if the colors are the same. Two (subjectively) different colors can have the exact same luminance. For example, here's a red circle in grayscale: here's a green one: and here's a rainbow-colored one:

 

22 hours ago, PeterBelow said:

Just replace transparent pixels with a default color for the comparison, e.g. black.

That only makes sense for fully transparent pixels. For alpha transparency, which is what most PNGs use, alpha premultiplying the RGB makes a little more sense. Not a lot of sense, because there's no "right way" to compare an opaque and a semitransparent color. It's really a question of how one chooses to define equality here.

[vfbb 275]

If you just want to check if 2 images are identical or similar, I have an optimal solution:

 

The Skia4Delphi unit test has an independent unit of image hashing (or similarity hashing).

 

https://github.com/skia4delphi/skia4delphi/blob/main/Tests/Source/Skia.Tests.Foundation.ImageHash.pas

 

Basically, this class allows you to:

- Compare 2 images and return the similarity between them (0..1)
- Generate a hash of the image
- Compare an image with a previously generated hash and return the similarity (0..1)

 

Note: It may seem strange but the hash it generates is a hash itself for similarity, that is, if you change part of the image, only part of the hash will be changed. Internally it implements 3 known similarity algorithms (like perceptual hash) and one of our own. The union of the 4 algorithms is to increase the final accuracy.

 

Logically there is a margin of error, but the accuracy is excellent. You can put for example a check to see if the hash similarity is >= 0.99 to decide if they are equal.

 

In the class, the input parameters of the images are of type ISkImage, which can be created in two different ways:

 

- TSkImage.MakeFromEncoded(Bytes)
- Bitmap.ToSkImage;

 

Note: Add the units Skia and Skia.Vcl (or Skia.FMX), to use the example above.

Edited December 19, 2022 by vfbb

[Anders Melander 1639]

1 hour ago, vfbb said:

I have an optimal solution

Optimal?

 

How do you define similarity optimally? In fact, how can you even define similarity=0?

 

I would say that similarity very much depends on what you need it for. A picture of a red apple and a yellow banana is "similar" because they are both pictures of fruits.

[vfbb 275]

31 minutes ago, Anders Melander said:

A picture of a red apple and a yellow banana is "similar" because they are both pictures of fruits.

When I said similarity of images, I meant similarity of pixels. It can check if the image is identical (similarity = 1) or if it is very close (similarity >= 0.99 for example).

 

We use this because it is normal for some drawings to vary a few pixels from platform to platform or from backend render to backend render. For example, text on Windows is slightly different from text on Android, even though both have the same font loaded. So in the unit tests we set up some tolerable similarity for each type of drawing.