smooth scaling of bitmaps

[wadepm 0]

I played around with the code by adding a couple of buttons to change the image scale.  I found that when the scale is not 1 the scrolling is a mess - the image jumps around and doesn't follow the cursor.  When the scale is 1 the scrolling works great.  Any ideas?    

[Anders Melander 1639]

17 hours ago, wadepm said:

I found that when the scale is not 1 the scrolling is a mess - the image jumps around and doesn't follow the cursor.  When the scale is 1 the scrolling works great.  Any ideas?

My guess is that you're using the wrong TImgView32.ScaleMode. Make sure TImgView32.ScaleMode=smScale.

 

I've attached a small example that demonstrates how to pan and zoom with TImgView32.

imgview32demo.zip

[wadepm 0]

I am using smScale.  I'll take a look at your example though.  Thanks.

 

[wadepm 0]

I have narrowed down the problem to when image.scale is less than 1.  If it is 1 or larger the scrolling works very nicely, very smooth.  When it is less than 1 it gets wonky.  

 

[Anders Melander 1639]

Reproduced.

I've actually encountered that problem in my own applications but I've never been able to pinpoint the circumstances that surfaced the problem. I'll investigate now.

[Anders Melander 1639]

Well that was easy. Luckily the bug was in my own code.

 

Here's the code that works:

procedure TFormMain.ImgViewMouseDown(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Integer;
  Layer: TCustomLayer);
begin
  if (Button = mbLeft) then
  begin
    FPanning := True;
    ImgView.Cursor := crSizeAll;
    FStartPos := Point(X, Y);
  end else
  if (Button = mbMiddle) then
    ImgView.Scale := 1;
end;

procedure TFormMain.ImgViewMouseMove(Sender: TObject; Shift: TShiftState; X, Y: Integer; Layer: TCustomLayer);
begin
  if (not FPanning) then
    Exit;

  var NewPos := Point(X, Y);
  var Delta := FStartPos - NewPos;
  FStartPos := NewPos;

  if (Delta.X <> 0) or (Delta.Y <> 0) then
    ImgView.Scroll(Delta.X, Delta.Y);
end;

 

[wadepm 0]

That works very nicely, thanks!  I see you removed ImgView.ControlToBitmap(Point(X, Y)).  I was trying to figure out exactly what that did...

[Anders Melander 1639]

Just now, wadepm said:

I see you removed ImgView.ControlToBitmap(Point(X, Y)).

Yes. That was just a brain fart on my behalf.

[wadepm 0]

Thanks for your help.  I now have a nicely scaled image that scrolls smoothly.  Onwards!

[wadepm 0]

...of course, I have more questions!

 

Now my problem is I need to know the location of the cursor above the image, not just the viewport.  I thought I could use the offsethorz and offsetvert properties.  They sound like they should give me the position of the image relative to the viewport.  However, they don't appear to contain any information.  Also, the manual says they are type single but in reality they are type extended.  I am not sure what they are intended to provide.  Is there a property of the image that contains the offsets relative to the viewport?     

[wadepm 0]

Looking at GR32_image.pas I see properties hscroll.position and vscroll.position that should give me what I want.  But I can't figure out how to get at them...  

[wadepm 0]

Solved!  This is where ControlToBitmap(Point(X, Y)) comes in handy.

[c0d3r 17]

On 12/28/2020 at 10:25 AM, Anders Melander said:

Something like this:

procedure Test;
var
  Source: TBitmap32;
  Target: TBitmap32;
  Resampler: TKernelResampler;
begin
  Target := TBitmap32.Create;
  try
    Source := TBitmap32.Create;
    try

      Source.LoadFromFile('foobar.bmp');

      // Make new bitmap twice the size. You can also make it smaller.
      Target.SetSize(Source.Width*2, Source.Height*2);

      Resampler := TKernelResampler.Create(Source); // Resampler is now owned by TBitmap32
      Resampler.Kernel := TLanczosKernel.Create; // Kernel is now owned by resampler

      // Stretch using kernel resampler
      Target.Draw(Target.BoundsRect, Source.BoundsRect, Source);

    finally
      Source.Free;
    end;

    // Do something with target bitmap...

  finally
    Target.Free;
  end;
end;

 

Try the above codes but with PNG files, seems the transparent colors got lost always, all in black,  even though I try to use like this:

 

LoadBitmap32FromPNG(Source, 'normal_left.png');

 

other than this problem everything else is working,  What am I missing?

 

 

 

Edited January 1, 2021 by c0d3r

[Anders Melander 1639]

22 minutes ago, c0d3r said:

Try the above codes but with PNG files, seems the transparent colors got lost always, all in black,  even though I try to use like this:

LoadBitmap32FromPNG(Source, 'normal_left.png');

other than this problem everything else is working,  What am I missing?

I don't use LoadBitmap32FromPNG and it's not part of Graphics32. Have you verified that the transparent PNG pixels are still transparent after conversion to TBitmap32?

Other than that the problem could be the blend and combine mode used when drawing but that should be the same regardless of the origin of the pixel data.

[c0d3r 17]

16 minutes ago, Anders Melander said:

I don't use LoadBitmap32FromPNG and it's not part of Graphics32. Have you verified that the transparent PNG pixels are still transparent after conversion to TBitmap32?

Other than that the problem could be the blend and combine mode used when drawing but that should be the same regardless of the origin of the pixel data.

The PNG file is working fine using TPngImage.  I did a very simple test with Graphics32:

 

1:

 

Source.LoadFromFile('XXX.PNG');

Source.DrawTo(Image1{TImage}.Canvas.Handle, 0, 0);

 

 

2: using GR32PNG:

 

LoadBitmap32FromPNG(Source, 'xxx.png');

Source.DrawTo(Image1{TImage}.Canvas.Handle, 0, 0);

 

 

Both transparent channels were got lost with different DrawMode (dmTransparent, dmBlend, ...), all in Black,  not yet do the resample things.

 

Edited January 1, 2021 by c0d3r

[Anders Melander 1639]

6 minutes ago, c0d3r said:

The PNG file is working fine using TPngImage.  I did a very simple test with Graphics32:

You didn't answer my question: Have you verified that the transparent PNG pixels are still transparent after conversion to TBitmap32?

 

7 minutes ago, c0d3r said:

Both transparent channels were got lost with different DrawMode (dmTransparent, dmBlend, ...), all in Black,  not yet do the resample things. 

As far as I can tell, from the 4 lines of code you posted, you're not really using graphics32 to draw anything here. If you draw a TBitmap32 onto a Windows GDI device (Image.Canvas.Handle) then the GDI StretchDIBits function will be used to draw it - hence no transparency.

The primary reason to use graphics32 is that it allows you to do the composition (blending, merging, layers, rendering, etc) really fast and then draw the final image onto the screen. If what you actually want is to draw something onto the screen with transparency then maybe you shouldn't be using graphics32 at all?

 

If you really want to draw a TBitmap32 onto a GDI device with transparency then you should assign it to a TBitmap, set TBitmap.AlphaFormat=afDefined and then draw the TBitmap instead. TBitmap will use the AlphaBlend function in this case.

var Bitmap := TBitmap.Create;
try
  Bitmap.Assign(Bitmap32);
  Bitmap.AlphaFormat := afDefined;
  Bitmap.DrawTo(Image.Canvas.Handle, 0, 0);
finally
  Bitmap.Free;
end;

Btw, are you really sure you want to draw onto the TImage.Canvas? That seems.... um... like a mistake...

[c0d3r 17]

As far as I know,  once the source loads file/stream from a PNG file/stream, the transparent got lost,  Here are the codes:

 

      Source.LoadFromFile('normal_left.png');  <-- Transparent got lost, transparent turned to White

 

      Target.SetSize(Round(Source.Width * 2), Round(Source.Height * 2));

      Resampler := TKernelResampler.Create(Source); // Resampler is now owned by TBitmap32
      Resampler.Kernel := TMitchellKernel.Create; // Kernel is now owned by resampler

 

      // Stretch using kernel resampler
      Target.Draw(Target.BoundsRect, Source.BoundsRect, Source);

 

      ABitmap := TBitmap.Create;
      try
        ABitmap.Assign(Target);
        ABitmap.AlphaFormat := afDefined;
        Image1{TImage}.Picture.Bitmap.Assign(ABitmap);
      finally
        ABitmap.Free;
      end;

 

if you replace the first line with the following, the transparent turned to Black:

 

LoadBitmap32FromPNG(Source, 'normal_left.png');

 

 

Edited January 2, 2021 by c0d3r

[Anders Melander 1639]

6 minutes ago, c0d3r said:

Source.LoadFromFile('normal_left.png');  <-- Transparent got lost, transparent in White

I don't really know what type Source is but assuming it's a TBitmap32 then loading a PNG with LoadFromFile will internally use a TPicture to load the file and then assign the TPicture.Graphic (which will be a TPNGImage) to the TBitmap32 using a generic import routine that replaces the transparency with white. This matches what you're observing.

 

14 minutes ago, c0d3r said:

if you replace the first line with the following, the transparent is in Black:

LoadBitmap32FromPNG(Source, 'normal_left.png');

As far as I can tell everything works "as designed".

 

16 minutes ago, c0d3r said:

Image1{TImage}.Picture.Bitmap.Assign(ABitmap);

What did you expect this would do?

If you want to display a 32-bit TBitmap with alpha using TImage then I suggest you create a 32-bit RGBA bmp file and experiment with getting TImage do display that. You have too many variable factors in your approach. Get one thing working first. Then move on to the next problem. The above has nothing to do with Graphics32.

[c0d3r 17]

Sorry.  My mistake.  Its working now, have to use LoadBitmap32FromPNG(Source, 'normal_left.png'), along with ABitmap.AlphaFormat := afDefined;  The reason why I'm using TImage was just for testing to see how the images scaling looks like, so I could use it for scaling my own page control tab bars:

 

Edited January 2, 2021 by c0d3r

[wadepm 0]

Anders,

 

One more question on scrolling. When I zoon in on the bitmap (scale<1) the scrolling is really slow, there is quite a bit of lag.  When scale = 1 there is no lag.  Is it doing a rescaling ever time it moves the image 1 pixel?  I have been looking at the code and I don't see it but that would explain the lag I think. 

[Anders Melander 1639]

4 hours ago, wadepm said:

Is it doing a rescaling ever time it moves the image 1 pixel?

It's sampling every time it draws something. That doesn't mean that it necessarily resamples the whole bitmap though. It will only sample the parts that it needs in order to paint the visible part of the bitmap. Read this:

https://graphics32.github.io/Docs/Additional Topics/Sampling and Rasterization.htm

https://graphics32.github.io/Docs/Additional Topics/Repaint Optimization.htm

 

Of course when scale < 1 then the whole bitmap will probably be visible and everything will be sampled. As far as I remember there's no repaint optimization with regard to panning - I guess that's something I could look into if I run out of other stuff to do
 

Anyway, even if you did pan 1 pixel at a time (which I doubt you're doing) it should be fast enough that you shouldn't experience any stutter. Of course it depends on the type of sampler/kernel you're using, your hardware and the size of the bitmap but on my old system I can pan a zoomed (in or out) 4000x4000 multi layer bitmap without any noticeable stutter.

[c0d3r 17]

@Anders Melander  Thanks so much for the help.  Now not only all the images in the application wre scaling smoothly, but also all the Toolbar2000s were all scaled perfectly under different High DPIs (100%, 125%, 150%, 200%, ...). Yes, TOOLBARS are working without adding different set of images!!!

 

 

 

 

Edited January 5, 2021 by c0d3r

[wadepm 0]

I played around with the resampler kernel and that makes a big difference.  I started using Mitchell but when I switched to Box the scrolling was much smoother but the scaling wasn't as nice.  I am surprised that the image is resampled when it is scrolled as the size is not changing.  I am going to try scaling a bitmap and then assigning that to the image before scrolling.  

[Anders Melander 1639]

14 minutes ago, wadepm said:

I played around with the resampler kernel and that makes a big difference.  I started using Mitchell but when I switched to Box the scrolling was much smoother but the scaling wasn't as nice.

I think TLinearKernel (or just the TLinearResampler) will yield the best down-sample quality with good performance.

 

4 hours ago, wadepm said:

I am surprised that the image is resampled when it is scrolled as the size is not changing.

The links I posted explains why. The update mechanism redraws the areas of the bitmap that changes. When you pan the bitmap doesn't change but the view of it does. Therefore the optimized update can't handle the pan so the whole viewport is repainted instead.

I'm currently investigating if I can shoehorn a pan optimization into the existing framework with minimal changes.

 

19 minutes ago, wadepm said:

I am going to try scaling a bitmap and then assigning that to the image before scrolling. 

That's what I do when previewing large bitmaps (for PixelFormat=pf32bit) but I only do it do avoid out of memory:

// Get RGBA from source
ImageView.Bitmap.BeginUpdate;
try
  // Limit preview image size
  const MaxPixels = 1024*1024;
  var Pixels := TBitmap(FImage).Width * TBitmap(FImage).Height;
  if (Pixels >= MaxPixels) then
  begin
    var Scale := Sqrt(MaxPixels / Pixels);

    // Proportionally scale bitmap so the result doesn't contain more than the desired number of pixels
    ImageView.Bitmap.SetSize(Round(TBitmap(FImage).Width * Scale), Round(TBitmap(FImage).Height * Scale));

    // Stretch draw the source bitmap onto the scaled bitmap
    TBitmap(FImage).Canvas.Lock;
    try

      SetStretchBltMode(ImageView.Bitmap.Canvas.Handle, COLORONCOLOR);

      ImageView.Bitmap.Draw(ImageView.Bitmap.BoundsRect, TBitmap(FImage).Canvas.ClipRect, TBitmap(FImage).Canvas.Handle);

    finally
      TBitmap(FImage).Canvas.Unlock;
    end;
  end else
  begin
    ImageView.Bitmap.SetSize(TBitmap(FImage).Width, TBitmap(FImage).Height);

    for var Row := 0 to TBitmap(FImage).Height-1 do
      MoveLongword(TBitmap(FImage).ScanLine[Row]^, ImageView.Bitmap.ScanLine[Row]^, TBitmap(FImage).Width);
  end;
finally
  ImageView.Bitmap.EndUpdate;
end;
ImageView.Bitmap.Changed;

 

[Anders Melander 1639]

5 hours ago, Anders Melander said:

I'm currently investigating if I can shoehorn a pan optimization into the existing framework with minimal changes.

Well, I couldn't resist the challenge so I have a nicely working implementation now. I'll create a pull request with it tonight (it's 6 in the morning here in Denmark) when I wake up again .