Renate Schaaf

I have update the repo on GitHub

https://github.com/rmesch/Parallel-Bitmap-Resampler

Changes made to the "modern" VCL- and FMX-version in the folder BitmapScaling:

   New resampling filters: Mitchell, Robidoux, RobidouxSharp, RobidouxSoft.
   Simplified and corrected MakeGaussContributors in uScaleCommon.pas.

@Anders Melander: It will pass the uniform color tests now. But it will fail the Gauss-RMS, since I changed to RadiusToSigma back.
   Tried to make Gamma-correction a little more precise.

On 10/18/2023 at 7:27 PM, Kas Ob. said:

please don't waste your time on compatibility now

I tried nonetheless. You already spent so much time digging through that ancient attachment, give the repo a shot. I also added the option in DemoScale.dpr to use a test-bitmap similar to yours. I can't see any of the color-artefacts you describe, though.  
 

30 minutes ago, prodeldpen said:

I suppose there are effects of "round"ing that lead to this uncertainty.

Right. You want to add 1 frame of your animation at a time, but you use bme.addStillImage, which is meant for adding the same image for multiple frames. So it will only work (roughly) correctly if the ShowTime is much larger than the frame time of the movie.

Try to use bme.AddFrame instead.

36 minutes ago, prodeldpen said:

customize the size of my movie to anything like 123 x 456

That just won't work, it's a codec limitation. You have to use at least even numbers, for some codecs the sizes might even have to be multiples of 4. I would stick to multiples of 4 to be on the safe side.

Another thing you might consider is to shorten the chain from animation to movie. To show the animation and make screenshots seems a bit roundabout to me, there must be a shorter way.

45 minutes ago, prodeldpen said:

And eventually, is there an easy way to write meta data (author, etc.) to the file.

There must be, but I haven't yet bothered to look at it 🙂, maybe I will.

OK, Maple computed the following simplified filters, to implement them was just a matter of extending the TFilter-Enum. I'll update my repo some time tomorrow, the new filters need to be implemented in the demos. Right now I feel more like surviving a few more days on The Long Dark.

// The following filters are based on the Mitchell-Netravali filters with
// restricting the parameters B and C to the "good" line B + 2*C = 1.
// We have eliminated B this way and scaled the filter to [-1,1].
// See https://en.wikipedia.org/wiki/Mitchell%E2%80%93Netravali_filters

const
  C_M = 1 / 3;

  // Mitchell filter used by ImageMagick
function Mitchell(x: double): double; inline;
begin
  x := abs(x);
  if x < 0.5 then
    Result := (8 + 32 * C_M) * x * x * x - (8 + 24 * C_M) * x * x + 4 / 3 +
      4 / 3 * C_M
  else if x < 1 then
    Result := -(8 / 3 + 32 / 3 * C_M) * x * x * x + (8 + 24 * C_M) * x * x -
      (8 + 16 * C_M) * x + 8 / 3 + 8 / 3 * C_M
  else
    Result := 0;
end;

const
  C_R = 0.3109;

  // Robidoux filter
function Robidoux(x: double): double; inline;
begin
  x := abs(x);
  if x < 0.5 then
    Result := (8 + 32 * C_R) * x * x * x - (8 + 24 * C_R) * x * x + 4 / 3 +
      4 / 3 * C_R
  else if x < 1 then
    Result := -(8 / 3 + 32 / 3 * C_R) * x * x * x + (8 + 24 * C_R) * x * x -
      (8 + 16 * C_R) * x + 8 / 3 + 8 / 3 * C_R
  else
    Result := 0;
end;

.... and so on. Just one function with different constants.

1 hour ago, Kas Ob. said:

I browsed your code in "View file" in WinRar , i saw MakeContributors and AntiNLanczos and i confident you can just duplicate AntiNLanczos for all the above in the table with B and C or replace them with the calculated value like mine for faster calculation, while MakeContributors most likely doesn't need any change or a may be a little, i am sure you can solve this as it is way easier than how it sound and look.

I know! Just missed the B and C-values for the Robidoux in the table-image you post. And then I just have to rescale the functions to have support in [-1,1], make sure it's integral is 1. Bang. It plugs right in.

Wish I could edit my original post and delete the attachment, it's ancient now, and include a link to my GitHub-repo. The AntiNLanczos is a spline to approximate the antiderivative of Lanczos, all that stuff isn't needed anymore.

26 minutes ago, Kas Ob. said:

but Min and Max are not inlined functions

They are in System.Math:

function Min(const A, B: Integer): Integer; overload; inline;

I had coded it with ifs in a previous version, but I changed that after I noticed the inlining, looks a bit less stupid.

1 hour ago, Kas Ob. said:

here are the raw formula

Oh, W is the weight you compute, and param is the x of the kernel. So you *did* post the kernel code, I was just too dense to see it. I think Maple and me can take it from there.

1 hour ago, Kas Ob. said:

things for me fall in place when i looked at how imagemagick implemented

I tried to find something in their source code, but gave up. Looks like you had a bit more stamina :).

On 10/13/2023 at 12:15 PM, Kas Ob. said:

And here are the parameters for Robidoux

What's the function these parameters need to be plugged into? All I can gather is that it might be some kind of cubic spline, and I don't feel like reading all of this guy's papers :). Would you mind posting the formula for the kernel?

MadExcept can check for frozen main thread, with or without debugger. Just surprised nobody mentioned it.

On 10/14/2023 at 3:42 PM, chmichael said:

Anyone knows how to copy the Bits from a ID3D11Texture2D  which it has different RowPitch than Bitmap.Width * 4 ?

 

It doesn't only depend to the pitch, but also on the pixel-format of the source. If that is BGR or BGRA, the following pseudo-code based on what you post

should be usable. If the color-order is different, like RGB or RGBA, you need to copy the single color-bytes. Best done by defining a record for the pixel.

// Pointers to Pixels in Source/Target to be copied
  var
    pPixS, pPixT: pByte;

    // Bytes per Pixel for the Source-Texture
    // would be 3 for BGR, 4 for BGRA
    // if the channels have a different order, like RGB,
    // then the single color bytes need to be copied.
  var
    BytesPerPixel: integer;

    // Common Width and Height of Source and Target
  var
    Width, Height: integer;

  for I := 0 to Height - 1 do
  begin
    pPixS := @FTexture.pData[FTexture.RowPitch * I];
    pPixT := FBitmap.Scanline[I];
    for j := 0 to Width - 1 do
    begin
      Move(pPixS^, pPixT^, BytesPerPixel);
      inc(pPixS, BytesPerPixel);
      inc(pPixT, 4);
    end;
  end;

3 hours ago, Anders Melander said:

I'm guessing your CPU is about twice as fast as mine

It's probably not. Your blur is slightly slower than mine for small radii. For large radii, yours is much faster. I'll mail you my benchmark-unit, then you can see for yourself.

1 hour ago, Kas Ob. said:

To my past working on resampling and resizing up and down, i found that Mitchell and Robidoux filter is superior to Lanczos(3) in speed and quality specially in halo reduction and color defects.

Thanks very much for the input, I hadn't looked at those filters more closely before, should be easy to test them out.

18 hours ago, Anders Melander said:

I'll include the test bench too.

Thank you!

Here is a first result for radius 10. It only passed the Gauss-RMS-test after I changed the sigma-to-radius-ratio to the same as yours. Need to give that a closer look.

For other radii my routine failed some of the uniform color tests, (and edge detection as a consequence,) so it's back to the drawing board for that.

1 hour ago, Alexander Sviridenkov said:

delphihtmlcomponents.com/blur.zip

I managed to get it, source or not. For the same amount of "blurriness" my parallel version needs about 1.5 times the time of yours. Source would still be nice, I'm sure we'd learn something.

Renate

22 minutes ago, Anders Melander said:

Premultiply, blur, unpremultiply.

I already did.

24 minutes ago, Anders Melander said:

If you start by posting to your own repository I can do the work required to get it integrated into Graphics32.

I meanwhile learned how to do pull requests 🙂

Looking forward to the mailman!

Renate

It won't work, unless the images are all of the same size. Changing the size would cause at least a partial re-initialization.

13 hours ago, Anders Melander said:

FastBlur is actually a box blur and not a true Gaussian blur. This is just fine for some setups, but not so great for others.

I was fooled by the names in the example-project. I just had a closer look and, yes it's box blur.

As far as I can see  GaussianBlur isn't Gaussian either, it just introduces weights of 1/square_distance_from_center.

They do handle the alpha-channel, though. I would have liked to see the alpha-blended result, but I couldn't get the ImageView to show it.

Seeing as my (unthreaded) routine is truly Gaussian, and that it outperforms GR32.GaussianBlur by a factor of roughly 2-3, would it make sense for me to suggest it as a replacement in GR32?

How can I better test the quality?

Where can I find your routine then?

Quality usually takes precedence over performance for me, unless the performance annoys me, well it's also fun to get something faster 🙂

On 10/3/2023 at 1:23 AM, Anders Melander said:

Have you benchmarked this against some of the existing Gaussian blur implementations?

OK, I plugged my unsharp-mask into the Blurs-example of GR32. Doing so, made me aware of the need to do gamma-correction when you mix colors. So I implemented that, but see below.

Also, I finally included options to properly handle the alpha-channel for the sharpen/blur.

The repo at GitHub has been updated with these changes.

Results:

  Quality: My results seem a tad brighter, otherwise I could see no difference between Gaussian and Unsharp.

  Performance: Unthreaded routine: For radii up to 8 Unsharp is on par with FastGaussian, after that FastGaussian is the clear winner.

                        Threaded routine: Always fastest.

If anybody is interested, I am attaching the test project. It of course requires to have GR32 installed. It also requires 10.3 or higher, I guess.

Gamma-correction:

  I did it via an 8bit-Table same as GR32. This seems very unprecise to me, but I wouldn't know how to get it any more precise other than operating with floats, no thanks.

  Sadly, this can produce visible banding in some images, no matter which blur is used. Here is an example (for uploading all images have been compressed, but the effect is about the same):

Original, a cutout from a picture taken with my digital camera.

Result of Gaussian with Radius = 40 and Gamma = 1.6

When gamma-correction is used for sharpening, bright edge-artifacts are reduced, but dark edge-artifacts are enhanced. My conclusion right now would be to not use gamma-correction.

But if anybody has an idea for how to implement it better, I'm all ears.

Thanks,

Renate

BlurTest.zip

31 minutes ago, chmichael said:

Imagine using a TMemoryStream you can do whatever you want with the frames and i'll don't care about the expensive initialization.

But you would have to tell the encoder that every frame is a key-frame, otherwise it only stores the differences. But it can be done ...

Also, the access to the frames for decoding needs to be sped up. But you could this way create a stream of custom-compressed images.

Of course, no other app would be able to use this format.

14 hours ago, chmichael said:

HEIF files containing HEVC-encoded images are also known as HEIC files. Such files require less storage space than the equivalent quality JPEG.

HEIF is a container format like .mp4, as far as I see Windows manages this file format via WICImage only. MFPack contains headers for this, but all that goes a bit over my head.

If you want to test HEVC-compression, you can do this via BitmapsToVideoWMF by creating an .mp4-file with just one frame using the procedures below.

This is anything but fast, because of the initialization/finalization of Mediafoundation taking a long time.

A quick test compresses a .jpg of 2.5 MB taken with my digital camera to an .mp4 of 430 KB. No quality loss visible at first glance.

uses VCL.Graphics, uTools, uTransformer, uBitmaps2VideoWMF;

procedure EncodeImageToHEVC(const InputFilename, OutputFileName: string);
var
  wic: TWicImage;
  bm: TBitmap;
  bme: TBitmapEncoderWMF;
begin
  Assert(ExtractFileExt(OutputFileName) = '.mp4');
  wic := TWicImage.Create;
  try
    bm := TBitmap.Create;
    try
      wic.LoadFromFile(InputFilename);
      WicToBmp(wic, bm);
      bme := TBitmapEncoderWMF.Create;
      try
        // Make an .mp4 with one frame.
        // Framerate 1/50 would display it for 50sec
        bme.Initialize(OutputFileName, bm.Width, bm.Height, 100,
          1 / 50, ciH265);
        bme.AddFrame(bm, false);
      finally
        bme.Free;
      end;
    finally
      bm.Free
    end;
  finally
    wic.Free;
  end;
end;

procedure DecodeHEVCToBmp(const mp4File: string; const Bmp: TBitmap);
var
  vi: TVideoInfo;
begin
  vi := uTransformer.GetVideoInfo(mp4File);
  GetFrameBitmap(mp4File, Bmp, vi.VideoHeight, 1);
end;

1 hour ago, chmichael said:

I was hoping i could use H264/5 hardware encoder/decoder for a single image. It should be faster and smaller than turbo-jpeg.

This is why that makes no sense (quoted from https://cloudinary.com/guides/video-formats/h-264-video-encoding-how-it-works-benefits-and-9-best-practices)

"H.264 uses inter-frame compression, which compares information between multiple frames to find similarities, reducing the amount of data needed to be stored or transmitted. Predictive coding uses information from previous frames to predict the content of future frames, further reducing the amount of data required. These and other advanced techniques enable H.264 to deliver high-quality video at low bit rates. "

15 hours ago, chmichael said:

Hardware encode only 1 image ?

Do you mean to Jpeg or Png? A video-format would not make any sense for a single image.

NVidea-chips can apparently do that, but I don't know of any Windows-API which supports that.

If you want to encode to Png or Jpeg a bit faster than with TPngImage or TJpegImage use a TWicImage.

Look at this nice post on Stackoveflow for a way to set the compression quality for Jpeg using TWicImage:

https://stackoverflow.com/questions/42225924/twicimage-how-to-set-jpeg-compression-quality

I'll see what I can do. Stream might be tricky, I would need to learn more about this.

New version at https://github.com/rmesch/Parallel-Bitmap-Resampler:

Has more efficient code for the unsharp-mask, and I added more comments in code to explain what I'm doing.

Procedures with explaining comments:

uScaleCommon.Gauss

uScaleCommon.MakeGaussContributors

uScaleCommon.ProcessRowUnsharp

and see type TUnsharpParameters in uScale.pas.

Would it be a good idea to overload the UnsharpMask procedure to take sigma instead of radius as a parameter? Might be easier for comparison to other implementations.

9 hours ago, Anders Melander said:

I have a benchmark suite that compares the performance and fidelity of 8 different implementations. I'll try to find time to integrate your implementation into it.

Hi Anders,

It's great that you think of it, but hold off on that for a bit. I noticed that I compute the weights in a horrendously stupid way. The weights are mostly identical, it's not like when you resample, dumb me. So taking care of that reduces memory usage by a lot and the subsequent application of the weights becomes much faster.

I've also changed the sigma-to-radius ratio a bit according to your suggestion. I find it hard to make results look nice with cutoff at half the max-value, I changed it to 10^-2 times max-value. But this still allows for smaller radii, and it becomes again a bit faster.

So, before you do anything I would like to finish these changes, and also comment the code a bit more. (Forces me to really understand what I'm doing 🙂

43 minutes ago, Anders Melander said:

But you have a ratio of 0.5

I took sigma = 0.2*Radius, but it's easy to change that to something more common. I just took a value for which the integral is very close to 1. With respect to other implementations, I'm ready to learn. I just implemented it as accurately as I could think of without being overly slow. Performance is quite satisfying to me, but I bet with your input it'll get faster 🙂

I just uploaded a new version to https://github.com/rmesch/Parallel-Bitmap-Resampler

Newest addition: a parallel unsharp-mask using Gaussian blur. Can be used to sharpen or blur images.

Dedicated VCL-demo "Sharpen.dproj" included. For FMX the effect can be seen in the thumbnail-viewer-demo (ThreadsInThreadsFMX.dproj).

This is for the "modern" version, 10.4 and up.

I haven't ported the unsharp-mask to the legacy version (Delphi 2006 and up) yet, requires more work, but I plan on doing so.

Renate

22 hours ago, chmichael said:

Just curious, anyone tried Skia for resampling ?

I did a quick test with the demo of the fmx-version of my resampler, just doing "Enable Skia" on the project.

In the demo I compare my results to TCanvas.DrawBitmap with HighSpeed set to false.

I see that the Skia-Canvas is being used, and that HighSpeed=False results in Skia-resampling set to

SkSamplingOptionsHigh  : TSkSamplingOptions = (UseCubic: True; Cubic: (B: 1 / 3; C: 1 / 3); Filter: TSkFilterMode.Nearest; Mipmap: TSkMipmapMode.None);

So, some form of cubic resampling, if I see that right.

Result:

Timing is slightly slower than native fmx-drawing, but still a lot faster than my parallel resampling.

I see no improvement in quality over plain fmx, which supposedly uses bilinear resampling with this setting.

Here are two results: (How do you make this browser use the original pixel size, this is scaled!)

This doesn't look very cubic to me. As a comparison, here are the results of my resampler using the bicubic filter:

I might not have  used Skia to its most favorable advantage.

Renate