Bitmaps2Video for Windows Media Foundation

[Renate Schaaf 64]

I'll see what I can do. Stream might be tricky, I would need to learn more about this.

[Renate Schaaf 64]

15 hours ago, chmichael said:

Hardware encode only 1 image ?

Do you mean to Jpeg or Png? A video-format would not make any sense for a single image.

NVidea-chips can apparently do that, but I don't know of any Windows-API which supports that.

If you want to encode to Png or Jpeg a bit faster than with TPngImage or TJpegImage use a TWicImage.

Look at this nice post on Stackoveflow for a way to set the compression quality for Jpeg using TWicImage:

https://stackoverflow.com/questions/42225924/twicimage-how-to-set-jpeg-compression-quality

[chmichael 12]

1 hour ago, Renate Schaaf said:

Do you mean to Jpeg or Png? A video-format would not make any sense for a single image. 

NVidea-chips can apparently do that, but I don't know of any Windows-API which supports that.

If you want to encode to Png or Jpeg a bit faster than with TPngImage or TJpegImage use a TWicImage.

Look at this nice post on Stackoveflow for a way to set the compression quality for Jpeg using TWicImage:

https://stackoverflow.com/questions/42225924/twicimage-how-to-set-jpeg-compression-quality

I was hoping i could use H264/5 hardware encoder/decoder for a single image. It should be faster and smaller than turbo-jpeg.
 

Edited October 9, 2023 by chmichael

[Renate Schaaf 64]

1 hour ago, chmichael said:

I was hoping i could use H264/5 hardware encoder/decoder for a single image. It should be faster and smaller than turbo-jpeg.

This is why that makes no sense (quoted from https://cloudinary.com/guides/video-formats/h-264-video-encoding-how-it-works-benefits-and-9-best-practices)

 

"H.264 uses inter-frame compression, which compares information between multiple frames to find similarities, reducing the amount of data needed to be stored or transmitted. Predictive coding uses information from previous frames to predict the content of future frames, further reducing the amount of data required. These and other advanced techniques enable H.264 to deliver high-quality video at low bit rates. "

Edited October 9, 2023 by Renate Schaaf

[chmichael 12]

4 hours ago, Renate Schaaf said:

This is why that makes no sense (quoted from https://cloudinary.com/guides/video-formats/h-264-video-encoding-how-it-works-benefits-and-9-best-practices)

 

"H.264 uses inter-frame compression, which compares information between multiple frames to find similarities, reducing the amount of data needed to be stored or transmitted. Predictive coding uses information from previous frames to predict the content of future frames, further reducing the amount of data required. These and other advanced techniques enable H.264 to deliver high-quality video at low bit rates. " 

That goes to my first request " 1) Encode to Stream".

 

As far the compression HEIF actually uses H.265 compression but there is not any hardware acceleration in Delphi or at least haven't found any!
https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format

HEIF files containing HEVC-encoded images are also known as HEIC files. Such files require less storage space than the equivalent quality JPEG.^[2][3]

Thank you

Edited October 9, 2023 by chmichael

[Renate Schaaf 64]

14 hours ago, chmichael said:

HEIF files containing HEVC-encoded images are also known as HEIC files. Such files require less storage space than the equivalent quality JPEG.

HEIF is a container format like .mp4, as far as I see Windows manages this file format via WICImage only. MFPack contains headers for this, but all that goes a bit over my head.

If you want to test HEVC-compression, you can do this via BitmapsToVideoWMF by creating an .mp4-file with just one frame using the procedures below.

This is anything but fast, because of the initialization/finalization of Mediafoundation taking a long time.

A quick test compresses a .jpg of 2.5 MB taken with my digital camera to an .mp4 of 430 KB. No quality loss visible at first glance.

 

uses VCL.Graphics, uTools, uTransformer, uBitmaps2VideoWMF;

procedure EncodeImageToHEVC(const InputFilename, OutputFileName: string);
var
  wic: TWicImage;
  bm: TBitmap;
  bme: TBitmapEncoderWMF;
begin
  Assert(ExtractFileExt(OutputFileName) = '.mp4');
  wic := TWicImage.Create;
  try
    bm := TBitmap.Create;
    try
      wic.LoadFromFile(InputFilename);
      WicToBmp(wic, bm);
      bme := TBitmapEncoderWMF.Create;
      try
        // Make an .mp4 with one frame.
        // Framerate 1/50 would display it for 50sec
        bme.Initialize(OutputFileName, bm.Width, bm.Height, 100,
          1 / 50, ciH265);
        bme.AddFrame(bm, false);
      finally
        bme.Free;
      end;
    finally
      bm.Free
    end;
  finally
    wic.Free;
  end;
end;

procedure DecodeHEVCToBmp(const mp4File: string; const Bmp: TBitmap);
var
  vi: TVideoInfo;
begin
  vi := uTransformer.GetVideoInfo(mp4File);
  GetFrameBitmap(mp4File, Bmp, vi.VideoHeight, 1);
end;

 

[chmichael 12]

bme.AddFrame(bm, false); It took 53 milliseconds to encode a 8k image faster than turbo-jpeg and with lower size as you say.

 

Imagine using a TMemoryStream you can do whatever you want with the frames and i'll don't care about the expensive initialization.

 

I don't understand why it creates a 10s video instead of 1s ...

 

[Renate Schaaf 64]

31 minutes ago, chmichael said:

Imagine using a TMemoryStream you can do whatever you want with the frames and i'll don't care about the expensive initialization.

But you would have to tell the encoder that every frame is a key-frame, otherwise it only stores the differences. But it can be done ...

Also, the access to the frames for decoding needs to be sped up. But you could this way create a stream of custom-compressed images.

Of course, no other app would be able to use this format.

[chmichael 12]

1 hour ago, Renate Schaaf said:

Of course, no other app would be able to use this format.

Well if you use it for internal use or streaming purposes you don't care.

[Renate Schaaf 64]

It won't work, unless the images are all of the same size. Changing the size would cause at least a partial re-initialization.

[chmichael 12]

8 hours ago, Renate Schaaf said:

It won't work, unless the images are all of the same size. Changing the size would cause at least a partial re-initialization.

For streaming it's always the same

[maXcomX 3]

Well, the code will not work on Delphi XE7.

See: Some issues

[maXcomX 3]

When examine the code, it's not clear to me what the parameter ShowTime means.

Lets say I want to render 3 images during the length of an audiofile in a slideshow, that would be the audiofile length div 3.

So, each image will be shown for 1/3th during the audiofile length. Is this value the "ShowTime" or does ShowTime means the time 2 images are fading to onenother?

[maXcomX 3]

On 10/10/2023 at 1:16 PM, Renate Schaaf said:

HEIF is a container format like .mp4, as far as I see Windows manages this file format via WICImage only. MFPack contains headers for this, but all that goes a bit over my head.

If you want to test HEVC-compression, you can do this via BitmapsToVideoWMF by creating an .mp4-file with just one frame using the procedures below.

This is anything but fast, because of the initialization/finalization of Mediafoundation taking a long time.

A quick test compresses a .jpg of 2.5 MB taken with my digital camera to an .mp4 of 430 KB. No quality loss visible at first glance.

 

uses VCL.Graphics, uTools, uTransformer, uBitmaps2VideoWMF;

procedure EncodeImageToHEVC(const InputFilename, OutputFileName: string);
var
  wic: TWicImage;
  bm: TBitmap;
  bme: TBitmapEncoderWMF;
begin
  Assert(ExtractFileExt(OutputFileName) = '.mp4');
  wic := TWicImage.Create;
  try
    bm := TBitmap.Create;
    try
      wic.LoadFromFile(InputFilename);
      WicToBmp(wic, bm);
      bme := TBitmapEncoderWMF.Create;
      try
        // Make an .mp4 with one frame.
        // Framerate 1/50 would display it for 50sec
        bme.Initialize(OutputFileName, bm.Width, bm.Height, 100,
          1 / 50, ciH265);
        bme.AddFrame(bm, false);
      finally
        bme.Free;
      end;
    finally
      bm.Free
    end;
  finally
    wic.Free;
  end;
end;

procedure DecodeHEVCToBmp(const mp4File: string; const Bmp: TBitmap);
var
  vi: TVideoInfo;
begin
  vi := uTransformer.GetVideoInfo(mp4File);
  GetFrameBitmap(mp4File, Bmp, vi.VideoHeight, 1);
end;

 

The video codec HEVC is well documented but difficult to understand . However, this codec needs to be initialized with a HEVC profile and optionally payload that is supported by Media Foundation. Now, the documentation is not very accurate about the supported profiles. Two of them are missing though. Note that Media Foundation only supports 2 payloads (0-1) 2 and 3 are not supported.

Another misunderstanding is how Media Foundation slices a media source. For a regulair file it has a video related codec an audio related codec, maybe a subtitle stream and the container file that holds those. For example a MP4 container (.mp4 file) can contain various types of codecs. Like for example the HEVC video codec and a mp3 audio codec, a H264 video codec and a Dolby_AC3 audio codec etc. 

The coming MfPack version will be expanded with a Videocheat and video profiles unit to make life easier.

New updates are regulair committed and can be found here

Note that the H.265 encoder implements by documentation properties like CODECAPI_AVLowLatencyMode etc. are DirectShow properties and therefore should not be implemented in Media Foundation.

 

[maXcomX 3]

By the way: 

wic.LoadFromFile(InputFilename);

Consumes the most time depending on size and type.