Bitmaps to Video for Mediafoundation

[Renate Schaaf 67]

I've just uploaded an update to my project 

 

https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation

 

What it does: 
Contains a VCL-class which encodes a series of bitmaps and video-clips together with an audio-file to video.

The result is an .mp4-file with H264 or H265 compression together with AAC-audio.

It uses windows mediafoundation, which is usually contained in windows. Hardware-encoding is supported, if your graphics-card can do it.

 

Requires:
Headers for mediafoundation from FactoryXCode: https://github.com/FactoryXCode/MfPack
Windows 10 or higher
Encoder (MF-Transform) for H264/H265, usually come with the graphics-driver
Delphi XE7 or higher, if I haven't messed it up again, I've only got the CE and Delphi2006
(Win32 and Win64 should be working, but Win64 recently crashes for me with "The session was disconnected".)

 

The demo-project shows some uses:
    Record a series of canvas-drawings to video
    Make a slideshow from image-files (.bmp,.jpg,.png,.gif) with music (.wav, .mp3, .wmv, ...) and 2 kinds of transitions
    Insert a videoclip into a slideshow (anything that windows can decode should work)
    Transcode a video-file including the first audio-stream.

 

Improvements: 
I think I now better understand how to feed frames to the encoder. With the right settings it makes stutter-free videos with good audio-video-synchronization. It's now usable for me in my "big" project, and I no longer need to rely on ffmpeg - dlls.

 

More info in changes.txt.

 

Just try it, if you're interested, I'd be glad. 

Renate

 

[Kas Ob. 140]

Hi @Renate Schaaf ,

 

Friendly reminder i pointed to in the past, the audio duration and handling is having hidden problem, it might be not visible (ok hearable) now, but it render the library not future proof and any change in the way codec API works will cause either errors or desynced audio/video

 

so here my thoughts on this part

about TBitmapEncoderWMF.WriteAudio https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation/blob/main/Source/uBitmaps2VideoWMF.pas#L1523-L1618

1) Ditch "goto Done;" and use try..finally it is safer and here there is no need for goto and loop is not complex, it is just exit.

2) https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation/blob/main/Source/uBitmaps2VideoWMF.pas#L1685 doesn't check, fail or warn about audio failure

3) after reading samples with pSourceReader.ReadSample https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation/blob/main/Source/uBitmaps2VideoWMF.pas#L1556-L1562 you should check the returned bytes size, are they aligned with the requested audio format ?!

This what bring the last discussion if my memory didn't fail me, the audio duration should be aligned, in other words

AudioBlock := nChannels * wBitsPerSample / 8 , this will give the exact amount in bytes that can't be divided, so any audio data passing should be multiple of AudioBlocks, but we almost always have these as integers then

AudioBlock := (nChannels * wBitsPerSample) div 8 ; this should do 

Now to do the the extra check for AudioDuration you can have it like this

AudioDuration := (BufferSize / AudioBlock) * (10000000 / nSamplesPerSec);

 

The difference between Audio and Video i am sure you know a lot about, but you may be didn't experience or witness when codec start to

1) fail with errors

2) desync the audio-video due dropping the less than block 

3) corrupt the quality with sound artefacts due internal padding of the samples on its own.

each of these is a bug could be in any codec, they all evolve and change as their implementation keep optimized and worked on.

 

I remember this very clearly, it was pain in the back with ASF and WMV, sometimes works next day doesn't on the same Windows, the root cause was the block alignment, even if the other codec handling the audio decoding did the mistake and returned wrong size you should hold on the left over and feed it later, example 2channels with 16bit samples the size is 4 bytes, for 6 channels and 24bits the size is 18 bytes , you can test different audio files like with 5.1 and 7.1 (6 channels and 8 channels) using sample from https://www.jensign.com/bdp95/7dot1voiced/index.html

 

Hope that help.

 

ps this part 

    // fAudioDuration can be false!
    // if fAudioTime >= fAudioDuration then
    // fAudioDone := true;
    if fAudioDone then
      hr := pSinkWriter.NotifyEndOfSegment(fSinkStreamIndexAudio);
    // The following should not be necessary in Delphi,
    // since interfaces are automatically released,
    // but it fixes a memory leak when reading .mkv-files.
    SafeRelease(pAudioSample);

Is disturbing, 

1) The commented "if fAudioTime >= fAudioDuration then" is right and should be used but "fAudioDuration can be false!" well i would love to hear how this happen.

2) "but it fixes a memory leak when reading .mkv-files." return us to (1) from above using try..finally is best and will prevent memory leak, but such a case for .mkv files is strange and should be investigated deeper as it could be serious problem and might cause huge leak in the loop it self depleting OS memory specially for 64bit.

[Renate Schaaf 67]

Hi Kas,

Good to see you again, and sorry for the long time of inactivity on my part. Thank you for the detailed input, which I need to digest first. Since you already invested so much thought, wouldn't you like to be a contributor? When I incorporate the changes you mention, I wouldn't even know how to list you as contributor. The issues you mention definitely need to be looked into. For the audio-part I was just glad it worked, and haven't put much thought into it lately. The wrong audio-duration was returned by some .vobs, which aren't really supported in the first place. The missing SafeRelease(pAudioSample) has caused memory leaks for me in a totally different context too, when I tried to write some code which simply plays an audio file through the default-device.

 

Renate

[Anders Melander 2012]

1 hour ago, Kas Ob. said:

1) Ditch "goto Done;" and use try..finally it is safer and here there is no need for goto and loop is not complex, it is just exit.

There's not even a need for try..finally since there no resources to protect; Get rid of hr, assign the results to Result directly and just exit.

 

Also, instead of:

raise Exception.Create('Fail in call nr. ' + IntToStr(Count) + ' of ' +
  ProcName + ' with result $' + IntToHex(hr));

I would use:

raise Exception.CreateFmt('Fail in call no. %d of %s with result %x', [Count, ProcName, hr]);

for readability.

[Renate Schaaf 67]

Hi Anders,

Thanks for that. I hate the format-strings, because I can never remember the code for the place-holders.  I had already thought before, that I should get used to them, though. Now I also see, that I forgot to use IntToHex(hr,8) 🙂