Renate Schaaf

9 minutes ago, Anders Melander said:

I'm guessing they used "cooked" jpegs because there's really not much magic that can be done here.

OK, I'll stop thinking about it. Time to get some sleep:)

Hi Anders,

Thanks for explaining. I had a feeling that the compression is too "global" for parallelizing. But ..

From what I have meanwhile read, it seems that parts of the decompression could be done in parallel.

This link is about compression, but couldn't it apply to decompression too? (not that I know anything about it 🙂

https://stackoverflow.com/questions/61850421/how-to-perform-jpeg-encoding-of-a-big-rgb-image-in-parallel

Anyway, there are research papers which claim that they got a speedup from doing the decoding partly in parallel.

6 hours ago, FreeDelphiPascal said:

Hi. Do you have something similar but for parallel jpeg decoding?

Sorry, no, but it sounds like a good idea. Naively. I have no idea how parallelizable jpeg-decoding is 🙂

38 minutes ago, David Schwartz said:

I don't think you can load any components into the CE versions

Sure you can.

The use of TVirtualImageList is totally unessential, just cosmetics. I have replaced it by the old TImageList now. Hope it works, and thanks!

4 hours ago, Kas Ob. said:

The problem in such calculation with that your are not using audio sample rate and the ability to divide the audio sample right, so my suggestion is to to round that fSampleDuration to what audio sample allow you and will be acceptable to the encoder and of course to the decoder in later stage.

I think I know what you mean, I'll play around with it some. Problem is that I really need to have a well defined video-frame-rate, otherwise the user can't time the input. I have meanwhile figured out the audio-sample-duration for given sample-rate and use it for the amount of audio-samples to be read ahead. Audio-sync is now acceptable to me. Problem is now, that some decoders are sloppyly implemented in Windows, like .mpg.

4 hours ago, Kas Ob. said:

the audio sample per frame should always be even number.

I'm not sure I even have control over that. Isn't the encoder just using everything that is in the leaky bucket to write the segments in what it thinks is the right way?

Why can't you compile the code? I think your feedback would be very valuable to me. Did you try the latest version on GitHub?

https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation

(there's one silly bit of code still in there that I need to take out, but it usually doesn't hurt anything)

I was hoping to have eliminated some code that prevented it to run on versions earlier than 11.3. Or have you meanwhile developed an aversion against the MF-headers? 🙂

A first version of TBitmapEncoderWMF is now available at

https://github.com/rmesch/Bitmaps2Video-for-Media-Foundation

It only supports .mp4 and codecs H264 and HEVC(H265) at the moment. See the readme for more details.

I had some problems getting the video-stream to use the correct timing. Apparently the encoder needs to have the video- and audio-samples fed to it in just the right way,

otherwise it drops frames or changes timestamps at its own unfathomable judgement.

I think I solved it by artificially slowing down the procedure that encodes the same frame repeatedly, and by reading ahead in the audio-file for "just the right amount".

Would be interested in how it fares on other systems than mine.

1 hour ago, Attila Kovacs said:

I have SO account but as far as I'm concerned, you can strike for a lifetime.

What makes you so negative about this? I certainly have my problems with participation on stackoverflow. After having received some unfair downvotes, I've stopped asking questions, and I only answer, if I'm dead sure of what I say, maybe a good thing.

But stackoverflow has been a valuable and mostly reliable source of information, and I would have to spend much more time finding good info, if I couldn't rely on content being mostly accurate anymore, or it the moderators would stop doing their job. I upvote every answer that has been useful, and I downvote every blatantly wrong answer, just to keep the quality up. I would hate for ChatGPT flooding the answers.

10 hours ago, maXcomX said:

However an av will never happen unless the API is wrong translated or when you use sloppy code.

Well, I must have written lots of sloppy code yesterday, but I can now encode audio to AAC together with video to H264 or H265. I'm using the CheckFail approach wherever possible. I wish I could keep all these attribute names in my head. Wonder whether it would be possible collecting them into records, so you could consult code completion about them.

1 hour ago, maXcomX said:

Learn C++ I would say. 

No thanks 🙂. But I've already translated parts of C++-code to Delphi. They could use stuff like Break_On_Fail(hr) and more, made me a bit jealous.

You wouldn't per chance know why I can't mux any audio into an HEVC-encoded video? The video stream is all there, but it seems to be missing the correct stream header. So only the audio is being played.

You are right. But then I have to think about whether or not a call is likely to produce havoc, and declare several new variables.

BTW: WriteSample pushes the sample to the sinkwriter and does not just set an index.

What do you think about the following construct instead:

procedure TBitmapEncoderWMF.WriteOneFrame;
var
  pSample: IMFSample;
  Count: integer;

const
  ProcName = 'TBitmapEncoderWMF.WriteOneFrame';
  procedure CheckFail(hr: HResult);
  begin
    inc(Count);
    if not succeeded(hr) then
      raise Exception.Create('Fail in call nr. ' + IntToStr(Count) + ' of ' +
        ProcName + ' with result ' + IntToStr(hr));
  end;

begin
  Count := 0;
  // Create a media sample and add the buffer to the sample.
  CheckFail(MFCreateSample(pSample));

  CheckFail(pSample.AddBuffer(pSampleBuffer));

  CheckFail(pSample.SetSampleTime(fWriteStart));

  CheckFail(pSample.SetSampleDuration(fSampleDuration));
  // Send the sample to the Sink Writer.
  CheckFail(pSinkWriter.WriteSample(fstreamIndex, pSample));

  inc(fWriteStart, fSampleDuration);
  fVideoTime := fWriteStart div 10000;
  inc(fFrameCount);
end;

@programmerdelphi2k

What if hrSampleBuffer is not S_Ok, causing pSinkWriter.WriteSample to fail with an AV or such? You would get an exception the source of which would be harder to trace.

Or am I just too dense to understand :). (It's late)

1 hour ago, Anders Melander said:

I would suggest you add it as a Git submodule.

Thanks, sounds like it's just the thing needed. Just need to figure out how.

Error: uToolsWMF uses a unit Z_prof. Entry can be safely deleted. Comes from having too much stuff in the path.

Meanwhile I think I figured out how to improve the encoding quality.

Around line 270 in uBitmaps2VideoWMF.pas make the following changes:

if succeeded(hr) then
    hr := MFCreateAttributes(attribs, 4);  //<--------- change to 4 here
  // this enables hardware encoding, if the GPU supports it
  if succeeded(hr) then
    hr := attribs.SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, UInt32(True));
  // this seems to improve the quality of H264 and H265-encodings:
  {*************** add this *********************************}
  // this enables the encoder to use quality based settings
  if succeeded(hr) then
    hr := attribs.SetUINT32(CODECAPI_AVEncCommonRateControlMode, 3);
  {**************** /add this *******************************}
  if succeeded(hr) then
    hr := attribs.SetUINT32(CODECAPI_AVEncCommonQuality, 100);
  if succeeded(hr) then
    hr := attribs.SetUINT32(CODECAPI_AVEncCommonQualityVsSpeed, 100);

Besides, I'm getting sick and tired of all those if succeeded...
 

I have worked on a port of my Bitmaps2Video-encoder to using Windows Media Foundation instead of ffmpeg, since I wanted to get rid of having to use all those dll's.
Now, before posting it on GitHub, I'd like to run it by the community because of my limited testing possibilies.

I also hope that there are people out there having more experience with MF and could give some suggestions on the problems remaining (see below).
Learning how to use media foundation certainly almost drove me nuts several times, because of the poor quality of the documentation and the lack of examples.

What is does:
  Encodes a series of bitmaps to video with the user interface only requiring basic knowledge about videos.
  Can do 2 kinds of transitions between bitmaps as an example of how to add more.
  Supports file formats .mp4 with encoders H264 or H265, or .wmv with encoder WMV3.
  Does hardware encoding, if your GPU supports it, falls back to software encoding otherwise.
  Uses parallel routines wherever that makes sense.
  Experimental routine to mux in mp3-audio. Only works for H264 and WMV3 right now.

Requirements:
  VCL-based.
  Needs the excellent MF headers available at https://github.com/FactoryXCode/MfPack.
    Add the src-folder of MFPack to the library path, no need to install a package.
  Needs to run on Windows10 or higher to make use of all features.
  Not sure about Delphi-version required, guess XE3 and up is required for sure.

Problems remaining:
  I'm not too thrilled about the encoding quality. Might be a problem with my nVidia-card.
  The audio-muxer should work for H265, because it works when I use ffmpeg. But with my present routine the result just plays the audio and shows no video.
  I haven't yet figured out how to insert video clips. Major problem I see is adjusting the frame rate.

Renate

Bitmaps2VideoWMF.zip

14 hours ago, Remy Lebeau said:

You don't need to scan the full 2nd level right away, though.

Good idea. Going back to FindFirst/FindNext might have been the better design right from the beginning, but right now I'm satisfied with the speed. I just had to fix the VCL-tree, because I forgot that the darn thing recreates its Window-handle on any DPI-change. Arghh!

You can see the brandnew explorer-trees in action here: https://github.com/rmesch/Parallel-Bitmap-Resampler. See the demos ThreadsInThreads*

Renate

9 hours ago, programmerdelphi2k said:

NOW, all works as expected!

Yes, it works, but it is still too slow to be useable as an explorer tree. Let me make a bold statement:

You will never be able to create a functioning explorer tree with the plain TTreeview in FMX, if you insist on reading in the full directory tree of a root directory.

2 hours ago, Fr0sT.Brutal said:

But you can scan one level deeper beforehand to show correct buttons.

Yes, that's why I want to scan 2 levels at a time, so the correct expand buttons are always shown. But for this I need to also know when the user expands a node without actually selecting it. By defining an interposer class for TTreeViewItem, which overrides TTreeViewItem.SetIsExpanded, I can now intercept this event. I would have been interested in learning, whether anybody sees a problem with this approach. I don't see any in the use case I need it for.

Renate

1 hour ago, programmerdelphi2k said:

programmerdelphi2k

Nevermind, I have it working my way, it's fast, and I can also avoid all those errors. But thanks for your time.

Renate

2 hours ago, programmerdelphi2k said:

maybe some like this; 

The speed isn't too bad, certainly better than filling the nodes recursively. But it still takes several minutes to display the content of my personal folder, whereas in my version you'd see it almost instantly.

There must be something wrong with the logic in your MyFillTreeView. It never displays sibling nodes right. This is for example the output I get when I choose the Embarcadero folder under documents as root:

I also really want to avoid reading in all directories in the root.

40 minutes ago, Remy Lebeau said:

Why not Expand() instead?

There was a reason I didn't do that, but I can't remember :).

What do you think about using the interposer class for TTreeViewItem?

Meanwhile I excluded Directories which have faHidden or faSystem, because that was creating errors. Other than that it seems to be working OK, unless you try to expand something like the Windows-folder, that still takes forever.

I would have liked to use TTreeView.ItemExpanded, but as you say, I can't.

I got it to "work" by defining an interposer class for TTreeViewItem

type
  TTreeViewItem = class(FMX.TreeView.TTreeViewItem)
  protected
    procedure SetIsExpanded(const Value: Boolean); override;
  end;

...

procedure TTreeViewItem.SetIsExpanded(const Value: Boolean);
begin
  inherited;
  if IsExpanded then
    if TreeView is TDirectoryTree then
      TDirectoryTree(TreeView).CreateSubNodesToLevel2(self);

end;

It doesn't seem to interfere with the inner workings of TTreeView, but it makes me feel a bit uneasy.

Also whenever you now use an item of the treeview you have to do a typecast TTreeviewItem(aTreeviewItem).

I have attached the a small test project. Tested for Windows platforms only.

DirectoryTest.zip

I'm building a Directory-tree for FMX as a descendent of TTreeview.

Because reading in the full directory-tree of a root-directory takes way too long, I only read in the first 2 levels. After that I only want to create sub-items as necessary for any item that the user selects or expands.

In the VCL-version this can be done by overriding TTreeview.Change and TTreeview.CanExpand.

For the FMX-version I can override TTreeview.DoChange, but I can't find an analogue of CanExpand. So I tried to override MouseDown. But the Treeview never gets the MouseDown when the user clicks the expand-button.

Is there anything else I could do?

Here some of the relevant code:

procedure TDirectoryTree.CreateSubNodesToLevel2(aItem: TTreeViewItem);
var
  DirArraySize1, DirArraySize2, i, j: integer;
  DirArray1, DirArray2: TStringDynArray;
  TreeItem, TreeItem2: TTreeViewItem;
  NewName: string;
begin
  DirArray1 := TDirectory.GetDirectories(GetFullFolderName(aItem));
  DirArraySize1 := Length(DirArray1);
  if DirArraySize1 > 0 then
  begin
    for i := 0 to DirArraySize1 - 1 do
    begin
      NewName := DirArray1[i];
      if aItem.Count = i then
      begin
        TreeItem := TTreeViewItem.Create(self);
        fDirectoryDict.Add(NativeUInt(TreeItem), NewName);
        TreeItem.Text := ExtractFilename(NewName);
        TreeItem.ImageIndex := 0;
        TreeItem.Parent := aItem;
      end
      else
        TreeItem := aItem.Items[i];
      DirArray2 := TDirectory.GetDirectories(NewName);
      DirArraySize2 := Length(DirArray2);
      for j := 0 to DirArraySize2 - 1 do
      begin
        if TreeItem.Count = j then
        begin
          TreeItem2 := TTreeViewItem.Create(self);
          fDirectoryDict.Add(NativeUInt(TreeItem2), DirArray2[j]);
          TreeItem2.Text := ExtractFilename(DirArray2[j]);
          TreeItem2.ImageIndex := 0;
          TreeItem2.Parent := TreeItem;
        end;
      end;
    end;
  end;
end;

function TDirectoryTree.GetFullFolderName(aItem: TTreeViewItem): string;
begin
  Result := fDirectoryDict.Items[NativeUInt(aItem)];
end;

procedure TDirectoryTree.MouseDown(Button: TMouseButton; Shift: TShiftState;
  X, Y: Single);
begin
  if (HoveredItem <> nil) then
    CreateSubNodesToLevel2(HoveredItem);
    inherited;
end;

6 hours ago, Patrick PREMARTIN said:

im.With := im.Bitmap.Width;

This will scale the image by the desktop-DPI. For example my desktop-display is set at 200%. The bitmap is displayed in original size because of im.Wrapmode = Orignal. But the Image will be twice as large as its bitmap, and the scrollbox it's in will have a scrollbar-range which is too big . I have corrected that by dividing the width by im.scene.scale, which works. I was just wondering, whether there isn't a simpler way.

5 minutes ago, programmerdelphi2k said:

and if you place a TImage in a form (or another container), define this container to occupy the entire screen (or the desired size), and, in the TImage, activate the option to use the entire client area of this container.

How does this relate to my question?

3 minutes ago, XylemFlow said:

However, I don't think anyone has suggested a way to get the GPU to handle this.

Well, if one could get one's hands on the DirectDraw-Canvas, and the DirectDraw-RenderingContext, one could write a descendent of FMX-TBitmap which uses the higher-quality setting possible with DirectDraw. I just can't see how, but I'm a newbie.