Parallel.For and Timer

[Benmik 1]

I read JPG files from disk into a TObjectList and create thumbnails from them that are painted onto a TImage. For creating thumbnails and displaying them I have been using AsyncCalls by Andreas Hausladen for a long time, but now I want to switch to OTL because AsyncCalls isn't maintained any longer.

Let's say there are 800 JPG in a directory, the first 30 of which fit on the screen. So I force an update of the TImage every time one of these 30 thumbnails has been painted. After that I display a message on a TStatusBar every 50 thumbnails or every second, whichever comes first. I noticed a considerable time lag of a few seconds before the first thumbnail appeared on the screen, so I tried to use a self created thread pool which didn't help much.

 ThumbPool := CreateThreadPool('VSBPool');
 ThumbPool.MaxExecuting := Environment.Process.Affinity.Count - 1;
 ThumbPool.MaxQueued := JPGList.Count - 1;
 Parallel.For(0,JPGList.Count - 1)
   .TaskConfig(Parallel.TaskConfig.ThreadPool(VSBPool))
//   .TaskConfig(Parallel.TaskConfig.MonitorWith(OmEvMon))
   .NumTasks(Environment.Process.Affinity.Count - 1)
   .Execute(
    procedure (i:integer)
    begin
      CreateThumbs(JPGList[i]);
    end);

Everything works OK, except for I cannot get the status bar to display the progress. I use a global variable for the count of the produced thumbnails that is incremented (TInterlocked.Increment) in "CreateThumbs". With AsyncCall's "TAsyncCalls.VCLInvoke(procedure begin DisplayProgress end)" in "CreateThumbs" this worked like a charm. But not in OTL. What I have tried so far:

 

1. A simple timer firing "DisplayProgress" every 500 milliseconds. Timer apparently doesn't get to fire.

2. Using TOmniEventMonitor. Warning: 'TOmniTaskControl.SetMonitor: Task can be only monitored with a single monitor'

3. Using Parallel.Async() + "DisplayProgress". Program is slowed down to a creeping pace.

 

What's the trick, what needs to be  done? Is Parallel.For the right option here? What can I do do get rid of that time lag (singlethreaded the thumbnails appear instantaneously)?

Edited June 18, 2020 by Benmik

[Primož Gabrijelčič 223]

Parallel.For by default works in "blocking" mode. IOW, while the Parallel.For is being executed, owner thread (main thread in your case) does nothing else. Most importantly - it does not process messsages.

 

There are two ways around this problem.

1.  Use the .NoWait qualifier: FFor := Parallel.For(...).TaskConfig(...).NumTasks(...).NoWait.Execute(...). (http://www.omnithreadlibrary.com/book/chap06.html#highlevel-for, section 3.11.1) Read here why you have to store the result of the Parallel.For call into a field: http://www.omnithreadlibrary.com/book/chap06.html#leanpub-auto-a-life-cycle-of-an-abstraction.
2. Execute the Parallel.For asynchronously via Parallel.Async mechanism (http://www.omnithreadlibrary.com/book/chap06.html#highlevel-async):

   Parallel.Async(
     procedure begin
       Parallel.For....Execute();
     end);

    

 

[Benmik 1]

Thank you so much, Primož. I had read everything but missed out on the fact that Parallel.For is blocking.

Right now it goes like this:

 var FFor:IOmniParallelSimpleLoop;
 ... 
 Timer.enabled := True;
 FFor := Parallel.For(0,JPGList.Count - 1)
   .NumTasks(Environment.Process.Affinity.Count - 1);
 FFor.Execute(procedure (i:integer)
   begin
     CreateThumbs(JPGList[i]);
   end);
 While not FFor.WaitFor(250) do
   Application.ProcessMessages;
 Timer.enabled := False;
 FFor := nil;

The timer fires now OK and due to "Application.ProcessMessages" the progress is being displayed in the status bar. It's a bit slower then before, though.

I'm still experimenting a little.