Delphi 12.0 TParallel.For performance. Threading.pas issues

[Vincent Parrett 750]

13 hours ago, Anders Melander said:

Looking at the Threading unit, it's rare to see professional code with this few comments. Is there some sort of rule within Embarcadero against commenting the code?

I believe Embarcadero strip comments as part of their build process. 

[Vincent Parrett 750]

13 hours ago, Dalija Prasnikar said:

I have written tons of self explanatory code, for which now I don't know what was I smoking at the time.

Haven't we all 🙋‍♂️ 😉 - I've been making a point lately to review code and just write comments when anything is unclear (once I figure out what I was thinking).

[Dalija Prasnikar 1396]

7 hours ago, Vincent Parrett said:

I believe Embarcadero strip comments as part of their build process. 

If they do that, some comments miraculously survive that process and others don't.

[Vincent Parrett 750]

On 2/16/2024 at 5:33 PM, Dalija Prasnikar said:

If they do that, some comments miraculously survive that process and others don't.

I was told this by an embarcadero (actuallly, it was probably borland at the time) employee when I queried what happened to the comments in some code I donated. 

 

In the DUnitX source they distribute, TODO comments are all stripped but not regular comments.  

[Anders Melander 1783]

16 hours ago, Vincent Parrett said:

I was told this by an embarcadero (actuallly, it was probably borland at the time) employee when I queried what happened to the comments in some code I donated. 

I've heard this too, but it was many, many years ago.

 

16 hours ago, Vincent Parrett said:

In the DUnitX source they distribute, TODO comments are all stripped but not regular comments.

Well that explains the complete lack of comments. All their comments are like:

// TODO : Document this, FFS!

and

// TODO : WFT is this shit?

 

[Anders Melander 1783]

Just now, Anders Melander said:

All their comments are like:

Whoops. I just remembered who wrote that code 🙂

[mitch.terpak 5]

On 2/15/2024 at 1:06 PM, Kas Ob. said:

assuming these cores doing near nothing except your stress test, right ?

Threading.pas from 12.1 is still less performant then 11.3. Growing the WorkerThreads in TThreadPool.TThreadPoolMonitor.GrowThreadPoolIfStarved() via the last statement

 

        else if FThreadPool.UnlimitedWorkerThreadsWhenBlocked then
          for i := 1 to Min(FThreadPool.FQueuedRequestCount, FThreadPool.FMaxLimitWorkerThreadCount div 2 + 1) do
            FThreadPool.CreateWorkerThread;

But its doing so erroneously. 

 

This seems to be in my case because the amount of work I do in a TParallel.For is large. So here:
 

procedure TThreadPool.TThreadPoolMonitor.Execute;
const
  MaxInactiveInterval = 30 * 1000;
  InactiveCountdown = MaxInactiveInterval div TThreadPool.MonitorThreadDelay;
var
  I: Integer;
  CPUInfo: TThread.TSystemTimes;
  CpuUsageArray: array[0..TThreadPool.NumCPUUsageSamples - 1] of Cardinal;
  CurUsageSlot: Integer;
  ExitCountdown: Integer;
  AvgCPU: Cardinal;
  CurMonitorStatus: TThreadPool.TMonitorThreadStatus;
  Signaled: Boolean;
begin
  NameThreadForDebugging(Format('Thread Pool Monitor Thread - %s ThreadPool - %p', [ClassName, Pointer(FThreadPool)]));
{$IFDEF MSWINDOWS}
  if ThreadPoolMonitorHandles <> nil then
  begin
    TMonitor.Enter(ThreadPoolMonitorHandles);
    try
      ThreadPoolMonitorHandles.Add(FThreadPool, Handle);
    finally
      TMonitor.Exit(ThreadPoolMonitorHandles);
    end;
  end;
  try
{$ENDIF MSWINDOWS}
  FThreadPool.FMonitorThreadWakeEvent.WaitFor(TThreadPool.MonitorThreadDelay);
  TThread.GetSystemTimes(CPUInfo);
  CurUsageSlot := 0;
  FillChar(CPUUsageArray, SizeOf(CPUUsageArray), 0);
  ExitCountdown := InactiveCountdown;
  while not Terminated do
  begin
    if not FThreadPool.FShutdown then
    begin
      Signaled := FThreadPool.FMonitorThreadWakeEvent.WaitFor(TThreadPool.MonitorThreadDelay) = TWaitResult.wrSignaled;
      FThreadPool.FCurrentCPUUsage := TThread.GetCPUUsage(CPUInfo);
      CPUUsageArray[CurUsageSlot] := FThreadPool.FCurrentCPUUsage;
      if CurUsageSlot = TThreadPool.NumCPUUsageSamples - 1 then
        CurUsageSlot := 0
      else
        Inc(CurUsageSlot);
      AvgCPU := 0;
      for I := 0 to TThreadPool.NumCPUUsageSamples - 1 do
        Inc(AvgCPU, CPUUsageArray[I]);
      FThreadPool.FAverageCPUUsage := AvgCPU div TThreadPool.NumCPUUsageSamples;
      if FThreadPool.FCurrentCPUUsage < TThreadPool.CPUUsageLow then
        GrowThreadPoolIfStarved;
      CurMonitorStatus := FThreadPool.FMonitorThreadStatus;
      if Signaled then
      begin
        FThreadPool.FMonitorThreadWakeEvent.ResetEvent;
        Continue;
      end;
      if FThreadPool.FShutdown then
        ExitCountdown := -1
      else if not (TThreadPool.TMonitorThreadStat.NoWorkers in CurMonitorStatus) then
        Dec(ExitCountdown)
      else
        ExitCountdown := InactiveCountdown;
    end else
      ExitCountdown := -1;
    if ExitCountdown <= 0 then
    begin
      if ExitCountdown < 0 then
      begin
        TInterlocked.Exchange(Integer(FThreadPool.FMonitorThreadStatus), 0);
        Exit;
      end else if TMonitorThreadStatus(TInterlocked.CompareExchange(Integer(FThreadPool.FMonitorThreadStatus),
        0, Integer(CurMonitorStatus))) = CurMonitorStatus then
        Exit
      else
        ExitCountdown := InactiveCountdown;
    end;
  end;
{$IFDEF MSWINDOWS}
  finally
    if ThreadPoolMonitorHandles <> nil then
    begin
      TMonitor.Enter(ThreadPoolMonitorHandles);
      try
        ThreadPoolMonitorHandles.Remove(FThreadPool);
      finally
        TMonitor.Exit(ThreadPoolMonitorHandles);
      end;
    end;
  end;
{$ENDIF MSWINDOWS}
end;

The MonitorThreadDelay is 500ms, but that's not sufficient for our application. So lets say you do a TParallel for from 0-100, you have 20 threads and each task takes 10 seconds. Then it can occur that within a 500ms interval none of the threads have finished work, then this new logic thinks it's deadlocked.

 

 

If they'd let us set these runtime it'd make a big difference already

  private const
    MaxThreadsPerCPU = 2;
    // Constants used for calculating CPU Usage
    CPUUsageHigh = 95; // Start retiring/removing threads when CPU usage gets this high
    CPUUsageLow = 80; // Add more threads if the CPU usage is below this
    CPUUsageLowest = 20; // Shrink the thread pool when CPU usage falls below this
    NumCPUUsageSamples = 10; // Keep a running list of CPU Usage samples over which the average is calculated
    MonitorThreadDelay = 2000; // Was 500
    SuspendInterval = 5000 + MonitorThreadDelay; // Interval to use for suspending work in worker threads
    SuspendTime = MonitorThreadDelay + 100; // Time to spend in SuspendWork;
    RetirementDelay = 5000; // Delay interval for retiring threads

Now I have to maintain my own Threading.pas still...

Edited April 8 by mitch.terpak