mitch.terpak

I once tested GPT4.0 for some assembly code. Its actually quite good at explaining and improving Assembly code, but absolutely horrendous at writing it from scratch.

We do, but have been needing to resort to a C++ DLL for some performance critical parts. But our Delphi code still outperformed Intel Math Kernel Library. Main reason was just that the Linux compiled code is unbearable slow. You're underestimating this quite frankly, the amount of time it'd take you to correctly optimize Assembly code instead of relying on the Compiler to do a decent job insane. Once again you're better off writing C++ since it's compiler will do only a tiny bit worse job then hand optimizing Assembly code, and better then Delphi compiler. Agree

Large spectrum between unpaid intern translating or well paid software engineer of course

Yeah, the header files seem easy enough that it can probably do a quite good job. But if you don't know what you're doing and make type mistakes you get an external error that will be very hard to track down. It will probably be a bit stubborn since it's so much code, so you'll have to step through it

Yea the articles linked by Brandon are on the spot. What you need is to make a C-style interface to the DLL's. The header files just tell you how you can implement the functions. I'd estimate an experienced programmer can easily do this within a week. For example struct _DATA_REGISTRATION: typedef struct _REGISTRATION_DATA { VARIANT registrationData; VARIANT signatureData; VARIANT issuingAuthority; } REGISTRATION_DATA; becomes type REGISTRATION_DATA = record registrationData: Variant; signatureData: Variant; issuingAuthority: Variant; end; Some of the IRegistration: #ifndef __IRegistration_INTERFACE_DEFINED__ #define __IRegistration_INTERFACE_DEFINED__ /* interface IRegistration */ /* [unique][helpstring][uuid][dual][object] */ EXTERN_C const IID IID_IRegistration; #if defined(__cplusplus) && !defined(CINTERFACE) MIDL_INTERFACE("0BE1D001-A538-476B-8F59-6594741D7720") IRegistration : public IUnknown { public: virtual /* [source][helpstring] */ HRESULT STDMETHODCALLTYPE Initialize( /* [retval][out] */ LONG *result) = 0; virtual /* [source][helpstring] */ HRESULT STDMETHODCALLTYPE Finalize( /* [retval][out] */ LONG *result) = 0; virtual /* [source][helpstring] */ HRESULT STDMETHODCALLTYPE GetReaderName( /* [in] */ LONG index, /* [out] */ BSTR *readerName, /* [retval][out] */ LONG *result) = 0; Becomes: IRegistration = interface(IUnknown) ['{0BE1D001-A538-476B-8F59-6594741D7720}'] function Initialize(out result: LongInt): HRESULT; stdcall; function Finalize(out result: LongInt): HRESULT; stdcall; function GetReaderName(index: LongInt; out readerName: WideString; out result: LongInt): HRESULT; stdcall; end; I just quickly scrabbled these together to give you an impression.

Delphi is fine, we have a huge code-base and if I had the resources I'd rewrite it in C#. The main gripe that makes it unbearable is the fact Delphi code compiled to Linux is sometimes up to 20x slower. And no code could technically be visual programming no? (no idea if visual programming is done for anything serious except for Unreal Engine)

We were pleased to hear that version 12.0 addressed the deadlock issue. However, we've encountered three major issues since upgrading: 1. Unfortunately, we've observed a 40% performance decrease with the updated threading.pas in version 12.0, which is unacceptable for our needs. The root of the problem appears to be the ThreadPool growing beyond the max workers we set in our custom TThreadPool. This necessitates the creation of new copies for the new thread within TParallel (we manage necessary copies dynamically using a Dictionary with TThreadID as key). The issue in threading.pas seems to stem from this property being explicitly set to true in TThreadPool.Create: constructor TThreadPool.Create; begin inherited Create; // Initialization code FUnlimitedWorkerThreadsWhenBlocked := TRUE; // More initialization code end; despite already having a default value or having been set to False: property UnlimitedWorkerThreadsWhenBlocked: Boolean read FUnlimitedWorkerThreadsWhenBlocked write FUnlimitedWorkerThreadsWhenBlocked default True; This leads to the TThreadPool.TThreadPoolMonitor.GrowThreadPoolIfStarved procedure unnecessarily increasing the number of worker threads. When I added debugging, I noticed it enters the else if FThreadPool.UnlimitedWorkerThreadsWhenBlocked block multiple times per TParallel.for loop. 2. In previous versions of Delphi, we encountered access violations when closing the debugger if we used TParallel.For without modifications (with the default thread pool). Switching to custom thread pools mitigated this issue, but with version 12.0, the problem seems to persist regardless. The access violation in 12.0 (using a custom thread pool) often occurs here: procedure TThreadPool.TThreadPoolMonitor.Execute; begin // Procedure code Signaled := FThreadPool.FMonitorThreadWakeEvent.WaitFor(TThreadPool.MonitorThreadDelay) = TWaitResult.wrSignaled; // More procedure code end; I've also seen it happen on different lines within the same procedure. We can only reproduce this in our VCL application. Our DUnit test project does not have the same issue 3. With the debugger attached, any operation involving TParallel.For runs excruciatingly slow—about 8-10 times slower compared to twice as slow in version 11.3. For full disclosure and additional information we're using https://github.com/pleriche/FastMM5 but none of my reported issues change with it. The base runtime just becomes a lot slower with it, because the built in FastMM4 doesn't do great with our workload. We're reaching out for insights or solutions to these issues, as the current performance and stability impacts are significant for our projects.

Threading.pas from 12.1 is still less performant then 11.3. Growing the WorkerThreads in TThreadPool.TThreadPoolMonitor.GrowThreadPoolIfStarved() via the last statement else if FThreadPool.UnlimitedWorkerThreadsWhenBlocked then for i := 1 to Min(FThreadPool.FQueuedRequestCount, FThreadPool.FMaxLimitWorkerThreadCount div 2 + 1) do FThreadPool.CreateWorkerThread; But its doing so erroneously. This seems to be in my case because the amount of work I do in a TParallel.For is large. So here: procedure TThreadPool.TThreadPoolMonitor.Execute; const MaxInactiveInterval = 30 * 1000; InactiveCountdown = MaxInactiveInterval div TThreadPool.MonitorThreadDelay; var I: Integer; CPUInfo: TThread.TSystemTimes; CpuUsageArray: array[0..TThreadPool.NumCPUUsageSamples - 1] of Cardinal; CurUsageSlot: Integer; ExitCountdown: Integer; AvgCPU: Cardinal; CurMonitorStatus: TThreadPool.TMonitorThreadStatus; Signaled: Boolean; begin NameThreadForDebugging(Format('Thread Pool Monitor Thread - %s ThreadPool - %p', [ClassName, Pointer(FThreadPool)])); {$IFDEF MSWINDOWS} if ThreadPoolMonitorHandles <> nil then begin TMonitor.Enter(ThreadPoolMonitorHandles); try ThreadPoolMonitorHandles.Add(FThreadPool, Handle); finally TMonitor.Exit(ThreadPoolMonitorHandles); end; end; try {$ENDIF MSWINDOWS} FThreadPool.FMonitorThreadWakeEvent.WaitFor(TThreadPool.MonitorThreadDelay); TThread.GetSystemTimes(CPUInfo); CurUsageSlot := 0; FillChar(CPUUsageArray, SizeOf(CPUUsageArray), 0); ExitCountdown := InactiveCountdown; while not Terminated do begin if not FThreadPool.FShutdown then begin Signaled := FThreadPool.FMonitorThreadWakeEvent.WaitFor(TThreadPool.MonitorThreadDelay) = TWaitResult.wrSignaled; FThreadPool.FCurrentCPUUsage := TThread.GetCPUUsage(CPUInfo); CPUUsageArray[CurUsageSlot] := FThreadPool.FCurrentCPUUsage; if CurUsageSlot = TThreadPool.NumCPUUsageSamples - 1 then CurUsageSlot := 0 else Inc(CurUsageSlot); AvgCPU := 0; for I := 0 to TThreadPool.NumCPUUsageSamples - 1 do Inc(AvgCPU, CPUUsageArray[I]); FThreadPool.FAverageCPUUsage := AvgCPU div TThreadPool.NumCPUUsageSamples; if FThreadPool.FCurrentCPUUsage < TThreadPool.CPUUsageLow then GrowThreadPoolIfStarved; CurMonitorStatus := FThreadPool.FMonitorThreadStatus; if Signaled then begin FThreadPool.FMonitorThreadWakeEvent.ResetEvent; Continue; end; if FThreadPool.FShutdown then ExitCountdown := -1 else if not (TThreadPool.TMonitorThreadStat.NoWorkers in CurMonitorStatus) then Dec(ExitCountdown) else ExitCountdown := InactiveCountdown; end else ExitCountdown := -1; if ExitCountdown <= 0 then begin if ExitCountdown < 0 then begin TInterlocked.Exchange(Integer(FThreadPool.FMonitorThreadStatus), 0); Exit; end else if TMonitorThreadStatus(TInterlocked.CompareExchange(Integer(FThreadPool.FMonitorThreadStatus), 0, Integer(CurMonitorStatus))) = CurMonitorStatus then Exit else ExitCountdown := InactiveCountdown; end; end; {$IFDEF MSWINDOWS} finally if ThreadPoolMonitorHandles <> nil then begin TMonitor.Enter(ThreadPoolMonitorHandles); try ThreadPoolMonitorHandles.Remove(FThreadPool); finally TMonitor.Exit(ThreadPoolMonitorHandles); end; end; end; {$ENDIF MSWINDOWS} end; The MonitorThreadDelay is 500ms, but that's not sufficient for our application. So lets say you do a TParallel for from 0-100, you have 20 threads and each task takes 10 seconds. Then it can occur that within a 500ms interval none of the threads have finished work, then this new logic thinks it's deadlocked. If they'd let us set these runtime it'd make a big difference already private const MaxThreadsPerCPU = 2; // Constants used for calculating CPU Usage CPUUsageHigh = 95; // Start retiring/removing threads when CPU usage gets this high CPUUsageLow = 80; // Add more threads if the CPU usage is below this CPUUsageLowest = 20; // Shrink the thread pool when CPU usage falls below this NumCPUUsageSamples = 10; // Keep a running list of CPU Usage samples over which the average is calculated MonitorThreadDelay = 2000; // Was 500 SuspendInterval = 5000 + MonitorThreadDelay; // Interval to use for suspending work in worker threads SuspendTime = MonitorThreadDelay + 100; // Time to spend in SuspendWork; RetirementDelay = 5000; // Delay interval for retiring threads Now I have to maintain my own Threading.pas still...

I don't completely understand your problem but I'm assuming the issue is the following. You're running something that has to be done on the background and you don't want your application to freeze up / be unresponsive until its done. Or you want to wait until a specific thing is done. https://docwiki.embarcadero.com/RADStudio/Athens/en/Using_TTask_from_the_Parallel_Programming_Library TTask.Run(procedure begin // Whatever async thing you want to run end); Let's say now you want to wait until its done, or check at least. var aTask : ITask; aTask := TTask.Run(procedure begin // Whatever async thing you want to run end); aTask.Wait(); What's important is that when you do something on VCL you do it on the main thread. The TTask.Run is always on a separate thread from the main. You'll have to use https://docwiki.embarcadero.com/Libraries/Alexandria/en/System.Classes.TThread.Synchronize to call the main thread and do VCL things from an async thread. aTask.Wait might achieve the opposite of what you want: Locking the main thread. If for some reason you have to use it elsewhere there's also a TTaskStatus that you could check. Or you can make another TTask.Run and put aTask.Wait inside it. Hope this helps!

Delphi compiled to Linux is borderline unusable, whatever the reason is, I'm seeing x20 slowdown on some code (see screenshot). For example in a procedure that does Sparse Matrix Factorization. I had to port parts of our code to C++ (DLL) and will probably have to port more as work around for this. Sure it can compile longer, even if it'd compile for 30min I don't mind.

For Linux64 to not compile to something 5-20x slower then Windows.

Actual use case: if not LoadflowObjectDict.TryGetValue(TThread.CurrentThread.ThreadID,LoadflowObject) then begin LoadflowObject := TLoadflowObject.Create(); // Assignment of new object // Copy of a large array // Setlength in the range of 10.000-50.000 // Move of a large array // Copy of a large array // Deepcopy of a 100-300mb object (object size in memory) // Deepcopy of a 50-100mb nested object (object size in memory) CriticalSection.Acquire; LoadflowObjectDict.TryAdd(TThread.CurrentThread.ThreadID,LoadflowObject); CriticalSection.Release; end; My test looked like this, note how it has nothing in front of the TryAdd that will introduce a lot of variance in the timings. The test was absolutely worse case. Adding the CriticalSection is quite insignificant. Just very noticeable on small problems. program ParallelLoopConsoleApp; {$APPTYPE CONSOLE} uses System.SysUtils, System.Classes, System.Generics.Defaults, System.Threading, System.Generics.Collections; var LoopCount: Integer = 0; ThreadDataDict: TDictionary<Cardinal, TThreadData>; procedure PerformCalculation; var CoreCount: Integer; LoadflowThreadpool : TThreadpool; begin CoreCount:=TThreadPool.Default.MaxWorkerThreads div 2; ThreadDataDict:=TDictionary<Cardinal,TThreadData>.Create(1000); LoadflowThreadpool := TThreadpool.Create; LoadflowThreadpool.SetMaxWorkerThreads(CoreCount); LoadflowThreadpool.SetMinWorkerThreads(CoreCount); while True do begin Inc(LoopCount); Writeln('Loop Count: ',LoopCount); TParallel.For(1,100, procedure(Index: Integer) var ThreadData: TThreadData; begin // Store or update thread-specific data // This is not what actually happens though, before this other things happen that make it very unlikely for threads to be synced. But for arguments sake if not ThreadDataDict.TryGetValue(Tthread.CurrentThread.ThreadID,ThreadData) then ThreadDataDict.TryAdd(Tthread.CurrentThread.ThreadID,ThreadData); // Adding a criticalsection here would make it safe // Perform some calculation here ThreadData.CalculationResult:=Index * Index div 17 + 231 * 2 - 4; end, LoadflowThreadpool); // Cleanup and prepare for the next iteration ThreadDataDict.Clear; end; end; begin try PerformCalculation; except on E: Exception do Writeln('An error occurred: ', E.Message); end; Writeln('Press Enter to exit...'); end. The reason I'm adjusting it now is because I think there's a quite significant collision chance on generating the bucket ID. Which can't be easily avoided. And I'd rather avoid this causing exceptions in a Docker using a DLL

Alright, I tested it in a console app it has a failure rate of about 1 / 35000 with 20 cores, that's too much. I wrapped adding new ThreadID's / ThreadObjects with a TCriticalSection. Thank you for making me reconsider my stubbornness.

Not to justify it, you're right, it's technically not thread-safe. But the odds for it going wrong are very low, if it doesn't cause an exception then it might cause a 300mb memory leak when it goes wrong. It's a conscious trade-off for a tiny bit more performance. A CriticalSection would solve this when adding the objects to the Dictionary.

Note that the person replying there Dmitry Arefiev is the same person that mentioned the solution here.

I tested this and it indeed works well for the smaller problems I have. It still seems to trigger: else if FThreadPool.UnlimitedWorkerThreadsWhenBlocked then begin writeln('entered'); for i := 1 to Min(FThreadPool.FQueuedRequestCount, FThreadPool.FMaxLimitWorkerThreadCount div 2 + 1) do FThreadPool.CreateWorkerThread; end; While there seems no reason for that. Which does end up being a problem for me on the larger problem sizes. (since it will still add new worker threads with different ThreadID's for which copies will have to be made). But I can confirm that it cuts the issue down to only a 15% performance loss in our benchmark opposed to 40% with UnlimitedWorkerThreadsWhenBlocked on true. I also got the ACCESS_VIOLATION again while attempting to test the performance with the debugger. This time self was inaccessible and it occurred on. I also had it happen for the first time ever in our DUnitX (console): FThreadPool.FCurrentCPUUsage := TThread.GetCPUUsage(CPUInfo); Name Value Self Inaccessible value I 10 CPUInfo (594602031250, 37442656250, 616291093750, 0) CpuUsageArray (5, 4, 5, 6, 5, 4, 4, 5, 5, 5) CurUsageSlot 3 ExitCountdown 37 AvgCPU 48 CurMonitorStatus [Created] Signaled False

Great question. It's indeed missing quite a bit of the actual implementation. Basically we're presizing the Dictionary to assure that in no circumstance at all it would be prompted to resize. Then we use a custom hash just to be extra sure that collisions from the ThreadID's are extremely unlikely. We used to have a CriticalSection before adding the LoadflowObject to the Dictionary. But after thorough testing, Infinite loop left running for days on an unused laptop that would continuously call TParallel.For with this Dictionary pattern has given me extremely high confidence in the addition of the item being threadsafe and not prone to race conditions. Technically I could see an hash being produced from an ThreadID that's so similar that it'd be inserted at the same index, but there's collision detection for that in the Dictionary itself, so it'd really have to be a simultaneous insert. The performance increase of removing the CriticalSection was about 3-4% for a standard calculation, so together with the testing with deemed it worth. Furthermore in practice we also see the threads being spun up with a slight delay, which further makes it unlikely that this collision occurs. We add the pointer to the Dictionary right away, so the natural delay that occurs with a TParallel.For is also contributing to the consistency of this implementation. And obviously the advantage is a dynamic high performance implementation that scales well across whatever amount of threads the system has. We for example had issues in the past where we chunked the calculations based on the amount of threads. But we would find on devices with 20+ threads the first few threads would already be "done" with the problem before the last ones started. Adding a few hundred milliseconds of extra duration to the calculations. Let me know if you have more questions about this.

I know, but that doesn't solve the issue with debugger performance and access violations. So our current solution is to not upgrade to 12.0. But yes you're right in the initialization of the pools I should just adjust it in 12.0. And I would have to be mindful no nested TParallel.For's occur. (Sidenote: often nesting TParallel For's ain't that useful anyway, a specific work that can't be made parallel at a higher level is rare, honestly I can't think of many realistic use cases where this would be the most performant solution) Also part of my remark is just that we identified this issue. And it does probably mean that the system thinks it's "blocked" when it ain't. Which might also impact other Delphi Pascal users. While for us there's a clear potential workaround. I do think the way this deadlock has been "fixed" may potentially not be optimized properly. It's just this one and there's an async thread that updates the progressbar but that's insignificant. The mainthread is on a ShowModal (progressbar). We're mainly using TParallel.For for it's workstealing capabilities. And on PC's with more then 16 threads we're intentionally not using two because it can help make the system feel more responsive while running the calculation.

There's no nested TParallel.For's though. Agree, definitely. My point is that these things seem to occur going from 11.3 -> 12.0. And my personal assessment would be that the deadlock check is triggered too easily. Here's the slimmed down version of what we're doing. Note that a large performance drawback occurs when the maxworker threads get put to 40 for example. Then we automatically make 22 more copies of everything we need because the ThreadID key isn't in the dictionary containing the reusable LoadflowObject function LoadflowMainParallel: boolean; var // Core threading and control variables CoreCount: integer; begin Result := FALSE; // Determine optimal number of threads based on system capabilities and specific workload CoreCount := TThreadpool.Default.MaxWorkerThreads div 2; // MaxWorkerThreads is always threads x 2, this is an easy way to get threads on Windows/Linux // Adjust CoreCount for efficiency based on ID4 calculation results // Example: Use fewer cores for smaller workloads to avoid overhead // Setup thread pool with optimized number of worker threads TThreadUtils.Instance.GetLoadflowThreadpool.SetMaxWorkerThreads(CoreCount); // For example 18 out of 20 available TThreadUtils.Instance.GetLoadflowThreadpool.SetMinWorkerThreads(CoreCount); // For example 18 out of 20 available // Parallel execution of loadflow calculations TParallel.For(iLF_begin, iLF_eind, procedure(iLF: integer) var // Threading-specific loadflow objects LoadflowObject: TLoadflowObject; begin // Check and/or initialize LoadflowObject for the current thread // Add them to a Dictionary with ThreadID as key // Perform network-related calculations and adjustments // Core loadflow calculation per iteration // There's no more threading inside the calculation. The only thing noteworthy that happens is calling a C++ DLL to speed up specifc functions. To avoid the slowdown that Delphi experiences when compiled to Linux (but that's another issue) // Conditional adjustments and post-calculation cleanup end, TThreadUtils.Instance.GetLoadflowThreadpool); // Cleanup and finalization of resources used in parallel calculations // Set result to indicate success or failure of the operation Result := TRUE; end; For clarification Instance is just a singleton pattern. GetLoadflowThreadpool returns a custom TThreadpool that's made with a double-locking pattern if it doesn't exist function TThreadUtils.GetLoadflowThreadPool: TThreadPool; begin if not Assigned(LoadflowThreadPool) then begin aThreadLock.Enter; try if not Assigned(LoadflowThreadPool) then begin LoadflowThreadPool := TThreadPool.Create; LoadflowThreadPool.SetMinWorkerThreads(TThreadPool.Default.MaxWorkerThreads div 2); LoadflowThreadPool.SetMaxWorkerThreads(TThreadPool.Default.MaxWorkerThreads div 2); end; finally aThreadLock.Leave; end; end; Result := LoadflowThreadPool; end; But perhaps this might triggers the nested behavior aTask:=TTask.Run( // Make this async so we can start the calculation while using ShowModal procedure begin LoadflowMainParallel(); // Close progress form (so we don't hang on ShowModal) end,TThreadUtils.Instance.GetSmallThreadpool); TCalculationProgressForm.GetInstance.ShowModal; aTask.Wait(); TThreadUtils.Instance.RepeatedTask.Cancel; // Async repeated task that updates the progress bar every 250ms until cancelled

Yeah... This seemed like a good idea to at least raise awareness. The details are clear. The deadlock detection is way too sensitive. And quite frankly its just not implemented properly. The deadlock fallback is being triggered within milliseconds of a TParallel.For starting. That's a very clear issue. If there's specific details missing I can of course do my best to deliver them.

Thank you for your insights regarding property defaults and constructor behavior. While helpful, the core issue remains with the system library's thread pool expansion beyond the specified limit for no clear reason. This over-sensitive mechanism to prevent deadlocks, where none exist in our TParallel.for implementation, suggests a misidentification problem. This is a blocker for upgrading to Delphi 12 for us. Regarding the access violation, despite all components appearing correctly referenced, the exact cause remains unclear. As this involves a system library, it's beyond our scope to debug extensively due to professional constraints and the fact that editing system libraries is something we'd like to avoid. The goal of my post is to highlight these blockers to upgrading to Delphi 12, hoping for a resolution. Detailed debugging is beyond our capacity, what I'd like are actionable solutions from the system's developers or having this be a priority for the next patch. I appreciate any further assistance or insights to address these critical issues.