Darian Miller

I've seen multiple cases of a timeout of 200ms returning in 160ms when running within VMWare. My first stress test included a fudge factor for returning too early just for that reason.

A new Quality Portal issue could be raised to ensure that the timeout values of the queue are >= 16ms. Anything less is apparently not achievable based on earlier comments in this thread.

1. Windows 10 build 18363. i7-7700K 4.2GHz, 4 cores, 8 logical processors, 64GB RAM. 2. pyscripter explained why a low pop timeout fails earlier in this thread and is why there's a minimum of 20ms in the stress tester now. The test originally failed less in a virtual environment (VMWare Workstation) so I switched to testing on physical machine. On Win32, you can try throwing more threads and run into problems, likely more with Windows than Delphi. I tested with 1,000 threads without any issue. Win64 could handle 50,000 threads without failing.

Two stress tests just completed without error after running for 24 hours straight: Win32 and Win64 running 1,000 threads each with a 20ms timeout with your latest MonitorWaitStackFix.pas.

I updated the GitHub repo with your changes and am running two more day-long tests (win32+win64) both with 1,000 threads and 20ms pop-timeout on Windows 10 physical machine. My last 24-hour Win32 stress test completed without failure. https://github.com/ideasawakened/DelphiKB/commits/master/Delphi Tests/Source/rtl/common/System.Generics.Collections/TThreadedQueue/StressTestPopItem

Is there a way to import the old versions into the ELC database? I just started this ELC hosted license for 10.3 but I should have access to the previous versions but they aren't available in my registered downloads and they don't show up in ELC. I can probably manually find/download the ISO in CodeCentral but the updates are in the MyRegisteredDownloads...not sure how a network license user is supposed to properly handle old versions and I've tried the official support channel twice. Thanks for any help!

Thanks! It works...but doesn't show 10.4 Sydney, rather still 10.3 Rio. Is that normal?

There's always hope for 10.4.1!

Now that 10.4 is out... what's the "proper" way of updating ELC to add a new version? I poked-n-hoped and it allowed me to continuing installing 10.4 but there doesn't seem to be much information available. Any guidance is appreciated! Might make a good blog post for you... : ) (Apologize for duplicate posts, mouse clicked too fast!)

Now that 10.4 is out... what's the "proper" way of updating ELC to add a new version? I poked-n-hoped and it allowed me to continuing installing 10.4 but there doesn't seem to be much information available. Any guidance is appreciated! Might make a good blog post... : )

It's mentioned: https://blog.marcocantu.com/blog/2020-march-getready104beta.html https://www.tatukgis.com/Blogs/Blog/May-2020/RAD-Studio-10-4-Sydney-is-coming.aspx https://community.idera.com/developer-tools/b/blog/posts/delphi-10-4-sydney---delphi104---tag-comingsoon List of links available: https://github.com/ideasawakened/DelphiKB/wiki/D27.SYDNEY.10.4.0.0

I removed your sleep and used an event to start all threads after they have all been created, your latest additions quickly fail in Win32 using the latest version of Delphi. iaStressTest.TThreadedQueue.PopItem.pas

I set the max thread count to 250 and it failed at 183 threads in 10 seconds. So I dropped it down to 175 and it lasted longer 2020-05-17 11.19.17: StressTestPopItem Start: Waiting up to [300] seconds for PopItem to prematurely timeout. 2020-05-17 11.19.17: Note: Using [10] as PopTimeout on TThreadedQueue creation 2020-05-17 11.19.18: TThreadCreator Start: Creating up to [175] threads 2020-05-17 11.19.18: Note: Creating threads with a StackSize of [65536] 2020-05-17 11.19.32: TThreadCreator End: [175] worker threads created 2020-05-17 11.19.43: New Wait objects created: 100000 2020-05-17 11.19.54: New Wait objects created: 200000 2020-05-17 11.20.06: New Wait objects created: 300000 2020-05-17 11.20.17: TTestThread ERROR: TThreadedQueue.PopItem returned [wrTimeout] unexpectedly after [0ms]

I set the max_worker_threads to 200 in Win32 and it bombed after 4 seconds and only 78 threads created. It's definitely hit-n-miss 2020-05-17 10.41.16: StressTestPopItem Start: Waiting up to [60] seconds for PopItem to prematurely timeout. 2020-05-17 10.41.16: Note: Using [10] as PopTimeout on TThreadedQueue creation 2020-05-17 10.41.17: TThreadCreator Start: Creating up to [200] threads 2020-05-17 10.41.17: Note: Creating threads with a StackSize of [65536] 2020-05-17 10.41.20: TTestThread ERROR: TThreadedQueue.PopItem returned [wrTimeout] unexpectedly after [0ms] 2020-05-17 10.41.20: TThreadCreator Note: aborting thread creation at [78] threads as a failure has already been detected. 2020-05-17 10.41.20: TThreadCreator End: [78] worker threads created 2020-05-17 10.41.21: StressTestPopItem End: After [4] seconds, a failure of PopItem was detected in at least one thread 2020-05-17 10.41.21: Hit <enter> to exit

On my Windows 10 system, the Win32 version of your code fails after 6 seconds with about 130 threads created before PopItem throws a premature timeout. (10.3.3)

I also have a wiki page on 10.4 containing many of these links that I try to keep current: https://github.com/ideasawakened/DelphiKB/wiki/D27.SYDNEY.10.4.0.0

If that's aimed my way, I'll gladly take it. I had more days than I can count starting at 8am and leaving work the next afternoon without sleep due to some crisis. CTO for 22 years at a highly active software company where we were always under-staffed, over-promised, and working on a shoe string budget (privately held, one man owner for much of that time until I became a minority owner.) I don't miss it, much. (My wife certainly doesn't.) The cool thing - we had 11 Delphi developers at one point and I was trying to hire another when the VC people showed up and bought us out. But even with that many developers I left the place with 5,000 open tickets on the tech side of the house, after spending a solid year concentrating on getting it down from a much higher number. (My custom built trouble-ticket system had 500,000+ tickets in its mid-2004 to late 2018 history.) But my history suggests - take the walks. There's always a crisis. And if you want some useless code tested for fun, send it my way. I may just do it.

Just out of curiosity, I ran the same test program on the same Windows 10 virtual machine that has many versions of Delphi installed. I built/ran the console project below multiple times on each Delphi version and they all resulted in a GetLastError of 0 when a new thread is spawned: Delphi 5, 7, 2005, 2009, 2010, XE, XE2, XE3, XE4, XE5, XE6, XE7, XE8, 10.0 Seattle, 10.1 Berlin. Delphi 10.2.2 resulted in nearly equal results of 87 and 0 - it seems to switch nearly every time I build the project. Delphi 10.3.3 results in 87 every time that I've built/ran the project. (>20 builds) To narrow it down further, I have Delphi 10.2.0 installed on another VM with the same OS but different patch levels and it comes up with 0 every time. I copied the .exe to the first VM and it always comes up with 0. It does appear that in Delphi 10.2.1 (or 10.2.2) and later there is a change in this (useless) behavior. So - even though it doesn't really matter as we should all agree by now that you should never call GetLastError unless you have a very specific context to do so otherwise the return value is less than useful, it is a tiny bit curious why this is happening only in Delphi 10.2.2 and later on the same OS with the same console project. This is especially true since the code in System provided with Delphi 10.2 and 10.2.2 is seemingly identical when it comes to BeginThread. And the code in System.Classes for ThreadProc seems identical but I didn't spend a lot time trying to figure out the differences. Now it could absolutely be the case that the return value of 87, or 209232, or some other random number could easily occur on a Delphi 5 project and that I simply didn't execute it enough times to see a different result at the start of new thread. However, I did build and execute the same code multiple times per version and there were 16 different versions of Delphi tested which all resulted in 0 every single time. Perhaps it has to do with the size of the executable which seems to be ever-growing in subsequent releases of Delphi, who knows. (and yes, who really cares) It's of little value to anyone. But if you are 'that one guy' with code stretching back 20+years to Delphi 5 which was somehow based on the incorrect assumption that a new thread started with a GetLastError of 0, that assumption started breaking a few years ago around Delphi 10.2.2 for some reason. And since I wasted a little time myself chasing an invalid GetLastError recently based on someone else's code, it tripped my curiosity. program TestGetLastError; {$APPTYPE CONSOLE} uses SysUtils, Classes, Windows; type TMyThread = class(TThread) procedure Execute(); override; end; procedure TMyThread.Execute(); begin WriteLn('GetLastError: ' + IntToStr(GetLastError)); end; var x:TMyThread; begin x := TmyThread.Create(False); Sleep(100); WriteLn('Hit enter to exit'); ReadLn; end. Now that my curiosity has been sufficiently satisfied, I can be done with this. If someone else wants to go deeper, have at it. cheers

No, it's absolutely not a pointless process just because of the topic. If any value changes over time then being curious as to why is not only expected, but typically mandated. This is the case even if your perception is wrong headed, as it's still your perception of reality. The situation is even more difficult if its a very long held perception backed up by years of observation - even if that observation was faulty or insufficient. Changing perception is not easy - and it's rarely changed by blunt force. It's the developers' job to question facts that go against personal perception when analyzing problems. You don't need people hitting you over the head with their own (perceived or real) realities. There is a process that needs to happen to alter a perception. And it's a process that we all have gone through with different topics at different times in our careers. I would assert it's a rather common occurrence that we should all be familiar with - and since that's the case why can't we just provide assistance on the journey and not try to end it quickly because we deem it to be an invalid destination? Every trip is valid as long as there is something learned along the way. If you tried to be helpful and failed to penetrate someone's reality, then being quiet is sometimes the most help you can be so they can figure it out on their own. As such, I'll certainly try to ignore any further comments in this thread. Let's just please not turn this valuable resource into StackExchange. I strongly object to that sort of behavior.

If the perceived behavior over the last two decades was that a newly created thread always had a GetLastError of 0 and now it's all of sudden 87, there's absolutely nothing wrong with being curious. Now, I don't think it was the perceived behavior of many as it's pretty very likely that most devs haven't really given this specific topic much thought - but, programmers are supposed to be curious and altered behavior (perceived or real) is an obvious red flag that we all have been forcibly trained over the years to instinctively question, regardless of the merits. How many times have we gone a journey chasing a bug that "shouldn't ever happen" when it actually happens "every single time" once we figured out the "obvious"? (only obvious of course after the fact) Occasionally that journey takes a while and it's probably the case that every seasoned developer on the planet has taken that particular trip more than once - blunt force here isn't merited. For fun, you could scan through Windows source code and see what's up behind the scenes and maybe even find out where 87 comes from, if you are curious. https://github.com/pustladi/Windows-2000 https://github.com/Zer0Mem0ry/ntoskrnl

That's probably the best answer to #2. If you don't know the API methods being called right before you call GetLastError, then it doesn't matter what GetLastError is. The best answer to #1 is Yes, it's a known fact that GetLastError is only meaningful in specific situations. "You should call the GetLastError function immediately when a function's return value indicates that such a call will return useful data" https://docs.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-getlasterror

If you put a breakpoint on the first line of code in function ThreadProc in System.Classes and call GetLastError, it will be 87. This is the start of the thread code in Delphi so it's apparently simply starting out that way every time (at least in my limited testing.) As far "was this a known fact"... not known to me, not known to the few MS document searches that I did, but it does show up on a question from 2011 on StackOverflow: https://stackoverflow.com/questions/7199139/createthread-getlasterror-returns-87

Added a quick blog post on inline variables: https://www.ideasawakened.com/post/newly-discovered-hidden-benefits-of-inline-variables-in-delphi

Now that is definitely worthwhile... I didn't realize that benefit until I read your comment, then re-read it, then said "no way", then created some test code knowing that it wouldn't work, and then said "cool"

I received a pre-release sales offer today which names it 10.4 Sydney and provides discount codes for Delphi and C++ Builder so I'm hoping the 10.4 release date is coming "soon". https://github.com/ideasawakened/DelphiKB/wiki/D27.SYDNEY.10.4.0.0 Time to GetExcited! I always look forward to a new version of Delphi to play with!