Kas Ob.

Forgot the moral of all that The best way for high performance is to not push the threads (job or whatever), let them pull/poll on their own time. By centralizing the email handling with watcher in intervals, you will be removing huge bottleneck in case there is many emails, while the dispatchers will be almost a sleep sending and using SMTP because it is slow, so yes you can have 64 dispatcher at the same time and will not notice the CPU usage goes above %1, while the watcher can read hundreds in one go and also this is very short and fast process.

Hi, Of course it is possible. You asked about threads and Remy gave a perfect answer to your question. Now let put my thoughts here: 1) Now you are thinking right and want to do the right thing, i mean eliminate the failing point or at least minimize and if possibly implement a recovery. 2) In my opinion best threads are threads that doesn't talk to each other. 3) You have Sqlite as storage and can serve as central coordinator or a hub while serving its propose to store data/task. What i suggest is to redesign you process a little differently, 1) Lets say you generated an email ( being to recover/restore/reset password or an invoice ...), this generated in a thread (any thread), then it will be pushed into a table in Sqlite, here comes the table itself with few extra columns, dtGenerated : will holds datatime of generation. dtSent : will hold the time it being picked to send ! intSend : the count of send tries. dtDelivered : hold the time when the email being delivered, and here when the SMTP finished receiving the email, we will consider this a success, enStatus : hold the status, Error, Failed, Pending , Delivered.... not so crucial but nice to have You can also put a an extra field to confirm the result of the sending from the SMTP, but this is an extra. Now one thread generate an email, and push it to the DB, there is another (one thread) its job to perform a query for emails in the DB with no dtSent value !, or with pending or ... the way you comfortable with. After grabbing these emails it will dispatch them to another one or more threads to perform the real SMTP sending, mean while and before dispatching them to these workers, that watcher threads will insert an ID of that email into special table called (lets say tbWorking) where that ID is key hence it is impossible to insert it twice, in Sqlite you can replace ID with simple ROWID from the email table, this is unique value, (also notice all DB servers has something similar like in Sqlite RowID is an Integer but in MYSQL it is hex or short string and there is ROWNUM i think..) If an email being sent successfully or not, then and only then the email table can be updated accordingly, and a delete should be issued to tbWorking, this tbWorking will server as lock free list that persist in case of catastrophic failure, like app crash or power cut, Threads will have a small and focused task to do, not generating emails, then store, them then, send them, then store the result... that is way too much work and it is also slow and will take time maximizing the risk of losing an email due to some other failure or Internet lose .... Anyway, what i wrote above is not a full solution that will fit you, most likely you will need to address the main point, what could happen and how to recover from each and every step and moment ? You need to think out side the box of generate-store-send-update and linear processing. I had done a project that similar to what you you need, but it was using MySQL and worked without a problem, With MySQL ?, the emails can be generated from completely different server running php and pushed to the MySQL, my server no matter if itself generated the email or from the web server remotely being added could and can send them all. Also don't forget to store logs for these operation, the logging the faster and more secure your process will be, i used two logging method with that, one on files and the other was in table in the DB, recoding the failures and successes. ps the watcher thread has one job to check for newly added or the emails not in tbWorking, it has low interval like 5-10 second, and will assume nothing on it first run, so it will/must update tbWorking on start, because filled value in that table on start means there is a conflict in need to be address, and the best way is to resend unless these are times email like resetting password then drop them. Hope that helps and give you insight.

@Vincent Parrett That is really fucking neat ! the more i look the more brilliant details a see ! It is a master piece done by a master. joking part : (hoping it is funny!) are you telling me this DPM is written in Delphi by Delphi developer in less than a decade ! @Sherlock there is another way 😎 https://medium.com/@peternjuguna76/hosting-a-json-file-on-github-pages-a-step-by-step-guide-52105a5a393a

Thank you, that is awesome !

I can't find many resources about USER_MAPPED_FILE, in fact there is near nothing on the internet , found this https://stackoverflow.com/questions/41844842/when-error-1224-error-user-mapped-file-occurs though not so helpful. Anyway, to decode this situation using my own logic, the file is already being opened for mapping by another process, hence the BDS call to CreateFile is failing, and there is a few ways to catch the culprit redhanded. we start with the easiest way to find the culprit : 1) grab Handle from https://learn.microsoft.com/en-us/sysinternals/downloads/handle 2) put it some write permitted folder ( don't use any root directory like C: \ or D:\ ...) 3) run cmd with Administrator rights (elevated) on that folder 4) run this command : handle64 > hlist.txt 5) open that hlist.txt file and search for your EXE name Now if we fortunate and things went as we wish for, then we expect to see is something like this, here the file i locked with mapping is 1.txt Share with us to further discussion if discussion is needed, but in all cases we appreciate your finding ! Newer and updated guessing part: 1) system protection for some reason has kicked in . 2) that folder or driver is shared and/or locked by some GoogleDrive (may be) or something similar, and remotely there is software is parsing or handling that file, it easily can be an AV on remote device ! 3) Again that Webroot on your device might be is doing some shenanigans. Don't forget to run handle.exe or handle64.exe with Administrator !

Can't agree more on these, yet ... GetIt could simply be pulling a well structured and well thought JSON file from specific GitHub, this is Embarcadero managed GitHub repository, it will hold a list of sources from GitHub or any where else, with a mark "Verified or Emb.Ready" or "Not Verified ! use on your own" So in GetIt you will have the Official sources and non-official ones, also it could list stuff like development or beta or debug.... or many many things, simplifying the whole process, anyone can upload or pull a request to a repository and GitIt Maintainer will list them under categories ... many things can be suggested here, the point is user of GitIt will have way more richer experience, resources, EULA ..., with minimal interaction from Emabrcadero staff or timing, also such GitIt can have its own or user supplied repository like the official one, again, here one can put his own code. Is that much or impossible to implement ?, i guess current GetIt took way more time to evolve to be stable (i never used it), yet to fail like that today, after all why it is not opensource itself ? any particular reason ?!

I am sorry ! I thought putting such interfaces resolver in the exception handler for failed "as" casting could show log better information about the missed interface implementation hence the failed casting.

May i suggest this alternated code from Remy (i hope with his permission) https://stackoverflow.com/questions/39584234/how-to-obtain-rtti-from-an-interface-reference-in-delphi type ITest1 = interface ['{5AB029F5-31B0-4054-A70D-75BF8278716E}'] procedure Test1; end; ITest2 = interface ['{AAC18D39-465B-4706-9DC8-7B1FBCC05B2B}'] procedure Test1; end; TTest = class(TInterfacedObject, ITest1, ITest2) public procedure Test1; procedure Test2; end; procedure TTest.Test1; begin //... end; procedure TTest.Test2; begin //... end; function GetInterfacesFromItsObj(AIntf: IInterface): string; var obj: TObject; IntfType: TRttiInterfaceType; ctx: TRttiContext; tmpIntf: IInterface; begin obj := AIntf as TObject; Result := 'Object: (' + obj.ClassName + ')_ -> : '; for IntfType in (ctx.GetType(obj.ClassType) as TRttiInstanceType).GetImplementedInterfaces do begin if obj.GetInterface(IntfType.GUID, tmpIntf) then begin Result := Result {+ 'Intf: '} + IntfType.Name + ', '; tmpIntf := nil; end; end; end; procedure TForm10.FormCreate(Sender: TObject); var Intf1: ITest1; Intf2: ITest2; begin Intf1 := TTest.Create as ITest1; Intf2 := TTest.Create as ITest2; ShowMessage(GetInterfacesFromItsObj(Intf1)); end; The result

If i to read this right, then the exception was not here at all, and this one is a nested one, in other words this one is raised while handing a previous (and original) exception. Also the stack is cut !, where is the rest of it ? does this dialog have more text without scroll bars ? or this is it ? Also would you please try to track the actual disk operations with Process Monitor ? in other words, were there a successful/failure File Write operations right before this dialog ?

Hi, Yes, it is half of the problem in your case, here you didn't mention who is the owner of the EXE and the owner of the parent folder, both these values plays role, The other half is either 1) System Protection has kicked in and sees that file (folder) as valuable and important, in this case there is copies and copies in the hidden/strict restore folder (System Volume Information) on the drive, you need to disable the protection and clear that folder then enable if you want it. 2) Windows Defender has kicked in and most likely sent the exe sample to Microsoft labs, i never ran this cloud protection (my main developing machine is still Windows 10), so don't know exactly what it does or how it behave, https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/cloud-protection-microsoft-antivirus-sample-submission?view=o365-worldwide Also you can run Process Monitor https://learn.microsoft.com/en-us/sysinternals/downloads/procmon remove/exclude all other processes and leave only BDS.exe in this case (to catch dcc32 operation), from there you should have the real and internal failure on API level this example of that same compiler error when the target exe is running and when was not

Internet ?! Don't be scared - I always try to put some humor into my writing.

Thank you! Just one think to explain and make it clear This is not exactly about laziness, but also about how smooth it goes in slow brain like mine: TNetFormat instantly invokes Network, Internet, protocol standard... FStr instantly invokes a local field in Self or even local variable... Anyway, thank you again and it is of course up to you to name it, for me i would not use it like that with these names, i will rename a type from it, something with clear path/history in my brain then use it, something shiny and unique 🤓

Why not VoStr or VoFmt ? Short and easy to remember.

https://www.yunqa.de/delphi/products/openssl/index remove the need to OpenSSL altogether if the need is only for secure connection https://www.devart.com/sbridge/ great price, full source, decent performance and extensive documentation with DevArt support engine, from https://www.devart.com/sbridge/editions.html also with full SSH support ! https://www.streamsec.com/index.php?id=orderstrsecii v2.3 is old but still working last time i used it https://www.streamsec.com/index.php?id=orderstrseciv v4 is more expensive, never used it

The line number is there, in the stack call. What needed is try.except to minimize the overhead in such places where errors/exceptions reported, while in the according exception handler a small RTTI piece of code to extract and report the offending class, the one that most likely shouldn't be there or simply lost one Interface from its definition because it is so rare to that had being missing such declaration, getting its type info should be trivial, (i think)

My thoughts here : 1) Evidently from the stack, the exception raised by using the "as" operator. 2) This could be captured or even prevented but in all cases it can be logged and reported in better detailed report (or error message) than leaving it to the IDE to handle the exception.

FreePascal capable of producing WASM files. https://wiki.freepascal.org/WebAssembly https://www.freepascal.org/~michael/pas2js-demos/wasienv/canvas/ https://www.freepascal.org/~michael/pas2js-demos/wasienv/canvas/canvasdraw.lpr

I have very simple question, why your Delphi built application asking for Microsoft CRT library ? CRT= "C RunTime" not "Common RunTime" ! I see there is Chilkat library in the folder, so either 1) Chilkat is the offensive one, in this case you contact them for a fix or replacement, all CRT can be built and linked statically removing the need for the external libraries. 2) Something else does need it, in this case you included/linked some code/library that need CRT, again find it and replace it may be.

I had MyDac since 2009 and it is documented and shown in the demos, same goes for UniDac. q.SQL.Add('SELECT * FROM users WHERE id IN :param'); ..... q.ParamByName('param').AsString := 'Iam doing here what ever i want, including a useless try to inject SQL with select * from user or even delete * from users; that will not be ran as SQL !!!'; As shown in my screenshot from that link.

I need to explain the difference between user mode part of the kernel and the real OS kernel, the hidden and protected one, lets take a simple OS API like GetTickCount, put it in Delphi code, stop the debugger on it and use step into, continue until return to Delphi code, the debugger will be able to walk it all, but if you take one of the sensitive OS API let say any of the file handling (or threads...) and tried to trace with step into using the Delphi debugger or any debugger then you will end up with such assembly instruction SYSCALL oe SYSENTER https://www.felixcloutier.com/x86/syscall https://www.felixcloutier.com/x86/sysenter These are the gates for protected mode, what happens on the other side nothing on our (user) side can see or know and predict. Now back to the exception raising and handling (trapping), all exception handlers are chained but will will not cross that protected mode, so no matter where the exception raised it will follow that chain, BUT will not cross that barrier. So if an exception been raised in user mode part of the kernel or in you application or any auxiliary library (DLL) those can be caught or at least can be inspected or hooked (override the SEH traps) by software like EurekaLog, but if the exception being raised in the protected mode then nothing will handle it from here and the only way to handle, inspect or trap it is by software running on that side, like kernel debugger or OS kernel its own exception handling code. Some API does indeed raise an exception on its own, by code (aka software) or hardware exception (eg within a driver), here come how it did implemented to do it or after it, either by mitigating the exception silently (eg. access violation to unallocated memory) then will eat that exception and return to the user mode with simple error code as API return value (saying something like the buffer invalid), but in user mode here no one will know, in other words neither code (exception handling) or debugger will be triggered, or in some very specific situation the exception is that rare, that Microsoft didn't take the time or the effort to mitigate it or it is indeed have some risk factor (security), and let it run through the kernel exception handler (OS layer debugger) calling the WER (which indeed also a debugger), this one can't be handled and will lead to terminating the process and no control will be brought back to user mode, i mean none, all the threads will be terminated after WER paused them and will not be allowed to exist after that, if WER is invoked, it could be risk factor and even WER will be called. Back to EurekaLog, it does good job with hooking on system level, but again all this will only apply to user mode part of the kernel, beyond that no software (in user mode) will help, but what will help is to read WER report and get sense out of it, it should have a call stack. It is definitely external, but we can't be sure where, if in user mode part of the kernel then EurekaLog should/might capture it, but if it is in protected mode then no. One thing for sure it is as per documentation STATUS_HEAP_CORRUPTION , which is merely something simple like use after free or an overflow in heap or something similar....., But again that an OS API tried to do read/write on what it did classify as corrupted heap, and OS didn't allocate that part but your software did (including OpenSSL library), and it is the one responsible for this wrong memory/pointer fed to the API, OR in a THOERY the API were fed a right pointer/memory/heap .... but another thread wrongfully accessed it and modified something or even freed it !, that what should be checked against. PS : if you supplied a blocking read with a buffer to fill (file or socket ...whatever) and while it is blocking on that operation you did free that buffer, what do you think should be raised as an exception ? yes it will be STATUS_HEAP_CORRUPTION, this might not be easy to reproduce with Delphi MM (FastMM) but will be very easy with VirtualAlloc/VirtualFree, with FastMM there will be no error because freeing memory is rarely a free and return to the system. Hope that was readable and helpful.

Then i will just suggest putting recovery handling code, and leave it there, if or unless the crash is due to security measure form the OS then you might have a chance to retrieve these disk unflushed logs files. like RegisterApplicationRecoveryCallback from https://learn.microsoft.com/en-us/windows/win32/api/_recovery/ It worth trying.

I have few things about the subject 1) The exception is raised in the OS kernel protected part, not the user part, hence the the failure for the application (or MadExcept..) to get hand on the exception. 2) When you say it doesn't happen with YuOpenSSL build, this bring to elephant in the room, YuOpenSSL build will be using/utilizing Delphi application memory manager instead of the System heap and memory manager, and there is a big difference, while FastMM is build for speed and other feature, it is very forgiving in the matter of out-bound write/access, because small or medium allocations are continuous and there is no check for overflow write or even as simple reading after freeing. 3) No sure what MadExcept does offer, but sure that EurekaLog have extended functionality to perform deep hooking and capture and report every handled exception 4) Also EurekaLog can helpful in performing some fuzzing to enhance the chance of capturing the read after free usage I think you can capture and eliminate this exception or at least get better understanding of its context, by going after YuOpenSSL build not the OpenSSL Dlls, focusing on that build in my opinion has the best chance to solve this once and for all, even when there is no exception being raised or any symptoms or memory miss usage.

Refer to the documentation https://docs.devart.com/mydac/modify_data.htm

I did the same thing with in few servers, and as Remy and Angus said it should handled on lower level from the event layer, here a snippet i have in a project if (BytesRecv > 0) then begin if (FInSSLBuffer[0] = $16){ and (FInSSLBuffer[1] = $03)} then begin FConnecitonIsTLS := True; TLSInit; // end else begin // Move(FInSSLBuffer[0], FInBuffer[FInBufferCount], FInSSLBufferCount); BytesToProcess := FInSSLBufferCount; FInSSLBufferCount := 0; end; end; Though i am not experienced in ICS, i would suggest to derive (inherit) your own server and make you detection, before triggering the event thus you will have your dual usage server, and drop in place, On side note, i needed this for some reasons: 1) To remove huge amount of logging from online scripts and bots that keeps requesting plain HTTP on port 443, and i replied with and empty HTTP, when it is was HTTP. 2) I needed to add SSH and HTTP (as above), but i didn't want to go with multiple ports, so used 443 for SSH too, the same way it detected few bytes, as above it seems i used only one byte for SSL/TLS. An argument for the need to have such functionality: I needed to switch off SSL/TLS traffic and leave the plain traffic in extreme debugging/testing and watching unstable networks (WiFi) traffic on WireShark, also needed to stress the socket engine itself not the Secure layer and its CPU demanding, so for such case the ICS server could have a Boolean property to disable SSL/TLS, with default as False, and when it is True the traffic will skip all Secure layer handling, such feature will have its consequences and can be be not allowed to disable after enabling, but it is the responsibility of the developer (ICS user) to understand how and when to flip this property. Not big or needed feature, but a feature non the less.

I am sorry and deleting this distasteful joke, really sorry.