Angus Robertson

I've just got it working with my live.com account, after accepting several application warning screens during the OAuth2 login process, I've previously had to increase the ICS timeout to waiting long enough for all this extra padding to be clicked past. Sorry, no idea about specific error, Microsoft does not any of this stuff easy for developers, at least not of my age. Angus

Sorry, the only documentation is at the top of the OverbyteIcsSslHttpOAuth.pas unit. I tested TIcsRestEmail against Gmail recently, but Microsoft was always much harder due to multiple APIs and account types. It was working when I initially wrote it, but my secrets are now expired and I need to refresh them to gain access again. Angus

ICS has a TIcsRestEmail component that supports the MS Graph interface, just basic functions to read and send email at the moment, demo in the OverbyteIcsHttpRestTst sample. Angus

ICS OAuth2 has been tested against smtp-mail.outlook.com and pop-mail.outlook.com, but they have several platforms, what changes did you need to make it work, so I can fix it for others. Sorry, ICS does not have IMAP4. Angus

I was going to suggest testing NTLM using the OverbyteIcsHttpRestTst sample which includes logging of all headers, but it seems this logging ignores the internal handshaking (and redirection) requests that the HTTP client component does, so will not help. Diagnosing will need some more logging added. But probably something we should do, NTLM is not going away in a hurry, despite modern OAuth2 , etc. Angus

There are no NTLM settings as such, all I can suggest is looking through the version notes in OverbyteIcsHttpProt.pas and OverbyteIcsNtlmMsgs.pas about changes over the years. You might also try tracing through the different messages passed during the handshaking to see where it dies. Are your two environments running the Windows Server version? Angus

Sorry, NTLM is primarily designed for use on corporate LANs in an Active Directory domain on Windows Servers, and I don't have Active Directory set-up. So our testing of NTLM is limited. It was implemented many years ago for Windows Server 2003 or something, and doubtless Microsoft has been changing it regularly since. I did test NTLM against the ICS web server recently, and you've found it sometimes works, but I've no idea why it fails in one environment. Angus

ICS V8.67 has packages for Delphi and C++ 11.0, currently only available from SVN and the overnight zip, with a final release due next week. http://wiki.overbyte.eu/wiki/index.php/ICS_Download There was a major new OpenSSL 3.0 release this week, just finishing testing and integration with V8.67. Currently, many older ICS icons appeared incorrectly with a magenta background in Delphi 11.0 due to transparency being ignored, these will be fixed for the final release. Angus

I was using ICS libraries built with 11.1 for a few days after installing 11.2 without any problems. It was only when I tried to build all packages with updated units that the Win64 issue appeared. Updated Delphi packages are now in SVN, sorry can not test C++. The issue may have been the Win64 linker. Angus

Just remove Winapi.Winsock from uses then, your application should not need it. Angus

We are aware of a problem building ICS packages for Win64 with Delphi 11 Update 2, aka 11.2. Something has changed in the way Delphi searches for DCPs. Spent the morning making changes to the ICS packages, just finished testing, will be in SVN shortly. Rewrote OverbyteHttpMulti earlier this week, will check your other error, that will be in SVN by tomorrow. Angus

Why do you have Winapi.Winsock in your uses? It is not needed in any ICS applications, and is only used in one component, probably unnecessarily. Angus

A lot of my applications have what I call triggers, there is a single timer firing every few seconds, and within that are Int64 triggers set for the required internal and tested against GetTickCount64 or GetTickCount. There is a small library of functions to set triggers for minutes or hours and check them. Angus

I'm afraid OpenSSL is the cross platform solution. Or one of it's forks. While there are native Delphi cryptography libraries, I've never seen a native library handling X509 certificates and PKCS-8 private keys.. Angus

No idea about that Indian company, but there are lots of similar companies offering SMS HTTP APIs, at varying prices and levels of reliability in delivery, I use https://thesmsworks.co.uk/developers (which is also supported by ICS). You'd have to adjust the parameters slightly. Angus

That code is creating a digital signature using a private key and Sha-256. Windows does have various old high level APIs for signing, using Capicom or Mssign32.dll, I use Capicom to check the digital signatures on EXE files. There must be low level APIs for signing as well, but I've never looked for them or used them, OpenSSL is so simple, albeit a large overhead since there is so much you don't need. YuOpenSSL links it's into your application to avoid DLL hell. Angus

Which ICS version were you using? As I said earlier, there is a limit to the number of new SSL/TLS connections per second due to socket server being a single thread. I've previously said in this forum there is a plan for a heavy socket server, which will be configurable for x clients per thread, two threads in theory would double the number of new connections per second, but it could be one thread per client. This will need a new web server as well. But I really need ICS end users with server applications involving thousands of clients to justify the effort developing it. Would that be of interest? Angus

It is important to handle the onBgException exception event, and Application.OnException. Recent ICS releases have improved BgException messages to help track the origin. I log the errors, send an admin email and then stop the service, so Windows can immediately restart it. Unfortunately, some SSL problems do not trigger any events and just crash the program, but it does restart. Angus

I was having a problem with one of my servers not listening under some circumstances, so implemented an internal watch dog that made a connection attempt every minute, very simple using TSslHttpRest;, just a few lines of code triggered from the maintenance timer. Worked well, but showed the server was still listening. The problem turned out to be two services using the same named firewall rule updated each time they started to the current EXE name, so dependent upon the order in which the servers started. Angus

My previous SSL/TLS testing shows there is a limit to the number of new connections per second, due to time taken to setup SSL/TLS and all the handshaking involved. There is a server setting ListenBackLog which defaults to 15, that means Windows will only queue 15 new connections before rejecting any further attempts. You could also try increasing your timeout beyond 60 seconds, to reduce the number of new connections. An established connection is low overhead, new one high overhead, provided you have sufficient handles and resources to open the total number. TLS/1.3 has faster setup than TLS/1.2, if you can use it. You may also be able to optimize certificate chain checking, the less the faster. Angus

Thanks, my issue does seem to be OpenSSL DLL related, although the same ResetSslSession function may have been called dozens of times during the FTP session, twice for each file uploaded or downloaded, or not once if the session fails due to authentication. I should really try with older DLLs and 3.0 built elsewhere. Although none of this explains why the same program built with Delphi 2007 has worked fine on all my servers for 15 years, with various OpenSSL releases, maybe that is PE header related. Angus

Logging works before SSL_free is called, but nothing afterwards. This function is called for every SSL connection in ICS, there is no way anything is getting corrupted on a platform basis. During a normal close down, the ResetSslSession function gets called a second time with all the various pointers nulled, and behaves itself. Setting up remote debugging will be tedious, and I doubt would be productive. It's only the FTP server that seems unhappy, the D10/D11 web server has been running for over a year. Angus

I added syslog logging to the sample so now have logs up to the point of crashing, which is in our function ResetSslSession while closing various handles. The crash happens after a call to SSL_free(FSsl), but the exception handler does not catch it. But this still only happens for a remote connection to the server, not locally. The remote connection does raise an earlier abort network error, but this is ignored. So ultimately the problem does not seem to be anything to do with our Delphi code, but with the runtimes in the OpenSSL DLL. Angus

Thanks, even if I managed to create a process dump at the moment of corruption, I really would not know how to go looking for the line that caused the problem. Or why that line only fails in certain but 100% reliably repeatable circumstances, which do not include under the Delphi debugger. Angus

I also have units with underscores, but this was a form so perhaps different rules apply, did not think much of it at the time since I was skipping past the hundreds of modal dialog box errors Delphi gives when opening a project with components that are not currently installed, why no 'ignore all' button and why no logging of all the errors? It's so frustrating when you migrate projects or update Delphi. Angus