Angus Robertson

Thanks, my issue does seem to be OpenSSL DLL related, although the same ResetSslSession function may have been called dozens of times during the FTP session, twice for each file uploaded or downloaded, or not once if the session fails due to authentication. I should really try with older DLLs and 3.0 built elsewhere. Although none of this explains why the same program built with Delphi 2007 has worked fine on all my servers for 15 years, with various OpenSSL releases, maybe that is PE header related. Angus

Logging works before SSL_free is called, but nothing afterwards. This function is called for every SSL connection in ICS, there is no way anything is getting corrupted on a platform basis. During a normal close down, the ResetSslSession function gets called a second time with all the various pointers nulled, and behaves itself. Setting up remote debugging will be tedious, and I doubt would be productive. It's only the FTP server that seems unhappy, the D10/D11 web server has been running for over a year. Angus

I added syslog logging to the sample so now have logs up to the point of crashing, which is in our function ResetSslSession while closing various handles. The crash happens after a call to SSL_free(FSsl), but the exception handler does not catch it. But this still only happens for a remote connection to the server, not locally. The remote connection does raise an earlier abort network error, but this is ignored. So ultimately the problem does not seem to be anything to do with our Delphi code, but with the runtimes in the OpenSSL DLL. Angus

Thanks, even if I managed to create a process dump at the moment of corruption, I really would not know how to go looking for the line that caused the problem. Or why that line only fails in certain but 100% reliably repeatable circumstances, which do not include under the Delphi debugger. Angus

I also have units with underscores, but this was a form so perhaps different rules apply, did not think much of it at the time since I was skipping past the hundreds of modal dialog box errors Delphi gives when opening a project with components that are not currently installed, why no 'ignore all' button and why no logging of all the errors? It's so frustrating when you migrate projects or update Delphi. Angus

Recent versions of Delphi seem to dislike units with underscores, I'm currently converting some Delphi 2007 projects to D11 and renamed two units so far to remove underscores. Angus

So what did you see in your browser when you viewed https://www.whatismyip.com/ ? Exactly what you are asking for? That particular site might just disappear, so long term it would be better to use a STUN client widely used for VoIP. Angus

Sorry, this was incorrect, the ICS SMTP component does automatically inline MIME encode any 8-bit characters, if you have SmtpCli.Allow8bitChars and SmtpCli.ConvertToCharset both false, and set SmtpCli.Charset to 'utf-8', which is not the default. Angus

ICS V8.69 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0 and C++ Builder 2006 to XE3, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. The distribution zip includes the latest OpenSSL 3.0.1 win32, with other versions of OpenSSL being available from the download page. Major Changes in ICS V8.69 include: 1 - V8.69 has a lot of minor improvements, more HTTP client and server features, better built-in authentication, and added built in file uploading to the REST client component. Also continuing improving all the HTTP samples so they now hopefully test all the features of the ICS HTTP server, specifically adding POST and PUT file uploading, and a new client login window for interactive authentication with servers. Also added OCSP (Online Certificate Status Protocol) to ICS, which is used to check SSL/TLS certificates are legitimately issued and not revoked. 2 - In the HTTP client, added new methods RequestAsync and RequestSync which start a specified request by parameter, useful to repeat the last request from an event (such as should be trust the certificate and repeat). When using a proxy, make sure Path is not blank which will break proxies, change to / as for non-proxy requests. GetRequestDoneErrorStr now handles TWsocket errors. Fixed NTLM authentication which got broken in V8.61. Added a new THttpAuthType of httpAuthDigest2 for Digest with the modern SHA-256 algorithm instead of MD5 used by httpAuthDigest, only with USE_SSL. Added new property WWWAuthInfos array filled after 401/407 failure by parsing AuthorizationRequest headers for AuthType and Realm that may be presented to select an authentication option, get login and repeat request. Remove # fragment anchor rom the URL unless the new Option httpoAllowAnchor is set. 3 - In the HTTP REST client, added built in file uploading using POST or PUT. The file name is specified in HttpUploadFile using type HttpUploadStrat ofHttpUploadSimple with parameters in the URL or HttpUploadMIME for multipart with parameters in the first MIME part. Note the applications needs to supply parameters like FileName so the server knows what to do with the file. This may be tested against the ICS web server samples. Upon request completion, ReasonPhrase now also has status, so OK becomes 200 OK, etc. Previously the SslRevocation property was only effective when checking the windows certificate store, now it also works with bundle files using the new TOcspHttp component and OCSP stapling if available. 4 - In the HTTP server component, now converting FLastModified to UTC/GMT time for the response header. Added new authentication type atDigestSha2 with a SHA256 hash instead of MD5 with atDigest only supported with USE_SSL. Note: Mozilla Firefox supports Digest SHA-256, Chrome and Edge do not. Added a new client connection OnHttpRespHdr event to allow response headers to be logged, previously only request headers could be logged. Cleaned up AnswerStream functions so separate lines don't go into send buffer. Added OcspSrvStapling property which should be set to enables OCSP checks and stapling only with AUTO_X509_CERTS define since it adds extra HTTP client code. A revoked certificate will be auto ordered. 5 - In TWSocket, added new method SendTB(const Data: TBytes; Len: Integer=-1) where Len is optional, also similar SendToTB and SendToTB6 with TBytes, as an alternative to casting such types to use Send(). Restored the TlsExtension_cb callback for client debugging since ClientHelloCallback only works with servers. 6 - In TSslWSocketServer added OCSP (Online Certificate Status Protocol) support with IcsHosts using the TOcspHttp component to confirm server SSL/TLS certificates are legitimate and not revoked for security reasons. The certificate OCSP response is also stapled to the initial SSL/TLS HELO handshake and sent to the client to avoid it needing to lookup OCSP using HTTP itself. OCSP responses are cached and saved to a file for reloading later, but are refreshed every time the certificate is validated, at least once a day. The new server property OcspSrvStapling enables OCSP checks and stapling only with AUTO_X509_CERTS define since it adds extra HTTP client code. A revoked certificate will be auto ordered. OCSP checking is done in LoadOneCert and the stapled response sent in TriggerSslServerName when checking SNI. The same OCSP support is available in all ICS servers that use IcsHosts, including FTP, HTTP and TIcsIpStrmLog. 7 - All three main HTTP SSL client sample applications FrameBrowserIcs, OverbyteIcsHttpsTst and OverbyteIcsHttpRestTst now support interactive authentication with a new Login window that displays the different methods the server will accept (from the new property WWWAuthInfos array) allowing one to be selected from Basic, Digest MD5, Digest SHA256 and NTLM logins. This window is displayed after an 401 error and the request then repeated. 8 - The OverbyteIcsHttpsTst SSL sample now has all the missing features from the non-SSL samples, Content Encoding Gzip tick box to support compression, persistent cookie support, POST/PUT support to either send simple data or upload files in various ways (from OverbyteIcsHttpPost1 sample), may be tested against the ICS web server samples. 9 - The OverbyteIcsDDWebService SSL web server sample now builds on unicode compilers. Added authentication for POST requests and new 'Password protected page (POST)' button on the demo menu to test authentication using POST. Fixed web logging to log correct multiple listener. Builds on unicode compilers. Added Digest SHA-256 authentication page DemoDigest2Auth.html and DemoDigestsAll.html that does both digests. DemoAuthAll.html no longer does NTLM, use the separate page. Now displays server response headers if box ticked, only displayed request headers before. File Upload Form and Email Form pages now work without exceptions. The OverbyteIcsSslMultiWebServ SSL web sample has similar new authentication features and logging fixes. 10 - Updated OpenSSL to 3.0.3 and ZLIB (HTTP compression) to 1.2.12, sorry for long delay in updating zlib, now including some important bug fixes, although never saw any issues with the minimal use ICS makes of it. Updated the various ICS CA Trusted Stores. 11 - The OverbyteIcsPemtool SSL sample now does OCSP checks when examining certificate files. There is also a new Test Host Certificates tab that tests SSL/TLS handshake and certificates (using TIcsIpStrmLog) for a list of host names and ports, building a list of host and intermediate certificates that may be further checked and saved as files or added to an intermediate bundle. This new tester should prove useful for debugging sites that return SSL or certificate errors. More detailed release notes are at http://wiki.overbyte.eu/wiki/index.php/ICS_V8.69

We build IcsCommonD110Run almost daily during development of ICS. Many others have installed it without any errors reported. This would suggest you either have a corrupted version of ICS, unlikely since GetIt also failed, or something is strange about your installation. Angus

The content of the ICS packages has not changed in over 10 years or longer, since Delphi XE2 except for new units being added. If you want only VCL you install only the two OverbyteIcsDxx packages, if you want FMX as well you only install the three IcsCommonDxx packages, as explained in the readme8 file. Using the install groups is much easier, DxxInstall or DxxInstallVclFmx. There is a Build Group in modern compilers, so a couple of clicks builds all the packages for Win32 and Win64. GetIt is the same, VCL only or VCL/FMX. Angus

Did Sectigo also ask you for a selfie with your passport? On a general note, worth mentioning that buying Code Signing certificates will become more expensive and difficult from this autumn when software private keys are banned, you'll need to buy your certificate on a physical dongle probably adding $100 or more to the price, plus shipping and customs hassle for those outside the country where the certificate is sold. This was the reason I bought a three year certificate earlier this year. Angus

Never used Windows Mailslots, don't see any advantage over using sockets, they are limited to messages no more than 424 bytes, and I'd guess are dependent on various network services running. Microsoft recommends named pipes or sockets as the modern replacement. Angus

K-Software support is almost non-existent, totally automated, probably how they are able to offer code signing certificates vastly cheaper than anyone else. I bought a three year certificate in January, it all worked, they resell Sectigo, you should get an email from certs@ksoftware.net with a link that starts the process, which is fun since it requires Internet Explorer that has now been discontinued. Eventually Sectigo will contact you to confirm your personal identity using your passport and the certificate will be issued, that is why I paid $188 for three years ($500 from Sectigo) to put off doing it all again. Good luck. Angus

I have the same opinion of Github, it's fine for complex projects with lots of contributors, but far more complicated than SVN for projects with a small number of trusted contributors. Github may be safer, but at the expense of extra time for project owner merging contributions. I only contributed to a Github project once, it was so tedious I now just email the project owner who is welcome to ignore my contributions. That is why ICS still uses SVN, which I host on my own public servers (mostly runs itself). But I'm also retired and not willing to spend much time learning new technology. Angus

The log shows a single un-authenticated request resulting in a 401 response, no attempt to send authentication or repeat the request. The sample would have brought up a dialog box asking for a password to be entered and repeated the request. Sorry, but there is no point in posting logs that don't illustrate any real activity or problems. BTW, beware of posting real authentication details in an open conference. Angus

The IcsLogger is designed for low level SSL development, not application protocol debugging. Try reproducing the problem using the OverbyteIcsHttpRestTst sample with debug logging set to HTTP headers and set a log directory, then send the complete log, not selected extracts, indicating where it fails. If you can not reproduce the bug in that sample in V8.69, the problem is your code. Angus

If I can not reproduce a problem, I can not fix it. You will need to reproduce using a public server. Or against the ICS web server sample which supports all authentication methods. Angus

If you want me to investigate a possible authentication issue, I'll need a failing example to test, my private email is in the ICS source code files, specifically a URL and login that works and fails, not interested in any source code. Angus

You should build the OverbyteIcsIpStmLogTst.dpr sample, and run it on two or more PCs. It uses the TIcsIpStrmLog component, which can be configured as a TCP or UDP, client or server, and sends lines of text between instances, with just a few lines of code for client or server. As François says, your main PC can broadcast UDP packets periodically to the clients running a UDP server, they then return their IP address, which you log. The client PCs also run a TCP Server, which the main PC then sends messages to, TIcsIpStrmLog can send to a list of IPs. Or your main PC can be a TCP server, and the client PCs keep a permanent connection open for messages, avoids needing to keep IP addresses. If the TCP client connection from TIcsIpStrmLog drops, there are settings to make it repeatedly retry connections until the server comes back. All this can be tested without any coding using the OverbyteIcsIpStmLogTst.dpr sample. If you decide the clients should be servers and you need their addresses, you can instead use the Address Resolution Protocol (ARP) that runs on all network devices to keep track of IP addresses and MAC addresses on the local LAN, it is how ethernet works. I have an Internet Protocol Helper Component that gets the ARP table. In the next few weeks, this will get added to ICS with a new sample to list the ARP and IP routing table, current TCP and UDP connections and network adaptors. Angus

The ICS mail components do not automatically MIME encode unicode text for email headers. If you look at the OverbyteIcsMimeDemo.dpr sample in the MailNewsDemos sample folder, there is a button Encode One Header which will inline MIME encode the Text you specify using various methods, such as ?utf-8?B? or ?utf-8?Q?. The actual inline encoding is done by HdrEncodeInLineEx in OverbyteIcsMimeUtils, and you set the output to the mail header property. Angus

The Web Socket Server was written by Stan Korotky some time ago, and adds a protocol on top of the normal TWSocketServer component, so you can not use client methods. But the component only sends AnsiStrings, so just copy your binary data into an AnsiString and send that. Angus

You didn't give any information on the device you are collecting files from, except the file was an image. If this is a CCTV camera, the OverbyteIcsXferTst sample and the TIcsFtpMulti and TIcsHttpMulti components it uses may be used to download images and movies from many CCTV cameras automatically, with a few lines of code. The sample is over complex, due to it supporting lots of different features and components. I have SV3C cameras, which allow downloads using a web server, TIcsHttpMulti indexes the web pages and downloads new images and movies every two hours, about 20GB a day, the cameras also upload motion capture images automatically to the ICS FTP server. TIcsFtpMulti will do the same thing for cameras with an FTP server. Angus

If ICS allowed to you send an empty password, you'd still get an error from the server. Your code is specifically sending the password command with FFTP.PassAsync;, remove that, and you won't get an error. You are using the low level FTP component, the example you should be looking at is OverbyteIcsXferTst.dpr which uses the high level FTP component that hides much of this complexity from you. Angus

That log says your download worked fine, so the authentication was ignored. Angus