Angus Robertson

ICS V9.1 is almost ready for release. Although there are no new components, there are many other SSL/TLS changes that will affect existing applications, but make ICS easier to use and support for the future. Before the final release in a week or two, I'd appreciate some feedback from user installing V9.1 using the new packages, and update one or more old SSL/TLS applications, it may help future users if I can improve the documentation. Please read readme9.txt and these note about V9.1 carefully when upgrading existing applications, you may get build errors that need minor code changes. But new applications should need be easier to create. 1 - Delphi 10.4 and later now use the same install groups and packages, IcsInstallFmx, IcsInstallVcl and IcsInstallVclFmx, making support a lot easier. Version specific groups remain for Delphi 10.3 and earlier, with new groups D(X)InstallVcl for VCL only replacing the old OverbyteIcs(X) groups, again to simplify support. 2 - The old samples directory has gone and many of the older and little used samples have been archived to a separate download. The active samples used to test and demonstrate all ICS components are now split into the following paths, in the ICS root directory: demos-delphi-vcl - 45 VCL samples for Windows. demos-delphi-extra - four VCL samples that need third party components to build. demos-delphi-fmx - seven FMX samples for Windows, not yet tested on MacOS. demos-cpp-vcl - all old C++ samples that have not been tested for 10 years, need help. demos-data - data files for samples, such as web pages. All these samples can now be built for Win32 and Win64 platforms. 3 - To ease development, linking and future support, some new units have been added by splitting existing units with multiple components, unfortunately this means many existing projects will need one or more of the new units adding to their uses section. Apologies for the pain, but this should have been done a long time ago. The main change is splitting out much of the SSL/TLS related code from the massive OverbyteIcsWSocket unit to a new unit OverbyteIcsSslBase. 4 - Distribution of the ICS OpenSSL files has changed. Earlier ICS versions required the OpenSSL DLLs to be distributed with applications, and a root CA bundle file to verify SSL/TLS connections, and these needed to be loaded using code. There was little standardisation over where the OpenSSL DLLs were located, applications tended to keep their own copies alongside other executables, leading to multiple DLL copies and needing the public variable GSSL_DLL_DIR set to a specific directory before OpenSSL was loaded. Likewise, root CA bundle directories had to be distributed with applications and loaded with code. ICS V9.1 allows five different ways of loading OpenSSL: 1 - DLLs linked into application as resource files 2 - DLLs loaded from common directory C:\ProgramData\ICS-OpenSSL\ 3 - OpenSSL DCU linked into application using commercial YuOpenSSL 4 - DLLs loaded from location specified in public variable GSSL_DLL_DIR 5 - DLLs loaded according to path, may be found anywhere on PC Which method ICS uses to load OpenSSL depends upon several defines in the .\Source\Include\OverbyteIcsDefs.inc file, please see the readme9.txt file for details. ICS currently includes resource files for three different OpenSSL releases, 3.0`13. 3.1.5 and 3.2.1, which version is linked is controlled by a define. If the OpenSSL DLLs are linked into the application, they are extracted to a version subdirectory, ie C:\ProgramData\ICS-OpenSSL\3012\ so different applications can use different OpenSSL versions. This happens only once if the files have not already been extracted. When updating existing projects without using any new defines, the ICS old behaviour of methods 3, 4 and 5 above remain with no changes needed. 5 - A common IcsSslRootCAStore component is now created at application start-up, to avoid different components needing their own CA stores to verify SSL/TLS certificates, and for applications to load those stores. The three different CA stores included with ICS are now supplied as resource files, with a define determining which is linked into applications. Another define causes OpenSSL and this store to be loaded at application startup, so OpenSSL is available for all components, without it needing to be loaded again, perhaps repeatedly. Without new defines, a CA Store can be loaded manually into IcsSslRootCAStore. The ICS servers use CA Stores now use IcsSslRootCAStore and no longer load any files specified. 6 - All SSL/TLS servers need a certificate and private key to start, even when testing. Previously ICS supplied some self signed certificates for testing, and also created such certificates automatically if they were missing or if the server was about to order a Let's Encrypt certificate. Accessing such servers for testing using browsers raised various warnings. ICS now has it's own SSL root certificate 'ICS Root CA' and two intermediates, 'ICS Intermediate' and 'ICS Intermediate Short', the last of which includes a private key so can be used to automatically sign new certificates by ICS server applications, rather than just self signed certificates as before. If the 'ICS Root CA' certificate is installed in the Window Store and browser stores, it should stop certificate warnings appearing. ICS applications automatically trust the ICS root, so will give no warnings. The short intermediate has a maximum 100 day expiry, so new versions will be issued regularly. There is a single function CreateSelfSignCertEx that created signed certificates, and another IcsInstallIcsRoot that installs the ICS root into the Windows Store, so easy to use. It is possible to replace the ICS root with your own private root certificate and have servers create their own certificates against that root, for internal networks. 7 - The TSslHttpRest component now allows TRestParams to be created as content type 'Form-Data Body' to create MIME multipart/form-data parameters that may include new TParamType of RPTypeFile that specifies a file name whose binary content will be added to the parameters as a file upload, allowing multiple files and extra parameters. TRestParams are now built into a TStream rather than a string to allow larger parameter sizes, tested up to 8GB. The ICS web server samples have improved MIME decoding to accept massive uploads. 8 - Several client and server components have a new property NoSSL which if set will prevent those components using SSL/TLS for HTTPS or FTPS, even if the application is linked with OpenSSL code. Beware the IcsSslRootCAStore component must not be initialised by the application. 9 - Updating projects to V9.1: Applications that have TSslContext on a form will need to be opened so the new unit OverbyteIcsSslBase is automatically added to the users clause. Units that reference TX509Base or TX509List mostly for the OnSslHandshakeDone event, may need OverbyteIcsSslBase adding manually if they don't also have TSslContext. The other new units are OverbyteIcsHtmlUtils (for TextToHtmlText, IcsHtmlValuesToUnicode, IcsFindHtmlCharset, IcsFindHtmlCodepage, IcsContentCodepage and IcsHtmlToStr), OverbyteIcsDnsHttps (for TDnsQueryHttp and IcsDomNameCacheHttps) and OverbyteIcsSslUtils (for TOcspHttp). Applications that use IcsExtractURLEncodedValue, ExtractURLEncodedParamList or GetCookieValue may need OverbyteIcsUrl adding to projects. When updating projects using a TSslContext component, setting the new property UseSharedCAStore to True causes the properties CAFile, CALines and CAPath to be ignored, and the new IcsSslRootCAStore component will be used instead, being automatically initialised if not done at program start-up. Don't use UseSharedCAStore for server components. High level ICS components such as TSslHttpRest that have an internal TSslContext component all set UseSharedCAStore and ignore properties like SslRootFile to load a root CA bundle. If a specific bundle is required, it may be loaded to IcsSslRootCAStore. With V9,1, the global variables GSSLEAY_DLL_IgnoreNew and GSSLEAY_DLL_IgnoreOld are ignored since only different minor versions of OpenSSL 3 are supported. V9.1 can be downloaded from SVN at https://svn.overbyte.be/svn/icsv9/ or the overnight zip at https://wiki.overbyte.eu/wiki/index.php/ICS_Download Angus

That is an old URL for the same server, it still works but https://svn.overbyte.be/svn/icsv9/ is preferred (or http) and it takes you to the correct repository, there are now several. The strange 8433 port was 15 years ago before I got a dedicated rack server with lots of IPv4 addresses. Angus

This is a simple function I wrote 20 years ago, should work to read strings from HLM keys with any login or none, only writing HLM is protected. function MagGetRegHlmStr (const RegKey, RegValue: string): string ; var IniFile: TRegistry ; begin result := '' ; IniFile := TRegistry.Create ; Try with IniFile do begin try RootKey := HKEY_LOCAL_MACHINE; Access := KEY_QUERY_VALUE ; if OpenKey (RegKey, false) then begin if ValueExists (RegValue) then begin if GetDataType (RegValue) = rdString then result := ReadString (RegValue) ; end; end ; CloseKey ; except end ; end ; finally if Assigned (IniFile) then IniFile.Free; end; end ; Angus

Why not just use one of several open source serial port components for your communication? No-one will want to debug your code. I've been using Async Pro for 20 years, you can install it from GetIt. Angus

STM32 seems to be an ARM based microcontroller, which is not a supported platform for Delphi. So no Delphi applications will run on it. The STM32 should be running an OS of some sort, probably several available, which may run an IP stack to support Ethernet or Bluetooth, in which case numerous Delphi components can be used to communicate with it remotely from Windows or other supported Delphi platforms. Angus

Is it just me, or do the posts by techdesk seem to be written by an AI? I've never seen a developer write such prose. Lots of words, but says very little. Angus

I told you before that the ICS MIME sample does not display file content. Exactly which ICS functions are you using to decode this file> What size is it supposed to be? Which online decoder is able to decode this MIME correctly, without editing? Just telling me it does not work is not helpful. Angus

Only if you installed 12 from GetIt/online originally. I always install from the ISO, and yesterday updated with patch 1 (it is not 12.1) and several component updates using GetIt. Angus

Can not recall what I said in my previous posts here, but can only reiterate that for serving files, threads are not necessary, there is minimal blocking activity. There is a limit to how many new SSL/TLS connections can be accepted each second due to the processing, but once the connection is open, hundreds of connections can run in a single thread. Threads are necessary to make use of more than one CPU. ICS does have a threaded server component, but it does not support SSL/TLS or the web server and has not been tested for 10 years. If you use threads in ICS, each thread has its own message queue and handler, so you can not send from a different thread. Angus

As I wrote, ICS decodes your file correctly into two parts, how your application is supposed to understand such an encoded attachment is a different issue. Angus

The ICS components are event driven, so have various state variables for control, they will need careful manipulation if opening a connection is skipped. Basically, test, fix, repeat. Angus

It would be possible to assign a socket to THttpCli and then call SocketSessionConnected that triggers the first request, perhaps in a descended component. But testing it without your clever device would be painful, it would need a modified web server that made the original connection instead of accepting it, which is where we cross over into proxies. Angus

If you are listening for a connection on port 80, that is a web server, not sure why you then respond with a web command. Or is the embedded device trying to connect to a proxy server? ICS has a proxy component, also TSimpleWebServer and TIcsIpStrmLog that will listen for connections. But none will pass a connection to THttpCli. Angus

I used the OverbyteIcsMimeDemo sample for your EML file, Decode File Extended button which uses TMimeDecodeEx.DecodeFileEx, it seems to work OK: MIME-Version: 1.0 Content-Type: application/octet-stream; name="image.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="image.jpg" Part 0, Content: application/octet-stream, Size: 0, Name: image.jpg, FileName: image.jpg, Encoding: base64, Charset: us-ascii, ApplType: , Content Id: , Reg Extn: Part 1, Content: text/plain, Size: 41, Name: , FileName: , Encoding: , Charset: UTF-8, ApplType: , Content Id: , Reg Extn: .txt Part 2, Content: image/jpeg, Size: 32563, Name: image.jpg, FileName: image.jpg, Encoding: base64, Charset: , ApplType: , Content Id: , Reg Extn: .jpg That sample does not write parts to files, so I can not view the image easily. I tried the same EML file on two online web sites,, and both failed to decode the file, one created an 46K image file containing the raw MIME content. If you want me to look at the other issues, I'll need similar files to run through the MimeDemo sample to reproduce the errors. Angus

I appreciate that, but some of the low level components like TAppSrvClient could be massively simplified if descended from newer ICS components, rather than being complicated to add SSL/TLS. Angus

Looking at the Midware client code, it is very similar to the ICS TIcsIpStrmLog component that can be configured as a client or server, to send textual or binary information between instances, look at the OverbyteIcsIpStmLogTst sample. Midware has some special processing escaping binary that would need adding, but TIcsIpStrmLog could replace much of the low level stuff while adding SSL/TLS. Angus

I will test your file with the MIME sample application next week. Not looked at the code for a long time, it just works, tested with 40GB files last year. Angus

The first question is are all your examples correctly encoded according to the RFCs, and decoded correctly by other MIME implementations. ICS can not be expected to handle non-compliant MIME. Looking at your test file, the MIME has 995 long lines which is allowed in the 30 year old RFC, but 76 is rather more common and recommended. Not sure if long lines are supported. Angus

The StatusCode is a web server response, although sometimes it is generated by the client if the server can not be reached for any reason, there is no official code for DNS failure. You should also be checking LastResponse, ReasonPhase, RequestDoneError and RequestDoneErrorStr for more information after a request. Angus

ICS V9.1 has major structural changes, hundreds of files are disappearing. You should read the svn log in the zip for full details. The release is almost finished and the readme will be done then. The relevant SVN comment: If you are using Delphi 10.4 or later, all existing version specific install groups and packages have gone, since that version of Delphi added a new package $auto suffix allowing the same packages to be used for future Delphi versions. So uninstall all ICS packages, and use one of the new install groups, IcsInstallFmx, IcsInstallVcl, IcsInstallVclFmx or CBIcsInstallVclFmx. These use packages IcsCommonNewRun/Design, IcsVclNewRun/Design and IcsFmxNewRun/Design, and build with the compiler version instead of a Delphi version, i.e. IcsCommonNewRun290.bpl. Angus

Because you installed from an ISO image or before the servers went down, GetIt servers provide much of the platform installation, Delphi, C++. etc. Had the same problem myself. Angus

https://www.theregister.com/2024/01/29/japan_government_floppy_disks/ https://www.floppydisk.com/ has bought up large stocks of floppies for those really in need. I used to carry boxes of 8in and 5in floppies around the world. Angus

I've been using an ICS FTP server on several of my servers for 15 years or so, compiled using Delphi 2007, although the server component is compatible with all compilers. I'm now trying to convert more of my applications to D11. But when built with D11, the FTP server application crashes with a Windows heap corruption exception upon completion of any SSL FTP session, irrespective of what commands were processed during the session. The crash only happens when the application uses OpenSSL DLLs, win32 or Win64, if I build it with YuOpenSSL which links the C code into the app, it does not fail. Despite all my error handling efforts including Madexcept, the application is unable to catch the error, just got lots of Windows Error Reporting and Application Error events. The application itself logs activity, but the last thing logged is the FTP QUIT command, the application crashes before the log writes anything more to disk. What is strange, this crash only happens on server operating systems, specifically Windows Server 2012 and 2022, and only when accessing the server from another computer, not locally. The same problem has been reproduced in two different server applications, running interactively or as a Windows service. The SSL code is well tested and widely used, it's strange the crash only happens on Windows servers in such rare conditions, and that Madexcept can not catch the error, although it is not reported as being in the DLL. Faulting application name: magfserver.exe, version: 2.0.0.7, time stamp: 0x62f69268 Faulting module name: ntdll.dll, version: 10.0.20348.803, time stamp: 0xbee6f04c Exception code: 0xc0000374 Fault offset: 0x00000000001044a9 Faulting process ID: 0xe00 Faulting application start time: 0x01d8ae7494fc7fe2 Faulting application path: C:\magenta\fileserver\magfserver.exe Faulting module path: C:\Windows\SYSTEM32\ntdll.dll Any thoughts on why an application can fail in such rare but very specific circumstances? Angus

I maintain and use DDService, but not Eurekalog. I've been using madExecept for many years, but only for logging errors, none of the restart stuff or emails. Strangely, my main web server has crashed twice during the last two nights, restarted within a few seconds, not happened since November and that was development bugs in ICS. Error was C0000005 buffer overrun. Using YUOpenSSL. But it could be hackers trying to exploit several low priority exploits in OpenSSL due to be fixed this week. Angus

Thanks, I'll look at Application Recovery and Restart, if it applies to Windows Services. Mine are all set to restart if the application stops, and my services go to a lot of trouble to try and save logs and terminate cleanly on any unexpected errors, including emailing me, although I've delayed that now until the service restarts, since corruption sometimes meant the service locked solid instead of stopping and restarting meaning manual intervention. Keep meaning to write a second monitoring application, but these problems are so rare I never get around to it. Angus