ICS V9.1 Highlights

[Angus Robertson 526]

ICS V9.1 is almost ready for release.  Although there are no new components, there are many other SSL/TLS changes that will affect existing applications, but make ICS easier to use and support for the future.

 

Before the final release in a week or two, I'd appreciate some feedback from user installing V9.1 using the new packages, and update one or more old SSL/TLS applications, it may help future users if I can improve the documentation.

 

Please read readme9.txt and these note about V9.1 carefully when upgrading existing applications, you may get build errors that need minor code changes.  But new applications should need be easier to create.

 

1 - Delphi 10.4 and later now use the same install groups and packages, IcsInstallFmx, IcsInstallVcl and IcsInstallVclFmx, making support a lot easier. Version specific groups remain for Delphi 10.3 and earlier, with new groups D(X)InstallVcl for VCL only replacing the old OverbyteIcs(X) groups, again to simplify support.

 

2 - The old samples directory has gone and many of the older and little used samples have been archived to a separate download.  The active samples used to test and demonstrate all ICS components are now split into the following paths, in the ICS root directory:

demos-delphi-vcl - 45 VCL samples for Windows.
demos-delphi-extra - four VCL samples that need third party components to build.
demos-delphi-fmx - seven FMX samples for Windows, not yet tested on MacOS.
demos-cpp-vcl - all old C++ samples that have not been tested for 10 years, need help.
demos-data - data files for samples, such as web pages.

All these samples can now be built for Win32 and Win64 platforms.

 

3 - To ease development, linking and future support, some new units have been added by splitting existing units with multiple components, unfortunately this means many existing projects will need one or more of the new units adding to their uses section.  Apologies for the pain, but this should have been done a long time ago.  The main change is splitting out much of the SSL/TLS related code from the massive OverbyteIcsWSocket unit to a new unit OverbyteIcsSslBase.

 

4 - Distribution of the ICS OpenSSL files has changed.  Earlier ICS versions required the OpenSSL DLLs to be distributed with applications, and a root CA bundle file to verify SSL/TLS connections, and these needed to be loaded using code.  There was little standardisation over where the OpenSSL DLLs were located, applications tended to keep their own copies alongside other executables, leading to multiple DLL copies and needing the public variable GSSL_DLL_DIR set to a specific directory before OpenSSL was loaded.  Likewise, root CA bundle directories had to be distributed with applications and loaded with code. ICS V9.1 allows five different ways of loading OpenSSL:

1 - DLLs linked into application as resource files
2 - DLLs loaded from common directory C:\ProgramData\ICS-OpenSSL\
3 - OpenSSL DCU linked into application using commercial YuOpenSSL
4 - DLLs loaded from location specified in public variable GSSL_DLL_DIR
5 - DLLs loaded according to path, may be found anywhere on PC

Which method ICS uses to load OpenSSL depends upon several defines in the .\Source\Include\OverbyteIcsDefs.inc file, please see the readme9.txt file for details. ICS currently includes resource files for three different OpenSSL releases, 3.0`13. 3.1.5 and 3.2.1, which version is linked is controlled by a define. If the OpenSSL DLLs are linked into the application, they are extracted to a version subdirectory, ie C:\ProgramData\ICS-OpenSSL\3012\ so different applications can use different OpenSSL versions.  This happens only once if the files have not already been extracted.   When updating existing projects without using any new defines, the ICS old behaviour of methods 3, 4 and 5 above remain with no changes needed.

 

5 - A common IcsSslRootCAStore component is now created at application start-up, to avoid different components needing their own CA stores to verify SSL/TLS certificates, and for applications to load those stores.  The three different CA stores included with ICS are now supplied as resource files, with a define determining which is linked into applications.
Another define causes OpenSSL and this store to be loaded at application startup, so OpenSSL is available for all components, without it needing to be loaded again, perhaps repeatedly.  Without new defines, a CA Store can be loaded manually into IcsSslRootCAStore. The ICS servers use CA Stores now use IcsSslRootCAStore and no longer load any files specified.

 

6 - All SSL/TLS servers need a certificate and private key to start, even when testing.  Previously ICS supplied some self signed certificates for testing, and also created such certificates automatically if they were missing or if the server was about to order a Let's Encrypt certificate.  Accessing such servers for testing using browsers raised various warnings.  ICS now has it's own SSL root certificate 'ICS Root CA' and two intermediates, 'ICS Intermediate' and 'ICS Intermediate Short', the last of which includes a private key so can be used to automatically sign new certificates by ICS server applications, rather than just self signed certificates as before. If the 'ICS Root CA' certificate is installed in the Window Store and browser stores, it should stop certificate warnings appearing. ICS applications automatically trust the ICS root, so will give no warnings. The short intermediate has a maximum 100 day expiry, so new versions will be issued regularly. There is a single function CreateSelfSignCertEx that created signed certificates, and another IcsInstallIcsRoot that installs the ICS root into the Windows Store, so easy to use. It is possible to replace the ICS root with your own private root certificate and have servers create their own certificates against that root, for internal networks.

 

7 -  The TSslHttpRest component now allows TRestParams to be created as content type 'Form-Data Body' to create MIME multipart/form-data parameters that may include new TParamType of RPTypeFile that specifies a file name whose binary content will be added to the parameters as a file upload, allowing multiple files and extra parameters. TRestParams are now built into a TStream rather than a string to allow larger parameter sizes, tested up to 8GB. The ICS web server samples have improved MIME decoding to accept massive uploads.

 

8 - Several client and server components have a new property NoSSL which if set will prevent those components using SSL/TLS for HTTPS or FTPS, even if the application is linked with OpenSSL code.  Beware the IcsSslRootCAStore component must not be initialised by the application.

 

9 - Updating projects to V9.1:

 

Applications that have TSslContext on a form will need to be opened so the new unit OverbyteIcsSslBase is automatically added to the users clause. Units that reference TX509Base or TX509List mostly for the OnSslHandshakeDone event, may need OverbyteIcsSslBase adding manually if they don't also have TSslContext.

 

The other new units are OverbyteIcsHtmlUtils (for TextToHtmlText, IcsHtmlValuesToUnicode, IcsFindHtmlCharset, IcsFindHtmlCodepage, IcsContentCodepage and IcsHtmlToStr), OverbyteIcsDnsHttps (for TDnsQueryHttp and IcsDomNameCacheHttps) and OverbyteIcsSslUtils (for TOcspHttp). Applications that use IcsExtractURLEncodedValue, ExtractURLEncodedParamList or GetCookieValue may need OverbyteIcsUrl adding to projects.

 

When updating projects using a TSslContext component, setting the new property UseSharedCAStore to True causes the properties CAFile, CALines and CAPath to be ignored, and the new IcsSslRootCAStore component will be used instead, being automatically initialised if not done at program start-up.  Don't use UseSharedCAStore for server components.

 

High level ICS components such as TSslHttpRest that have an internal TSslContext component all set UseSharedCAStore and ignore properties like SslRootFile to load a root CA bundle.  If a specific bundle is required, it may be loaded to IcsSslRootCAStore.

 

With V9,1, the global variables GSSLEAY_DLL_IgnoreNew and GSSLEAY_DLL_IgnoreOld are ignored since only different minor versions of OpenSSL 3 are supported.

 

V9.1 can be downloaded from SVN at https://svn.overbyte.be/svn/icsv9/

or the overnight zip at https://wiki.overbyte.eu/wiki/index.php/ICS_Download

 

Angus

 

Edited February 27 by Angus Robertson

[FPiette 367]

1 hour ago, Angus Robertson said:

ICS V9.1 is almost ready for release.

Hard work Angus! Thanks a lot.

[ertank 21]

On 2/27/2024 at 5:31 PM, Angus Robertson said:

V9.1 can be downloaded from SVN at https://svn.overbyte.be/svn/icsv9/

Is this a different server than http://svn.overbyte.be:8443/svn/

[Angus Robertson 526]

Quote

Is this a different server than http://svn.overbyte.be:8443/svn/ 

That is an old URL for the same server, it still works but https://svn.overbyte.be/svn/icsv9/ is preferred (or http) and it takes you to the correct repository, there are now several. 

 

The strange 8433 port was 15 years ago before I got a dedicated rack server with lots of IPv4 addresses. 

 

Angus

 

Edited February 29 by Angus Robertson