Angus Robertson

My point was the OpenSSL error is so serious it is not even reported, which is why I want to escalate it to OpenSSL support, let them examine your certificates and hopefully fix it? Angus

Is it okay to put your SSL certificates in the OpenSSL mailing list, to try and find out why some can not be read? I've improved the reading of certificate files and bundles to get some meaningful errors, rather than stack error. For instance if I remove some lines of base64, I get 'Error Cert 4 - error:0D07209B:asn1 encoding routines:ASN1_get_object:too long' and more ASN1 errors, but reading your bad certificates just says 'Error Cert 1 - error:00000000:lib(0):func(0):reason(0)' which essentially means no error found. Angus

They are both ICS sites, the wiki site is linked from the main site as 'Find very latest update here' and is updated more often. As you have been told twice, the latest version is V8.64 from May 2020, not the 18 month old version on the main site. Angus

The best place to download ICS and OpenSSL is http://wiki.overbyte.eu/wiki/index.php/ICS_Download and the latest is V8.64. However the zero seconds fix was 15 years ago. Angus

Not sure from this thread what you are doing with these certificates, but if they are on a public web server you can try a test at https://www.ssllabs.com/ssltest/ which is good at testing certificate chains for errors. Angus

Probably the certificate files are incorrectly formatted but previously the error was ignored, and this is corrected. There is a web site that parses ASN1 https://lapo.it/asn1js/ and it reads all the files OK, but that does not mean they are valid X509 certificates. ICS is not the only server using OpenSSL 1.1.1, Certum should make their certificates are compatible with the latest and greatest or others will complain as well. Angus

Thanks for the files. ICS will read two of them OK, but not the other two. Nor can the openssl.exe tool read the bad files using the x509 command. The asn1parse command is unable to read any of your files, but is fine with all mine. So there is something strange about the construction of these Certum certificates. Angus

Yes, please email your bad certificate file, my address is in readme8.txt. Angus

Okay, so something changed in 1,1,1 reading certificates. Unfortunately we are using an old undocumented API PEM_X509_INFO_read_bio which has worked for 15 years, must see if there is something better. Angus

OpenSSL 1.1.0 is also old and no longer supported, you should be using 1.1.1g from http://wiki.overbyte.eu/wiki/index.php/ICS_Download which is the latest, but is unlikely to make much difference to certificate errors. I am currently making certificate reading more robust, but the offending certificates in the bundles have been there a long time so there is no clear reason why we are now seeing errors on just a few PCs. Angus

The problem being discussed here is PEM certificate files with extra content outside the base64 blocks, where there may be text outside the 7-bit ASCII range, and something gets confused deciding whether this is UTF8 or ANSI, not quite sure what yet, or why the new Windows 10 force UTF8 APIs setting would have any effect on content embedded in a De[phi application and not being read from a file. So unless your XAdES file has extra stuff you did not post, it's not this issue. ICS currently supports three versions of OpenSSL with two different DLL names, If you set the global GSSLEAY_DLL_IgnoreNew to true before calling any SSL functions, it will load the old unsupported libeay32.dll instead of the modern one, and it's possible they do not understand UTF8 so work for you. But Indy still uses the older DLLs, so confused as to why removing the new ones will upset it. There is another global variable GSSL_DLL_DIR you can set to the directory from which to load OpenSSL DLLs, again set before calling anything, needs trailing backslash. Angus

Our root stores were taken from the Windows store on one of my servers until last year, but Windows now only includes certificates needed by the server, and downloads new ones it needs. http://wiki.overbyte.eu/wiki/index.php/FAQ_SSL/TLS_Certificate_Authority_Root_Stores explains how we build them now. But all the textual comments are mine, to keep track of them. Angus

There is a fix for the internal root certificate bundle in SVN, I simply suppressed literal comments from the constants which saves space in binaries and avoids any 8-bit characters being read. I added a button in the PemTool sample that loads and displays the various ICS bundles, to make sure they work. I need to look more closely at the various functions in ICS that save and read certificates to make sure they all support UTF8, and so that only real base64 PEM content is sent to OpenSSL to avoid any problems reading existing files, not just roots. That will all need careful testing. Angus

The trusted root certificate currently used by Let's Encrypt, DST Root CA X3, expires in September 2021. Let's Encrypt issued it's own root certificate, ISRG Root X1, some time ago and it is now available in all major browsers. ICS added it to our bundles three years ago in June 2017 with V8.49. From 8th July 2020, Let's Encrypt will start issuing new certificates signed by an intermediate using the new ISRG root certificate, and the chain will only successfully validate for ICS applications that have the new root. If the internal ICS root bundle is used it must be a new release, but a newer external PEM bundle file can be distributed for use with older ICS applications so they can still access web sites with Let's Encrypt certificates. Because Let's Encrypt certificates expire in three months, they will all be using the new root by mid October 2020. Angus

The safest way to read bundle files with unicode characters will be to strip out all the comments I add before passing them to OpenSSL. For maintenance, adding certificate fields makes life easy, I hate long blocks of base64, and previously OpenSSL has seemed good at ignoring added text. But if the comments are causing trouble, they should go. Angus

Thanks, I've fixed certs 10 and 12, that was me on auto pilot copying selected certificates from the automatically generated bundle files and manually converting them into constants with global replace, really need to automate that to avoid errors. But since all the certificates are concatenated in order, it should not make any difference. in theory. Not sure how those • 8-bit characters got into the literals, but I've removed them, and from TrustedCABundle, the larger bundles are built in a different way. Very strange that errors don't show up consistently. I have seen a problem with PSTACK on servers, where stacks were being continually rebuilt to validate certificates,, but that went away when I simplified the code to avoid rebuilding so often. These fixes should be in SVN tomorrow, when I've finished some other stuff. Angus

Perhaps FastCGI instead, which runs continually. I'm just updating the Jose stuff to handle the server side of Json Web Signing/Tokens for REST servers, currently it only does JWS/JWT for REST clients, and was thinking about how it could be used with web servers other than ICS, FastCGI seems a possibility. TIcsMailQueue can be used separately to queue and send email, the queue format is standard SMTP pick-up as used by Microsoft, although I've not used it like that. Angus

You could also try using TIcsMailQueue which sends mail in the same way as TSslSmtpCli but runs a mail queue with repeated retries to multiple servers on failure. Your application would need to keep running until the queue was empty, and not run multiple copies, but mail is more likely to be delivered. Angus

Your console application will need events so you can not really take any short cuts. There is a sample Samples\delphi\MailNewsDemos\OverbyteIcsConSmtp.dpr that should help. Angus

Thanks, will be fixed in SVN later today. Don't fire up Delphi 7 very often. Angus

Historically I prefer major non-Microsoft applications at root level, easier to find. no access restrictions is a bonus. I also installed all Delphi samples and common files in the same directory, and third part libraries, to keep versions from different releases separate. And I do that in VMs with multiple copies. I also prefer my own directory naming convention of real names rather than RAD studio version numbers which almost no-one uses. Angus

I keep the last two versions on my main development PC, plus D2007, so uninstalled 10.2 before installing 10.4, and 10.3 is still fine. I have VMs with all the old versions. For the past 20 years, I've installed Delphi in my own named root directories, never program files, but just realised that I was never offered a choice of install directories while installing 10.4 from the ISO, or if I was, missed it on some cluttered screen. So it now seems to be buried in program files... Angus

Depends what you think SFTP means, there is no standardised protocol with that name. If you mean FTP over SSH with a private key, ICS does not support that. If you mean FTP using SSL/TLS, then ICS does support it, But the source code is not as simple as you think, clients don't need a private key or SSL certificate, only servers, unless you are trying to implement client certificates which is very rare. ICS also does not recognise files with extension PPK, nor whatever content you have in it which is what the OpenSSL error effectively says. But you don't need it. There are several ICS FTP example applications, I suggest you build Samples\delphi\sslinternet\OverbyteIcsXferTst.dpr first. Angus

ICS V8.64 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo and 10.3 Rio, and C++ Builder 2006 to XE3, 10.2 Tokyo and 10.3 Rio. ICS supports VCL and FMX, Win32, Win64 and MacOS targets. The distribution zip includes the latest OpenSSL 1.1.1g win32, with other versions of OpenSSL being available from the download page. Major Changes in ICS V8.64 include: 1 - ICS now supports International Domain Names for Applications (IDNA), i.e. using accents and Unicode characters in domain names. Domain names can only contain lowercase ASCII letters and numbers and a couple of symbols, so Unicode U-Labels (nodes in a domain) must be converted to A-Labels (Punycode ASCII) with an ACE (ASCII Compatible Encoding) prefix. So www.mâgsÿstést.eu becomes www.xn--mgsstst-pwa1e4l.eu and мособлеирц.рф becomes xn--90aijkdmaud0d.xn--p1ai. 2 - Added a new WMI component based on the Magenta Systems WMI and SMART Component which allows listing all WMI classes, adding and updating network adaptor IP addresses, and adding and updating records in Windows DNS Server (2012 and later). 3 - For SSL/TLS servers, the initial client hello sent before protocol and cipher negotiation starts is now available in the onSslServerName event, when it is easier to select the SslContext to be used on shared servers supporting lots of hosts. 4 - There are various improvement for SSL/TLS servers using IcsHosts. NonSSlPort may be zero for to allocate a random port (not for SSL), the new BindPort and BindSrvPort properties show the real port while listening, reported by ListenStates. If SslCert can not be found the server will create a self signed SSL certificate with all the HostNames so the server can start. 5 - There are many X509 SSL certificate ordering improvements from Let's Encrypt using ACME. All the ACME challenges are now supported. tls-alpn-01 which works on SSL port 443 using a special self signed certificate thus avoiding a server running on port 80 as well. dns-01 which updates Domain Name Server TXT records to avoid using any servers, allowing wild card orders, currently Windows DNS Server on the same server, but Cloudfare is planned. Note the OverbyteIcsX509CertsTst sample running on a Windows DNS Server is able to order a certificate with multiple wildcard and normal domain names automatically in a few seconds, where a similar commercial certificate would cost at least $100 per wildcard. 6 - TRestParams used with the TSslHttpRest component, has more parameter content types: PContXML, PContBodyUrlEn, PContBodyJson and PContBodyXML. The existing PContUrlEn and PContJson now specify REST params are sent as URL ? arguments, while the PContBodyxx version send params as content body. XLM parsing into an ISuperObject has been added so SOAP and XML REST responses can be processed similarly to Json responses. 7 - The TFtpServer component has a new TFtpOptions ftpsAuthForceSsl which requires SSL/TLS before allowing the LOGIN to be processed so that clear text credentials can not be sent. May also be set using IcsHosts with AuthForceSsl=True for specific Hosts only. Failure gives '533 USER requires a secure connection'. 8 - Added a Websocket server implementation (HIXIE and HYBIE protocols) ported from phpws project, written by Stan Korotky <stasson@orc.ru>. There is no a specific component for websockets, they are handled internally from a custom TWebSockSrvClient. 9 - Did a refresh of the three SSL/TLS trusted root certificate bundles included with ICS, to add new roots and remove untrusted certificates. 10 - Added CreateSelfSignCertEx to OverbyteIcsSslX509Utils to create self signed certificates with subject alternate names and specific key types. More detailed release notes are at: http://wiki.overbyte.eu/wiki/index.php/ICS_V8.64

The ICS V8.64 distribution includes packages and project groups to install on Delphi and C++ 10.4 Sydney. and is available via GetIt for 10.3 and 10.4, Angus