Angus Robertson

As François has said, to send binary files you should really be using a high level protocol like HTTP or FTP. If you really want to invent your own protocol, you will save a lot of time and effort by using the newish TIcsIpStrmLog component that can be configured as a client or server, and is really a much easier to use version of WSocket. There is a sample application OverbyteIcsIpStmLogTst.dpr that has a local mode, where the sample runs as both a client and server sending and receiving lines of data to itself. The component has a simple method SendStream that sends a stream of any size read from memory or a file, your earlier example with Send assume Windows can buffer your entire image, which will only work for smallish files. There is a onLogRecvEvent event that returns a string with received binary data you can write to a stream, but you will need to design a protocol to know the file name or type of data being received, it will be returned in variable sized chunks. Angus

Let’s Encrypt / ISRG has today fixed a problem in the server software that issues certificates validated with the “TLS Using ALPN” method, that meant some existing certificates could have been incorrectly issued (they estimate 1%) and is therefore revoking those certificates at 16:00 UTC on 28 January 2022. This will cause servers using these certificates to display untrusted warnings in most browsers and applications that check for revocation. ICS applications using servers with Hosts that automatically order SSL certificates using CertChallenge with ChallAlpnApp will be using these soon to be revoked certificates. Although ICS servers check the validity of SSL certificates, they do not currently check for revocation, mainly since this is the first time in 20 years of my using SSL certificates it has happened. So manual intervention is needed in the next two days, simply delete the certificate file specified in the host property SslCert and restart the server. Upon startup, the server will create a self signed certificate to allow it to start, then immediately order a new Let’s Encrypt certificate which should be downloaded and automatically installed within about 15 seconds. If the server application implements regular certificate checking with the RecheckSslCerts method (the OverbyteIcsSslMultiWeb/Ftp samples do that every two hours), the new certificate will be ordered without restarting the server. ICS client application are not directly effected by these certificates being revoked, unless they access servers that have not replaced the revoked certificates and implement certificate chain checking using the Windows store with the SslRevocation property set true. Because checking revocation slows down connection time, many applications don't do it. But I will look at implementing it in ICS for use with our PEM CA bundles and servers in particular. Anyone whose applications have ordered Let's Encrypt certificates that are about to be revoked should have received an email warning already. https://community.letsencrypt.org/t/2022-01-25-issue-with-tls-alpn-01-validation-method/170450 Angus

You should be using the OverbyteIcsSslMultiFtpServ.dpr sample which was introduced with V8.65, supporting implicit and explicit connections at the same time using multiple listeners, the FTP server hosting ics.ftptest.org has 10 listeners for various services. It also orders Let's Encrypt SSL certificates automatically, although not the multi-domain wild card one shown earlier which was done using another sample. It seems I did not update the old OverbyteIcsSslFtpServ sample for V8.,65, so perhaps the server changes were not backward compatible (we try to avoid that) and I never tested it. I'll put it on my list to check, but won't be making your suggested change since that will break new servers. Angus

To clarify, you are reporting a problem in the ICS FTP server relating to SSL port 990 connections relating to changes made in V8.65 a year ago? What is your actual problem, the code change works fine in my testing with implicit FTP, you can try itself by accessing my public server ics.ftptest.org with anonynous login: Connect/Logon to FTP Server: ics.ftptest.org:990 ! SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD ics.ftptest.org SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD ics.ftptest.org SSL Chain Verification Succeeded ics.ftptest.org 3 SSL Certificates in the verify chain: #3 Issued to (CN): *.ftptest.co.uk Alt Domains (SAN): *.ftptest.co.uk, *.ftptest.org, *.ftptest.org.uk, *.ftptest.uk Issuer (CN): R3, (O): Let's Encrypt Expires: 2022-03-21T16:06:48, Signature: sha256WithRSAEncryption < 220-ics.ftptest.org < 220-ICS TFtpServerW (c) 1998-2021 F. Piette V8.67 < 220 Server: MAGPUB5 at 2022-01-16T13:52:52 FTP Session Connected OK to: [2a00:1940:2:2::142]:990 > HOST ics.ftptest.org < 220 HOST Ok, FTP Server ready. Angus

So there is no HTML and your questions are not really about our proxy, but just how to convert a binary buffer into a string, for which you can use IcsMoveTBytesToString and IcsMoveStringToTBytes which is what the proxy uses. Angus

Please stop sending your comments as private messages as well, I do read this forum, when I'm in the office. Why do you specifically want to use OnDataSendTar and OnDataRevcTar events, they are very low level. If you want to modify headers and/or body, you should be using onHttpReqBody, onHttpRespBody, onHttpReqHdr, onHttpRespHdr, which have a simple String property you can update. If you change the body length, you map also need to change header fields. Angus

Also, our recent OpenSSL DLLs no longer work on Windows XP either, and ICS has removed support for unsupported OpenSSL versions that might still work on XP. We also digitally sign the OpenSSL DLLs and older versions of Windows XP do not recognise the root certificate used today. If you want to support the latest security standards, you need Windows 10. Angus

530 5.7.0 Must issue a STARTTLS command first. - this simply means the server requires an SSL/TLS connection, and you have not sent the command to start it. You are using the old sample that does not support SSL, you should be using OverbyteIcsSslMailSnd.dpr or OverbyteIcsMailQuTst.dpr (but that may not be in old versions). The ncrypt.dll missing error means new versions of ICS are no longer supported on Windows XP, sorry. I recently added some functions to access SSL certificates and private keys that needed newer Windows APIs not in Windows XP. ICS V8.66 is probably the last that supported Windows XP, I'll update the documentation. Angus

Since XP has been obsolete for several years, we don't test ICS against it, only Windows 7 and later, and that will be dropped soon. But there shouldn't be anything specific in ICS to stop it installing. What specific errors did you get with V8.58, and with V8.68? Angus

ICS V8.68 is now also available from GetIt in RAD Studio 11.0. Angus

Did a quick Google search for the error, seems the developer has messing with this 'security feature' for years causing a lot of problems, and recently with TLSv1.3 that changed how sessions are created. and in fixing 1.3 probably broke 1.2. What is really needed is the tick box to turn off the feature, which seems to have gone. I'm removing the port from the session cache name anyway, since conceptually it is wrong even if it does not fix this particular bug, it should save one TLS session set-up. Angus

You said you did not get the error with Indy, was that using TLSv1.3? If this is down to re-using TLS sessions, it might our caching is broken, despite the logging suggesting it is attempting to re-use an old session. Without tracing TLS packets, which is tedious, hard to know if caching really works. Angus

Okay, I can reproduce it by forcing ICS to use TLSv1.2. So not sure if this is really a FileZilla bug given it works on the older versions that did not support TLSv1.3. The old version also has a configuration option 'require TLS session resumption of data connection when using PROT P' which sound like the error message, but I have that ticked on the old beta, and that setting and lots of other interesting and useful ones have disappeared from the new version. Seems like users should go back to the reliable beta rather than the 'release' version <g> Angus

I can find no problem accessing my own FileZilla servers. My hosted server had v0.9.60 beta from a year ago: > PASV < 227 Entering Passive Mode (217,146,102,143,82,95) ! Passive connection requested to: 217.146.102.143:21087, control channel: 217.146.102.143 > MLSD /webapps/telerest/templates/testing/ Check for Old SSL Session Old SSL Session Found Cached < 150 Opening data channel for directory listing of "/webapps/telerest/templates/testing" ! SSL Connected OK with TLSv1.2, cipher ECDHE-ECDSA-AES256-GCM-SHA384, key auth ECDSA, key exchange ECDH, encryption AESGCM(256), message auth AEAD filezilla.ftptest.org SSL Connected OK with TLSv1.2, cipher ECDHE-ECDSA-AES256-GCM-SHA384, key auth ECDSA, key exchange ECDH, encryption AESGCM(256), message auth AEAD < 226 Successfully transferred "/webapps/telerest/templates/testing" It seems after a decade of beta releases, v1 finally came out this summer, so I installed v1.2.0 on my hosted server, unfortunately Windows Firewall blocks it, despite it being added manually, so installed it locally, and it also works. 15:05:41:693 > PASV 15:05:41:693 Starting SSL Session 15:05:41:693 Cache SSL Session: New 15:05:41:693 < 227 Entering Passive Mode (192,168,1,105,251,19) 15:05:41:693 ! Passive connection requested to: 192.168.1.105:64275, control channel: 192.168.1.105 15:05:41:693 > LIST 15:05:41:694 Check for Old SSL Session 15:05:41:695 Old SSL Session Found Cached 15:05:41:695 < 150 Starting data transfer. 15:05:41:697 ! SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD 15:05:41:697 pc21-web5.magenta SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD So no idea why you are seeing error 425, is there something more useful in the FileZilla server log? If this is something to do with re-using SSL sessions, when ICS caches a session it adds the port number to the IP address when saving it, to prevent different services being accessed by the same session, but this is effectively what happens with the FTP data channel. So in TIcsFtpMulti you could try removing FtpCli.ControlSocket.PeerPort from xxNewSession and xxGetSession and see that improves matters. I'm not going to change this until I find out how FileZilla is configured to cause the error. Angus

Thanks, I already have FileZilla installed, albeit an older version, will test later and see what FileZilla has broken. Angus

OverbyteIcsXferTst.dpr is a complete ready to build testing project, takes a couple of minutes to start downloading stuff, with logs. I have FileZilla on one of my public servers, will test it later. Angus

The ICS FTP components have been tested regularly against FileZilla Server since 0.9.10 beta 15 years ago as you can read in the source code, can not recall testing it for a while, since I don't recall it every being broken. If FileZilla has re-invented the FTP protocol in some non-standard way, I'll look at it, once I see full logs from OverbyteIcsXferTst. Angus

You should never use TDateTime for duration calculations, users can change the system time, and summer time saving changes it twice a year (unless you use UTC time). Always use the difference between two GetTickCount64 Int64 values. Angus

No real idea what that response means, ICS does not share TLS sessions, they are new for each connection. Which ICS component are you using? It should be TIcsFtpMulti for which there is a sample OverbyteIcsXferTst.dpr which will allow you to test again FileZilla saving a proper log that may show the error. Angus

Can you please try and install the latest ICS from SVN or the overnight zip, we've made various C++ package changes (CPP 11.0 only) this week that should resolve the lib files not being updated, and fixes some CPP warnings. Same applies for any other CPP users, V8.68 is finished and will be released next week, so now is the time to test it installs correctly. V8.68 is a minor release, mainly install problems, added OpenSSL 3.0.1, and support for new HTTP request and response methods to help caching (Entity Tags), the REST component will now download files of any size, including resuming failed partial downloads, and HTTP error reporting is improved, as illustrated in the previous post here that previously would have said just Abort without any explanation. Angus

Probably unrelated, but just been reading an article in PC Pro magazine about the new Intel 12th generation processors, which have two types of core, performance and efficiency, something ARM has had for a while. Only Windows 11 has the extra code to receive telemetry from the Intel Thread Director in the CPU to negotiate on which cores processes should run. Windows 10 application performance may be more random, and different each time you run it. Which is slightly frightening. So just saying modern CPUs may affect applications in ways you have not considered. VMs are even worse. Angus

All I can say is you must build the common and vcl packages before the design package. But as you say, if you don't put any components on forms you don't need any packages, just build the units that your application actually uses. Angus

BTW, for C++ you should be using the early version of V8.68 from SVN or the overnight zip (same wiki page as the stable version), I fixed a load of C++ package and sample issues back in October so that 10.4 and 11.0 build again, including the ones you mention, sorry only just remembered. Angus

Sorry, I don't support C++, I just distribute files updated by other ICS C++ users. Hopefully one of them will be along shortly to help. OverbyteIcsSslThrdLock.pas has long gone, just remove any reference to it. ICS builds with Delphi 11.0 without any warning, if C++ gives warnings just ignore them. Angus

All that happens internally within the ICS TRestOAuth component, it holds the expiry date so knows when to refresh the access token. Getting a new refresh token has various options since many applications are used unattended, not just Windows services, so it will notify an administrator that a new OAuth2 login us required, but API access will fail until it happens. It is unfortunate that OAuth2 was designed without a refresh expiry date being known, so things could be planned better. Angus