Angus Robertson

I just tested the 'HTTP Rest Download' function in the snippets sample, that downloads a 300M file from my server, same code as yours, I get a 300M file. How are you reporting the size downloaded? Beware the V9.1 snippets sample won't download from my server any longer, I added server authentication to test something else, and you'll need a new snippets from SVN or the overnight zip that adds authentication as well. Angus

Beware TWinCertificate is only declared in Delphi 12, perhaps ImageHlp as well. But I'll look at those functions to allow verifying the signing chain using OpenSSL instead of the slow capicom.dll COM object used by my component at the moment, but not looked at the component for several years. Angus

This Win64 exception was due to using Free on a stream that was already closed, instead of NilAndFree which checks first, and is used elsewhere in the same function. Not sure why Win32 hides the bug, but Win64 does not, this was within finally/end. Not in SVN yet, and the V9.1 HTTP snippet is now broken because last week I added authentication to the web server directory it uses to test something else, and forget snippets uses that directory. Angus

The sample OverbyteIcsIpStmLogTst is the proper test bed for the TIcsIpStrmLog component. For a successful remote connection, the events triggered are: 11:55:09 C[0] State: Starting 11:55:09 C[0] TCP/Client Opening Connection to [2a00:1940:0:c::128]:80 11:55:09 C[0] TCP/Client Connected OK 11:55:09 C[0] State: OK (manually stopped) 11:55:24 C[0] State: Stopping 11:55:24 C[0] TCP/Client Disconnected 11:55:24 C[0] State: None But for an unsuccessful connection it normally keeps trying to reconnect to the remote IP, so there is no immediate state change: 11:50:56 C[0] State: Starting 11:50:56 C[0] TCP/Client Opening Connection to [2a00:1940:0:c::128]:5000 11:51:17 C[0] TCP/Client Failed Connection - Connection timed out (#10060) - Retrying in 10 secs 11:51:27 C[0] TCP/Client Opening Connection to [2a00:1940:0:c::128]:5000 This is one of the 'features' of the component, if the client connection fails or drops it keeps trying to restore the connection, although that is exactly what you don't want! With RetryAttempts set to -1, the events called are: 12:06:45 C[0] State: Starting 12:06:45 C[0] TCP/Client Opening Connection to [2a00:1940:0:c::128]:6666 12:07:06 C[0] State: Stopping 12:07:06 C[0] State: None 12:07:06 C[0] TCP/Client Failed Connection - Connection timed out (#10060) 12:07:06 C[0] TCP/Client Disconnected So you need to check Stopping and/or None for failure, there is no specific State: Failed event, unless you watch the literals for timeout out. Angus

Most ICS functions are async, not blocking, you missed IpLogClient.onLogProgEvent := onCliLogProgEvent; from snippets, and in the event you check IpLogClient.States[0] for logstateOK or something else if failed. As is often discussed here, it is rarely necessary or useful to use ICS components in a thread, but if so you can not use application.processmessages since that is the main thread, you have to create the component within the thread Execute method, set the MultiThreaded property to true, and call the component ProcessMessages method instead. Angus

The magic word was simplest, less code, fewer errors. Windows supports none blocking DNS lookups and sockets by using a thread, what is the API to stop those threads early? Angus

The simplest way is with the TIcsIpStrmLog component, look at the doSocketRemoteClick procedure in the OverbyteIcsSnippets sample, although you can ignore SSL. Failing to open a normal TCP connection will timeout after about 30 to 40 seconds, you can not easily make this any shorter, and you can not re-use the socket until the connection attempt fails, even if you abort it earlier. If you need to check a lot of ports, either use multiple components running in parallel (no threads needed for hundreds) or use the ping feature of TIcsIpStrmLog to see if at least the IP address exists before checking the port. Angus

ICS V9.2 has started the beta process, and can be downloaded from https://svn.overbyte.be/svn/icsv9/ or the overnight zip from https://wiki.overbyte.eu/wiki/index.php/ICS_Download or https://www.magsys.co.uk/delphi/magics.asp This beta version of V9.2 adds a new feature release of OpenSSL 3.3.0 and fixes a number of bugs mostly introduced in V9.1, but also two long term HTTP URL in the client and server software where missing / delimiters could cause problems, and fixing server authentication issues with POST requests. Several others issues discussed on Delphi-Praxis are also fixed, but not all, yet. This beta also includes a new 'ICS Intermediate Short' SSL certificate to replace the one in V9.0 that has just expired, it is used by ICS to generate temporary server certificates to allow SSL servers to run until a Let's Encrypt or commercial certificate is installed. The OverbyteIcsSslMultiWebServ sample has various improvements to test authentication more thoroughly (the DDService version is not done yet). Only Delphi 10.41 and 10.42 (10.4 with updates 1 or 2) will install correctly with the new install packages, the original RTM version does not support the package LIB suffix: $(Auto) so you must change it manually for each package to 21.0. OpenSSL 3.3.0 is now the default in the OverbyteIcsDefs.inc file and the ICS-OpenSSL path, ICS does not use any of the new features (nor those in 3.1 or 3.2). Now that OpenSSL is more closely integrated with ICS, updating for security fixes will become more complicated, needing files in two or more directories to be updated. When OpenSSL does the next batch of security fix versions (scheduled quarterly), I'll generate a zip with all the new files and directories that can be extracted over an existing ICS installation with all the new files. Angus

{.$DEFINE OpenSSL_Resource_Files} will stop OpenSSL being linked, {$DEFINE] OpenSSL_ProgramData} then helps determine where the DLLs are located. readme9.txt now has a lengthy section describing all the defines. Angus

You are concerned about a one second longer download on a tiny file? With different SSL protocols and ciphers. Angus

In HTTP headers, there is always a space after the colon, seems simple but sufficient to confuse servers. Angus

TSslHttpRest is an ICS component, and to add a special header field you use the component ExtraHeaders: Strings property to add the full header and value, ie ExtraHeaders.Add('Store-Token: 22345673301244567896663456789012'); Angus

I believe the authentication POST problem was mainly a simple literal, if FOutsideFlag and (not (hoAllowOutsideRoot in FOptions)) then Flags := hg403 else Flags := hg404; where hg404 should be hgSendDoc. But something else is going on I'm still tracking, RequestDone should be called for a 401 error to reset the state machine, but is not, although it still seems to work. The biggest problem is our samples test all the authentication variations for virtual and normal pages, but not for template pages or POST pages, so that all had to be added first. I always test server fixes on my public servers for a day or two, so the changes won't be in SVN until later in the week. Angus

OpenSSL has released new versions of the three active versions, 3.2.1, 3.1.5 and 3.0.13 which have three low priority security fixes. Windows binaries are available in SVN and the overnight zip file and separately from https://wiki.overbyte.eu/wiki/index.php/ICS_Download or https://www.magsys.co.uk/delphi/magics.asp In addition to the three DLL files, the zips include compiled RES resource files that contain the same DLLs, text files and version information, see the RC file. The RES file may be linked into application EXE files and code then used to extract the DLLs from the resource to a temporary directory to avoid distributing them separately. ICS V9.1 and later optionally support loading the resource file, currently in SVN and the overnight zip. Beware V9.1 has a lot of other changes that may need application changes, please read the SVN change log very carefully. There will be a lot of new documentation about V9.1 over the next two weeks. Separately, YuOpenSSL has released 3.0.13 as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs. Angus

Correction, OpenSSL 3.3 was released last week and does not add QUIC for servers, that is scheduled for OpenSSL 3.4 due in October 2024. https://github.com/orgs/openssl/projects/11/views/3 OpenSSL 3.3 for Windows will be released later this week with ICS V9.2 beta. But there are no new features particularly relevant to ICS. Angus

I use been using the same two news readers for 25 years, Forte Agent and Ameol2 (very specialised for the UK). I read a number of uk news groups. But there are no active Delphi groups, since Embarcadero moved away from a web forum that allowed NNTP access. I have an account with https://www.astraweb.com/ they sell fixed usage so 25GB for $10 that will last for ever if you don't download binaries. One free news server is news.gmane.io that holds several mailing lists, I read OpenSSL mailing lists via it. Angus

Took some detective work to work out where that template has gone, it was written last autumn to test a major rejig of web server get/post parameter processing using streams for multi-gig uploads. But the template never got added to SVN and then got lost when the samples were re-organised. So I've had to recreate it from my public web site: https://www.telecom-tariffs.co.uk/testing/postinfo.htm Not in SVN yet, still looking at your real problem. Angus

Still got no idea why are distributing openssl.exe and why this is a problem. The directory your screen shows has the correct DLLs for the EXE. We don't need two separate openssl.exe files since we don't use them. The only problem here is you decided to change the way OpenSSL is distributed for your own reasons. Angus

No idea how that screen shot error is relevant to ICS, we don't use openssl.exe. Our openssl zips have a different versions of openssl.exe in the Win32 and Win64 zips, you have mixed therm up. Angus

Thanks, so I need to amend our documentation for those that have not kept D10.4 patches up to date. Angus

If your Delphi app is built for Win32, you only use Win32 DLLs, even when running in a Win64 OS. This has been the case for 25 years. If it is failing, there is another reason. Angus

Support for LIB suffix $(Auto) was only added to D10.4, if the 'new' ICS packages using it are installed with older compilers, $(Auto) will appear in the BPL package file names. Perhaps it was added in a service version of 10.4 you have not installed. Angus

Since $(Auto) was in the fatal error message, it must exist somewhere in your path. ICS only uses it within a package to define where files are built. Angus

I guess you have used the $(Auto) macro as part of the path, which is unfortunately not supported (to my knowledge). You need to replace that with 21.0 for D10.4 for the path. Angus

IcsLogger is really designed for low level event and message handling development, particularly for SSL, its output is rarely of any benefit for application development, I've not used it for development for many years, and it's not used in any of my ICS applications, nor is it used in any modern ICS samples. Most new ICS components and samples have much better embedded logging that is user friendly, the samples use TIcsBuffLogStream to write logs, and supports date/time mask characters in the file name using the FormatDateTime function, so just use that when preparing the file name for IcsLogger. Angus