Angus Robertson

ICS has a component TIcsBlackList that can be used by servers to count access attempts by IP address, and block after a specified number of attempts until after several hours of inactivity. It's use is illustrated in the OverbyteIcsSslMultiWebServ sample. Just noticed these lines in the log for one of my web servers, someone using Alibaba Cloud in Hong Kong has made almost three million access attempts to my web site over several weeks, trying to read access data that is limited to 50 accesses per day. And still trying despite those requests being rejected. 47.76.209.138 attempts 1,481,269, first at 12:18:52, last at 20:00:17 BLOCKED 47.76.99.127 attempts 1,478,638, first at 12:04:36, last at 19:58:57 BLOCKED Should really be reporting the date of first access, but don't normally see hackers continuing this long. The sample shows various ways to detect hackers, such as web site access by IP address instead of host name, that stops hundreds daily on my sites (no HTTP allowed). Angus

CloudFlare is the obvious solution for most commercial web sites, although I find my link site checker app being blocked from some sites CF 'protects'. But this is an ICS web server, and developers have vastly more control over checking and blocking connections than sites using Apache, etc, that need extras to protect them. Although I get the usual general hackers, they are normally easy to block, anyone accessing the SSL site using an IP address immediately goes on the blocked list, or trying to access CGI script, etc. Anagus

Thanks for the thoughts. The user agent strings are partly randomised, lots of different Chrome/xx versions, the Safari version seems to be the same, but is probably legitimate. The SSL HELO packet has some unknown EC groups, but Chrome often has test groups. The ALPN is always blank, and the requests use a URL without www, but blocking either of those would also hit legitimate API users. The server does not currently log any request headers, not sure if VPNs would add anything to identify themselves, as proxies normally do. One possible solution would be counting IP accesses within a /24 or larger block, although that might include some corporates with outgoing blocks, I'd need to update my white lists as well. Don't want to spend too much time on a rare problem... Angus

My Chinese hackers have changed strategy to get around my IP address blocks and access my web site database, that restricts free access to 50 requests a day, paying for unlimited access seems beyond them. So now they are using VPNs, making two requests at a time from thousands of different IP addresses around the world, 3,500 over the last 48 hours, with requests now repeating after 24 hours, |previously I cleared the block list after six hours of no repeat access. I've not yet managed to define an automated strategy to block relatively random IPs, a CAPTCHA would work, but don't want to annoy my users, likewise giving them a free login. Has anyone got a better strategy for blocking unwanted access by IP? Meanwhile, I'll add /24 level IP blocks manually for a few dozen VPN ranges, to means the server will immediately close any connections from those ranges. Last time I did this to block TOR nodes, I accidentally blocked some large corporates resulting in some interesting telephone calls. Angus

ICS V9.4 has been released at: https://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10, 10.1, 10.2, 10.3, 10.4, 11 and 12 and C++ Builder 10.4, 11 and 12. ICS supports VCL and FMX, Win32 and Win64 targets. The distribution zip includes the latest OpenSSL 3.0.16, 3.2.4, 3.3.3 and 3.4.1 for Win32 and Win64. Changes in ICS V9.4 include: 1 - Completed the ICS Application Monitoring system added in V9.3, designed to locally and remotely monitor ICS servers and applications, and to locally restart applications on demand or if they crash. It comprises a small TIcsAppMonCli client monitoring component that is added to ICS applications, usually Windows Servers, but also client applications. This client component communicates with a TIcsAppMonSrv server component, usually running as a Windows Service on the same server so it is able to restart applications, but can also support clients on a LAN. The monitoring server has web and Websocket servers, allowing remote browsers to view the state of all applications being monitored by the server with a continually updated web page. There is also ICS Application Monitor - Remote Manager application that provides remote monitoring of multiple ICS Application Monitor servers using Json web and Websocket requests on a single screen. 2 - The SMTP client component fixes a bug introduced in V9.3 which could corrupt the Content-Transfer-Encoding header line. 3 - In TWSocket, fixed a potential problem using multiple threads where a new connection opened very quickly (ie localhost) and then stalled due to an unexpected connection state. Made DataToString Unicode compatible, only used for diagnostic dump logs. 4 - In the HTTP client, fixed a check for an overflowing buffer when receiving very long headers that could cause failure detecting headers end. Made several URL validation functions public: GetProtocolPort, IsSSLProtocol, IsKnownProtocol and IsKnownProtocolURL. 5 - When creating PKCS12/PFX certificates, change the 3DES cipher to AES256 if the legacy provider is not loaded. 6 - In the TIcsFtpMulti component, skip download of zero length file by creating an empty file, previously this got SSL handshake error. Don't report directories as being downloadable, they are not. If extended passive mode allowed, send EPSV ALL at start so firewalls and NAT routers can handle sessions more efficiently. Added CheckBadUnicode property defaults to false, so that checks for bad Unicode to Ansi conversions with ? are skipped, allowing more complex paths without errors. 7 - The FTP server FEAT request now returns EPRT and EPSV which have been supported for IPv6 for years, but were not advertised for IPv4. 8 - The OverbyteIcsSnippets sample adds two new simple REST snippets to Get/Post Parameters that send them to an ICS server, and the server echoes back those params so you can check what was actually sent. 9 - When loading the OpenSSL DLLs, no longer check they are digitally signed for Windows XP, 2003, Vista and 2008, they don't recognise SHA-256 code signing, never tested since no longer have those old versions available. 10 - The TRestParams method AddItemSO to add a SuperObject now has an Escape parameter defaulting to True, so non-ASCII characters are escaped by default. 11 - The Proxy component TProxyTarget now has a SocketFamily property so target connections can be restricted to TSocketFamily values. Added property SrvTotSess count of server session connections for logging. 12 - In the HTTP Application Web Server, fixed a memory leak with multiple virtual PUT and POST documents. 13 - The OverbyteIcsJoseTst sample 'Sign/Verify Data' tests now support hashes other than SHA-256, selected from the Key and Signing Hash Algorithm drop down list. Also, a private key matching that selection is created automatically, including X25519. 14 - There is a new unit OverbyteIcsWinUtils that contains Windows API functions, built from selected Magenta Systems libraries, with functions needed to build and control Windows Service applications, accessing the Windows registry, Windows firewall, Windows tasks, hardware, and with simple encryption for passwords. Most of these functions are used by the TIcsAppMonSrv server component and IcsAppMon sample, but should have much wider use for Windows Service server applications in particular, like allowing firewall access. It's planned to move most other Windows specific function here for ICS V10. 15 - In the ICMP component, fixed a problem setting property PingMsg to the text to ping. 16 - In the TIcsIpStrmLog component, added method ListenStates which for logprotUdpServer and logprotTcpServer returns a multiline string listing the IP, port, SSL and state of all socket listeners. The CurSockets property now reflects actual TCP Server clients. 17 - In the TIcsMailQueue component, don't keep retrying email that is too short to send with no body or with no sender headers. Added more error handling if the SMTP component fails to build the EML spool file. 18 - In OverbyteIcsUtils, finished the cleanup of old Base64 functions by adding new IcsBase64 functions using TBytes internally to replace old Base64 functions that used AnsiChars, with no overloaded versions for simplicity. Old Base64 versions retained as deprecated for user applications, please update to the IcsBase versions. Added IcsTBytesCompare to compare two TBytes. Added IcsOutputDebugStr for Posix and Windows. Added IcsDateToAStr and IcsDateTimeToAStr with alpha month (Jan/Feb). 19 - The ICS C++ packages for C++ 10.4 and later have been updated with the correct paths for the three supported platforms, and all build and install correctly for Win32. Win64 should also build, but not Win64x Modern which needs fixes in a future release of C++ 12. The release notes for V9.4 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.4 All ICS active samples are available as prebuilt executables, to allow ease of testing without needing to install ICS and build them all. There are four separate zip files split into clients, servers, tools and miscellaneous samples which can be downloaded from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples Angus

That sort of error usually happens if you don't open the form in the IDE, and the new properties are not saved to the DFM, so fail to be read when executed. That property was added six months ago, and no-one else has reported a problem in that time. Angus

Sorry, not sure what point you are making here. The SslCipherList13 property was added in V9.3 and defaults to sslCipherSuitesTLS13, they both exist in V9.4. Angus

Is the server dead once the problem arises, or does it start accepting connections again at some point? The backlog of 15 suggests the default is not being changed, but it is set immediately before Listen so can not be skipped. There is a fix in V9.4 relating to the wrong connection state when connections open very quickly, usually localhost, that could stall WSocket, not sure if it applies to your situation. Angus

Bug now fixed, it was a late change in V9.,4 flushing the log file to disk in case the request failed, but not actually checking the log was opened. You can fix it by removing the lines marked with V9.4. Angus

How do you propose I reproduce that problem? It is most likely something specific to your installation. That URL works fine here: Built With Delphi 11.3 Win64 12:36:51 Opened log file: D:\weblogs\ics\https-rest\ics-httprest-2025-02-25.log 12:36:51 25/02/2025 12:36:51 Starting REST request for URL: https://jsonplaceholder.typicode.com/users 12:36:51 GET https://jsonplaceholder.typicode.com/users 12:36:51 https://jsonplaceholder.typicode.com/users, Getting headers 12:36:51 Async REST request started 12:36:52 Connected OK to: jsonplaceholder.typicode.com (104.21.48.1) Edit; thinking back, it might be first time issue saving INI settings or something, Is it repeatable? Angus

Strangely, that function was on my screen, I've just added some more parameter encoding and decoding functions to that unit. I've removed the / after text, it's a pretty basic function to avoid trying to convert images and files into unicode. Not sure why V9.3 changed anything, that text/ check was previously in the TriggerRequestDone2 function. Angus

Treat API keys and tokens as if they are passwords, and encrypt them, as we have forever, at least in properly written applications. The type of encryption and how you protect the key really depends on your security threat level. Angus

Refreshing the access token (expires within a day) using the refresh token (may not expire for a year or more) is a standard part of any proper OAUTH2 implementation. But getting that refresh token usually requires a browser login, either embedded Edge or separate browser. The ICS library has several examples for OAUTH2 with refresh tokens. Angus

Correct, provided you use the RestRequest method. The data received during the request is handled internally, and made available from various properties, ResponseRaw, ResponseStream, ResponseOctet and ResponseJson, depending on what you want to do with it, or might have been written to a file. Because TSslHttpRest descends from TSslHttpCli, it publishes all it's properties and events, so you can use it without RestRequest is required. Angus

I understand some commercial certificate suppliers are now using the ACME system designed by Let's Encrypt to automatically issue certificates. Presumably adding an authentication step for payment, this might be something ICS should be supporting. Angus

SSL/TLS is continuously changing with new protocols added, old ones obsoleted and rules changing. We can not ignore such changes. OCSP (Online Certificate Status Protocol) Currently, ICS supports OCSP as the way to confirm SSL/TLS certificates are legitimate and not revoked for security reasons. But the industry does not like OCSP, expensive to implement and several flaws that mean most browsers now ignore it. So Let's Encrypt is closing it's OCSP service over the next six months, ICS should gracefully ignore it as the OCSP URL disappears from X509 certificates. In theory, CRLs (Certificate Revocation Lists) are still supported, ICS has supported them since the beginning, but has never provided a means to automatically download them, it would mean an archive of CRL files for each intermediate certificate received that needed updating continually. This could be added to ICS, but would be a lot of effort and slow down connections as the files are downloaded. Google Chrome has it's own version of CRLs, where those from multiple certificate issuers are combined into a single combined list. Has anyone looked at it? The industry solution is X509 certificates with short life times that are replaced automatically regularly. Apple wants to reduce maximum life to 47 days, others want less. Let's Encrypt is planning to optionally issue six day certificates later this year which ICS will support shortly. RPK (Raw Public Keys) SSL/TLS use a private/public key pair to negotiate encryption, usually contained in an X509 certificate to also prove ownership of the server. For many local applications, ownership is less important than encryption, so self signed certificates are often used instead, but even those can be burdensome, despite ICS servers automating them. OpenSSL has now added support for Raw Public Keys, where the server uses just the private key without a certificate, and the clients have a list of acceptable public keys that can be checked to ensure the connection comes from a known private key. Supporting RPKs will involve a minor change to IcsHosts for servers and updating TSslRootCAStore to store a list of public keys that can be checked during the SSL handshake. Note RPKs are really only for app-to-app communication, browsers don't support them. I know one ICS already uses a version of RPKs by checking the public key and not the certificate, does anyone else see an application for RPKs? Let's Encrypt Changes Let's Encrypt is planning several changes this year. Closing OCSP has already been mentioned. Not sure how many ICS users actually make use of Let's Encrypt certificates, I never hear about bugs or problems, so perhaps they just work, or ICS users are still paying for commercial certificates? Let's Encrypt is adding account profiles to allow different types of certificate to be ordered. ICS will allow the profile to be selected using IcsHosts. 'Classic' profile will be the same as now, 90 day certificates. 'TLSServer' profile will be simplified certificates missing the Common Name that servers shouldn't use and several other fields will go, all to reduce certificate size. 'Shortlived' profile will be six day valid certificates. Let's Encrypt is also planning to support IP Addresses in certificates, ICSHosts will also support this. HTTP/2 and HTTP/3 (QUIC) Adding support for HTTP/2 and HTTP/3 to ICS as Delphi code would be a major project, and the only current benefit would seem to be a small performance improvement with complex pages with hundreds of elements to download. So not planning any HTTP/2 or HTTP/3 support. Angus

The issue here is how much time I can spend attempting to support long obsolete versions of Delphi, instead of concentrating on new features and stuff. My time is free and not unlimited. I have not yet removed support for these old Delphi versions from ICS, but those using them need to help investigate problems, not just report them. Angus

The large change with V9.3 was consolidating many type definitions into the Types unit, with their original versions commented out with (* *). I then remove old code a release or two later. Comparing the prior unit with the latest, I see there are a couple of embedded (* *) pairs I did not disable, near lines 1428 and 1467, maybe D2010 has trouble there. Try removing all that code. Or indeed any code that is commented out, which I'll do for the next release anyway. As I said earlier, I can not test this since my D2010 license no longer works. Angus

Do you mean you are using Caddy or nginx as a proxy in front of non-SSL ICS servers? Angus

Clearly some minor recent change in OverbyteIcsWinsock.pas has upset Delphi 2010, but seemingly not older or newer compilers. But our support of ancient compilers is limited, it is more important to support newer compilers. So please look back at the last ICS version you installed in D2010, see what has changed since, and fix it. Angus

The new certificate issued this morning is fine, but the server should really be setup to automatically renew it a few days before it expires, rather than waiting until after it has expired, I saw the expiry warning this morning as well. Angus

ICS V9.4 is now available from GetIt for Delphi 11 and 12. Angus

ICS has always supported the Content-Disposition: form-data header for POSTing files, but seems to bypassed Content-Disposition as a HTTP response header, declaring content as inline or attachment with a file name so it can be saved. We tend to look at Content-Type to determine if content should be saved. So I plan to add web server support to send Content-Disposition: inline and Content-Disposition: attachment, and client support to parse the header and file name. But also wondering whether Content-Disposition should be sent as a request header when POST/PUTing a single file as binary, to avoid needing to base64 encode it within form-data, which is a major overhead for gigabyte sized uploads. Actually easy to add to the REST component that already has such an upload and the web server samples that save such data, but taking the file name from a URL parameter. Done a little Googling, but can not see any official use of Content-Disposition as a request header, maybe I missed something? Any thoughts, is it worth adding to ICS? Angus

The release notes and DelphiPraxis announcements covered that: Also note when building the ICS packages for the first time with 3.4, there may be a dialog 'entry point could not be located', because the new DLLs are only extracted from the resource files when the first application is run, but the packages have built OK. Angus

Sorry, unless I reproduce bugs, hard to fix. Angus