Angus Robertson

What is the benefit of keeping the internal TLS keys? Wireshark? Angus

Thanks, I'll add it shortly. Angus

I'm updating the ICS MQTT client for v3.1.1 at the moment, should be this week. Not planning v5 at the moment, that is totally different. Most servers still support 3.1.1. Angus

I've installed the Mosquitto Broker server locally, and it works OK against the ICS MQTT client. Our sample currently always adds a default username/password, and this was upsetting test.mosquitto.org on the unauthenticated port, so the sample now allows them to be set specifically, and leaving them blank connects OK to test.mosquitto.org, which is also now on a ComboBox dropdown to make testing easier. But this does not explain why PicoMQTT fails (unless it's also authentication). Mosquitto supports 3.1, 3.1.1 and 5, I'll see if I can disable 3.1. There is a comment in the conf file about clients connecting with 3.1.1 and a zero length client Id so the server allocates one, perhaps this explains the difference. Angus

I get Connection NOT AUTHORISED connecting to test.mosquitto.org on ports that say unauthenticated, I'll spend a few minutes looking to see why, but my knowledge of MQTT is zero, and I don't propose to learn the protocol, maintenance of the ICS MQTT component really needs to be done my users that understand and use MQTT. Angus

I would assume the difference between 3.1 and 3.1.1 is relatively small, so it should be easy to change the ICS component. But we'd need to know what that difference is, and a server to test against. Angus

I am not going to change ICS to check for Windows XP automatically. But if you follow my earlier instructions, ICS can be built with SSL, but not loaded automatically, so you can choose whether to use SSL for each application, as happened prior to V9.1. Angus

OpenSSL 3.0 and 3.4 both use ConvertThreadToFiberEx so I guess you were using an older OpenSSL with v8. Never even heard of Fiber APIs before. Not easy to check, Dependency Walker took 18 minutes of CPU time to fill it's treeview for those two DLLs, slowing my desktop to a crawl. Angus

The readme9,txt file explains how to change defines to stop OpenSSL being loaded automatically, some components also have a NoSSL property. Generally, we make it easy to use SSL, not to not use it. Angus

Sorry, we ceased support for Windows XP several years ago, the OpenSSL DLLs do not support it. The oldest OS supported is Windows 7, and that is rarely tested. ICS does not use ConvertThreadToFiberEx, perhaps OpenSSL does. So you may be able to disable all SSL functionality to keep XP working, but you can not use very old OpenSSL DLLs. Or stick with old unsupported code for an unsupported OS. Angus

Sorry, no simple answer, the MQTT component was contributed, and only tested against itself. I don't have any other local MQTT servers to test against, although it was tested against some public servers. I'm aware there are various versions of MQTT and some servers are non-standard, questionable whether ICS should break the protocol to work with devices that don't follow it, please search back in this forum, I think it's been discussed before. Otherwise, you'll need to use Wireshark or something to see how the other clients are communicating with the server, and change the component to match that. Angus

ICS V9.4 has been released at: https://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10, 10.1, 10.2, 10.3, 10.4, 11 and 12 and C++ Builder 10.4, 11 and 12. ICS supports VCL and FMX, Win32 and Win64 targets. The distribution zip includes the latest OpenSSL 3.0.16, 3.2.4, 3.3.3 and 3.4.1 for Win32 and Win64. Changes in ICS V9.4 include: 1 - Completed the ICS Application Monitoring system added in V9.3, designed to locally and remotely monitor ICS servers and applications, and to locally restart applications on demand or if they crash. It comprises a small TIcsAppMonCli client monitoring component that is added to ICS applications, usually Windows Servers, but also client applications. This client component communicates with a TIcsAppMonSrv server component, usually running as a Windows Service on the same server so it is able to restart applications, but can also support clients on a LAN. The monitoring server has web and Websocket servers, allowing remote browsers to view the state of all applications being monitored by the server with a continually updated web page. There is also ICS Application Monitor - Remote Manager application that provides remote monitoring of multiple ICS Application Monitor servers using Json web and Websocket requests on a single screen. 2 - The SMTP client component fixes a bug introduced in V9.3 which could corrupt the Content-Transfer-Encoding header line. 3 - In TWSocket, fixed a potential problem using multiple threads where a new connection opened very quickly (ie localhost) and then stalled due to an unexpected connection state. Made DataToString Unicode compatible, only used for diagnostic dump logs. 4 - In the HTTP client, fixed a check for an overflowing buffer when receiving very long headers that could cause failure detecting headers end. Made several URL validation functions public: GetProtocolPort, IsSSLProtocol, IsKnownProtocol and IsKnownProtocolURL. 5 - When creating PKCS12/PFX certificates, change the 3DES cipher to AES256 if the legacy provider is not loaded. 6 - In the TIcsFtpMulti component, skip download of zero length file by creating an empty file, previously this got SSL handshake error. Don't report directories as being downloadable, they are not. If extended passive mode allowed, send EPSV ALL at start so firewalls and NAT routers can handle sessions more efficiently. Added CheckBadUnicode property defaults to false, so that checks for bad Unicode to Ansi conversions with ? are skipped, allowing more complex paths without errors. 7 - The FTP server FEAT request now returns EPRT and EPSV which have been supported for IPv6 for years, but were not advertised for IPv4. 8 - The OverbyteIcsSnippets sample adds two new simple REST snippets to Get/Post Parameters that send them to an ICS server, and the server echoes back those params so you can check what was actually sent. 9 - When loading the OpenSSL DLLs, no longer check they are digitally signed for Windows XP, 2003, Vista and 2008, they don't recognise SHA-256 code signing, never tested since no longer have those old versions available. 10 - The TRestParams method AddItemSO to add a SuperObject now has an Escape parameter defaulting to True, so non-ASCII characters are escaped by default. 11 - The Proxy component TProxyTarget now has a SocketFamily property so target connections can be restricted to TSocketFamily values. Added property SrvTotSess count of server session connections for logging. 12 - In the HTTP Application Web Server, fixed a memory leak with multiple virtual PUT and POST documents. 13 - The OverbyteIcsJoseTst sample 'Sign/Verify Data' tests now support hashes other than SHA-256, selected from the Key and Signing Hash Algorithm drop down list. Also, a private key matching that selection is created automatically, including X25519. 14 - There is a new unit OverbyteIcsWinUtils that contains Windows API functions, built from selected Magenta Systems libraries, with functions needed to build and control Windows Service applications, accessing the Windows registry, Windows firewall, Windows tasks, hardware, and with simple encryption for passwords. Most of these functions are used by the TIcsAppMonSrv server component and IcsAppMon sample, but should have much wider use for Windows Service server applications in particular, like allowing firewall access. It's planned to move most other Windows specific function here for ICS V10. 15 - In the ICMP component, fixed a problem setting property PingMsg to the text to ping. 16 - In the TIcsIpStrmLog component, added method ListenStates which for logprotUdpServer and logprotTcpServer returns a multiline string listing the IP, port, SSL and state of all socket listeners. The CurSockets property now reflects actual TCP Server clients. 17 - In the TIcsMailQueue component, don't keep retrying email that is too short to send with no body or with no sender headers. Added more error handling if the SMTP component fails to build the EML spool file. 18 - In OverbyteIcsUtils, finished the cleanup of old Base64 functions by adding new IcsBase64 functions using TBytes internally to replace old Base64 functions that used AnsiChars, with no overloaded versions for simplicity. Old Base64 versions retained as deprecated for user applications, please update to the IcsBase versions. Added IcsTBytesCompare to compare two TBytes. Added IcsOutputDebugStr for Posix and Windows. Added IcsDateToAStr and IcsDateTimeToAStr with alpha month (Jan/Feb). 19 - The ICS C++ packages for C++ 10.4 and later have been updated with the correct paths for the three supported platforms, and all build and install correctly for Win32. Win64 should also build, but not Win64x Modern which needs fixes in a future release of C++ 12. The release notes for V9.4 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.4 All ICS active samples are available as prebuilt executables, to allow ease of testing without needing to install ICS and build them all. There are four separate zip files split into clients, servers, tools and miscellaneous samples which can be downloaded from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples Angus

I'd make the general point that ICS is an async library, generally you should never use Sleep(), but events. If you want to delay something, use triggers within a timer. Having multiple ProcessMeesages everywhere is also bad design. Your code should be packaged into an object with events, called before a single message loop. In fact, this design would mean you can test and debug your WS code in a simple GUI before using it in a DLL. From your various comments, I gather you are writing a Websocket client DLL that sends large binary blocks of data to a server, All my testing and the ICS samples are server to client communication, although in theory the code is two-way and similar in client and server. But I simply don't have a way to easily test your requirement. Angus

Sorry, no idea off hand, not used Websockets for sending large binary blocks, only simple ANSI/HTML packets. Reproducing your scenario is not trivial and would take some effort, I'll put it on my list, but it might take a while. Are both server and client ICS WS apps? Angus

I've seen problems with Cloudflare and ICS, it can be sensitive to the User-Agent or strange request headers, it tries to be too clever and fails. Using a real browser User-Agent might help. Angus

Try Sleep(0) or MsgWaitForMultipleObjects before calling the pump, that is what the ICS synchronous methods do. BTW, Snippets is simple examples. Angus

ICS has a specific component for sending email, TIcsMailQueue, it uses a thread with a TSslSmtpCli component to send email previously saved in EML files, using multiple servers and retries over many hours to ensure email is sent. It handles all the SSL stuff for you. The main sample is OverbyteIcsMailQu which has OAuth2, but the component is also used in several server samples. Haven't written an ISAPI for many years, but the component should run in a DLL easily. Angus

ICS has a component TIcsBlackList that can be used by servers to count access attempts by IP address, and block after a specified number of attempts until after several hours of inactivity. It's use is illustrated in the OverbyteIcsSslMultiWebServ sample. Just noticed these lines in the log for one of my web servers, someone using Alibaba Cloud in Hong Kong has made almost three million access attempts to my web site over several weeks, trying to read access data that is limited to 50 accesses per day. And still trying despite those requests being rejected. 47.76.209.138 attempts 1,481,269, first at 12:18:52, last at 20:00:17 BLOCKED 47.76.99.127 attempts 1,478,638, first at 12:04:36, last at 19:58:57 BLOCKED Should really be reporting the date of first access, but don't normally see hackers continuing this long. The sample shows various ways to detect hackers, such as web site access by IP address instead of host name, that stops hundreds daily on my sites (no HTTP allowed). Angus

CloudFlare is the obvious solution for most commercial web sites, although I find my link site checker app being blocked from some sites CF 'protects'. But this is an ICS web server, and developers have vastly more control over checking and blocking connections than sites using Apache, etc, that need extras to protect them. Although I get the usual general hackers, they are normally easy to block, anyone accessing the SSL site using an IP address immediately goes on the blocked list, or trying to access CGI script, etc. Anagus

Thanks for the thoughts. The user agent strings are partly randomised, lots of different Chrome/xx versions, the Safari version seems to be the same, but is probably legitimate. The SSL HELO packet has some unknown EC groups, but Chrome often has test groups. The ALPN is always blank, and the requests use a URL without www, but blocking either of those would also hit legitimate API users. The server does not currently log any request headers, not sure if VPNs would add anything to identify themselves, as proxies normally do. One possible solution would be counting IP accesses within a /24 or larger block, although that might include some corporates with outgoing blocks, I'd need to update my white lists as well. Don't want to spend too much time on a rare problem... Angus

My Chinese hackers have changed strategy to get around my IP address blocks and access my web site database, that restricts free access to 50 requests a day, paying for unlimited access seems beyond them. So now they are using VPNs, making two requests at a time from thousands of different IP addresses around the world, 3,500 over the last 48 hours, with requests now repeating after 24 hours, |previously I cleared the block list after six hours of no repeat access. I've not yet managed to define an automated strategy to block relatively random IPs, a CAPTCHA would work, but don't want to annoy my users, likewise giving them a free login. Has anyone got a better strategy for blocking unwanted access by IP? Meanwhile, I'll add /24 level IP blocks manually for a few dozen VPN ranges, to means the server will immediately close any connections from those ranges. Last time I did this to block TOR nodes, I accidentally blocked some large corporates resulting in some interesting telephone calls. Angus

That sort of error usually happens if you don't open the form in the IDE, and the new properties are not saved to the DFM, so fail to be read when executed. That property was added six months ago, and no-one else has reported a problem in that time. Angus

Sorry, not sure what point you are making here. The SslCipherList13 property was added in V9.3 and defaults to sslCipherSuitesTLS13, they both exist in V9.4. Angus

Is the server dead once the problem arises, or does it start accepting connections again at some point? The backlog of 15 suggests the default is not being changed, but it is set immediately before Listen so can not be skipped. There is a fix in V9.4 relating to the wrong connection state when connections open very quickly, usually localhost, that could stall WSocket, not sure if it applies to your situation. Angus

Bug now fixed, it was a late change in V9.,4 flushing the log file to disk in case the request failed, but not actually checking the log was opened. You can fix it by removing the lines marked with V9.4. Angus