Angus Robertson

Provided you have set the RefrToken or RefreshToken property, you won't see a login window. It;'s expiry is unknown to the application, but can happen if the account secrets are changed, if you invalidate it through the online console, or at the whim of Google or Microsoft. So applications need to handle token failure, as I mentioned a couple of days ago. The same refresh token can be used on multiple computers, at least for Gmail which is the service I use regularly as a backup when my own SMTP server is down. This why the ICS MailQueue component handles multiple email servers, with OAuth2. Angus

I think you are confusing the two tokens that OAuth2 should return. In ICS, AccToken property is the short lived token used for HTTP requests. The RefreshToken property is a long lived token that you should store safely like a password, and may be used repeatedly by the component to refresh AccToken, in background without any interaction. I tested GMail yesterday, and the saved refresh token meant it just worked without a new login. So you need to check if Azure is returning a refresh token and whether you are saving it for re-use. You may need to provide specific scope settings to get a refresh token, see the constant OAuthUriMSRest. Angus

The ICS packages all build okay with FMX and YuOpenSSL, why do you think it is incompatible? Or at least only with lots of deprecated warnings from OpenSSL, that we never see when using the DLLs. Just built one of the FMX samples and that works fine, once I'd commented out a couple of old lines. Need to update those old samples. Angus

I only removed TSuperWriterSock last year, perhaps there are references to it in an old HPP file built from an earlier ICS version? We don't distribute HPP files, maybe deleting it would cause it to be rebuilt correctly - but I know almost nothing about C++. I'd prefer not to restore unused code. Angus

I assume you are referring to an OAuth2 login page appearing in a browser during authentication. For Google, the Refresh Token you receive after an interactive OAuth2 login remains effective for several months or longer, generally, so can be used by services, believe this is the case with Microsoft as well, but don't recall from testing. Your service can email an admin to update the token manually if it expires. You must store the Refresh Token securely as if it were a password, because that's really what it is, it is used by OAuth2 to get a new 12 hour or something Application Token which is the one you use to access APIs. If you set AuthType to OAuthTypeMan, an event will be called in which you can send the email. The event could launch an interactive application, if the service is running on PC with someone watching the screen. Angus

I only remember one security patch for Delphi, well over 10 years ago, relating to a graphics function that did not check a PChar string length properly. Have there been any others? Angus

Sorry, never done any testing with THttpTunnelWSocket, it does seem to be used by the FTP components as a proxy option, but not tested that feature for a long time either. Angus

UDP application have to be designed to work around data loss, that is not the fault of a low level UDP component. If one loses packets and another does not, the first is probably less efficient. UDP is the basis of HTTP/3 and QUIC, which is reliable. Angus

If you have to send a response on SQL completion, you have to process all JSON in that request before sending a response, so a queue is not that useful, So this is down to your server design, if you are listening for connections that is a server, and you have still not explained how you are doing that, it is not a 'web site', it's a server. Another web server may be sending your server requests, but that is not relevant. Angus

You have still not clarified how exactly you are receiving the JSON, just a vague 'Cliente Server'. But if a new request is stopping an old request, it sounds like you have a single listener for receiving requests, whereas any proper server would accept multiple requests and handle each one separately, sometimes in a thread, but not necessarily with good program design and the ICS internet components. While a FIFO queue can help, there is a problem if you need to send a response for the SQL update status, particularly if you are continually opening and closing SQL connections. None of this design is really relevant to how many JSON records you receive, you just process them in one go, ideally with one SQL update. Angus

How are you receiving the JSON, with a web server or TCP service? I have a similar application that accepts data in various ways and writes to a SQL database, I use a FIFO queue (a stringlist) for the SQL stored procedures, so any that arrive faster than the SQL can accept them (about 20 per second) get queued. If the volume is really high, you can open multiple connections to the SQL server and do some in parallel. Angus

So you are looking for the friendly names for serial COM ports? There are various methods to find them, depending on how were installed: 1 - Enumerate HLM\HARDWARE\DEVICEMAP\SERIALCOMM registry 2 - Enumerate Device Installation Class 'Ports' which finds most serial ports including USB 3 - Optionally add Disabled 'Ports' (hardware not currently installed) 4 - Enumerate Device Installation Class 'CNCPorts' (used by com0com serial port emulator) 5 - Enumerate Device Installation Class 'Modem' which finds USB and internal modems I have an old component that does all this, never got around to publishing it, maybe in December. Angus

You would be better starting with the OverbyteIcsHttpRestTst.dpr sample, even if you don't actually make use of the REST features. The TSslHttpRest component is a high level version of THttpCli that bundles all the extra components for extra functionality, including SSL configuration and certificate validation with a root bundle, SSL session caching, content compression, content code page decoding, persistent cookies, Json handling, logging, client SSL certificate. You ignore SslContext and SslVerifyPeer, and just set CertVerMethod to CertVerBundle or CertVerWinStore and all the chain checking is done for you. You still need to implement your SSL pinning in your own onHandshakeDone event, checking whatever aspect of the server certificate you have been given. My very brief reading about HPKP suggests you hard code a hash of the public key, not a hash of the certificate, so you will need to calculate that yourself, ICS does not have a method to get that hash. But all browsers have abandoned HPKP so I'm not sure what you are trying to achieve. If you are trying to ensure your SSL connection has not been intercepted by an SSL proxy or firewall that has generated a fake local SSL certificate, you can check the server certificate is signed by the expected root authority, although the chain should fail verify anyway if you are using CertVerBundle with the ICS built-in bundles since they won't include the private root certificate used to fake the SSL certificates in the proxy or firewall. Beware this is my understanding of how SSL certificates are faked, I've no direct experience of such corporate environments. Angus

ICS does not provide any built-in support for HTTP Public Key Pinning (HPKP), which is now deprecated in favour of Certificate Transparency. Most client applications should set SSLVerifyPeer since this causes the server certificate chain to be checked against the roots you supply, with the onHandshakeDone event being called, you can then check your pinning public key hashes or Certificate Transparency server in that event. Support for Certificate Transparency is on the ICS wish list, it involves checking the SSL certificate received from the server for the URL of the CT server, then doing another HTTPS request to that server and checking the response, so can cause a significant overhead in performance unless a local database is used to cache results, and then periodically updated. But no-one has ever asked for Certificate Transparency or HPKP support before, so not done anything. Angus

I have tested the threaded multi FTP component with hundreds of simultaneous sessions to the ICS FTP server, but not using a tiny pool of 10 ports, as I thought I explained clearly. I would have been mostly testing SSL which always uses passive mode. That tiny port pool and always starting from the beginning of the pool is your design failure in attempting to support multiple sessions. Sure the component handling of in-use ports could be better, but should rarely be needed in well designed applications. Angus

Which resets the port pool to the start, it is not necessary to continually recreate components. As Remy pointed out, Windows can listen on the same port more than once deliberately, but applications are mostly written to avoid that happening and use unique ports. Your use of the component is exposing the difficulties in handling these errors. The solution is to use the component properly so that errors don't happen, as I've tried to explain. I will update with FTP client with ExclusiveAddr to stop duplicate listeners, but you should not rely on this. Angus

ICS does have a property ExclusiveAddr which sets SO_EXCLUSIVEADDRUSE, but it's not used by the FTP client, mostly by servers. FTP client is really half client half, server in active mode. I've still not had a proper response to how this application is designed, but it sounds like there are multiple instances of the FTP client running in the same thread, sharing a small pool of 10 ports, and each client starts from the beginning of that pool for each transfer getting errors if the port is in-use, then trying the next port. If it was a single client, the ports would simply increment each time, ditto if Windows was allowed to assign the port. This scenario really requires a common port pool between clients, but nobody has reported this problem before. But I'm not looking at this further until I understand what problem we are trying to fix. One simply solution is for each FTP client to use a separate range of 10 ports, 21001 to 21010, 21011 to 21020, etc. That avoids all conflicts and errors. Angus

So you may have several copies of your FTP application running at the same time, uploading to the same server in parallel Why not run a queue so the files are uploaded sequentially. Angus

It would be really cool if Embarcadero broke with tradition of ignoring anything older than one year and included a unified include file that provides all the conditions that numerous third party libraries need to keep supporting multiple compilers, which Embarcadero then updated and included with each new release. We could even write it for them. Component vendors would need a one-off update to the new format, and to include the initial version for the existing users, but would no longer need to keep updating their own versions. Angus

What is your FTP actually attempting to do? Download from multiple FTP servers at the same time? In one thread? It's unlikely I'll have time to investigate this or look at your logs or code unless this is a widespread problem. Angus

If you want to avoid third party components, you are stuck with TNetHTTPClient which is a bare bones solution requiring extra code. That is why there are several better third party components, some free, some expensive. I support the free ICS which has a forum here, it includes a component that accesses the GMail API with OAuth2, most other Google APIs are probably similar in concept. Angus

TNetHTTPClient uses different technologies on different platforms. Never used it myself since it did not exist 20 years ago when I stated writing internet applications, but understand it uses SChannel for SSL on Windows, so capabilities will vary dependent upon Windows version. I have a lot of problems with IIS SSL on Windows Server 2012, each time Windows Update runs and I reboot, I lose all my ECDSA ciphers and the web server does not start since I use ECDSA certificates. Those are the sort of support headaches you get from using the native functionality and why most people use alternates. Angus

Which Delphi HTTP client component are you talking about, there are many different ones from component packages. Angus

If you want to avoid OpenSSL DLLs, you can buy YuOpenSSL from https://www.yunqa.de/ which is separate compiled DCUs for Delphi 5 to 11.0, offering the two latest OpenSSL releases for 1.1.1 and 3.0. Available for ICS, Indy and Synapse. With ICS, you just change a define, and YuOpenSSL works without any other code changes. While using the OS HTTP client may seem easier, you may hit limitations or bugs that take years to get fixed, like access to new protocols like TLS/1.3. Angus

Suggest you follow the recommendations previously suggested, like reporting logs without SSL, or using the OverbyteIcsXferTst sample instead. Some of the things you've reported simply can not happen, like opening old OpenSSL DLLs with the current ICS version, so something is seriously wrong. Angus