Jump to content

Angus Robertson

Members
 View Profile  See their activity

  • Content Count

    1733

  • Joined

  • Last visited

  • Days Won

    33

Posts posted by Angus Robertson

  1. Posted

    ICS does not provide any built-in support for HTTP Public Key Pinning (HPKP), which is now deprecated in favour of Certificate Transparency.  

     

    Most client applications should set SSLVerifyPeer since this causes the server certificate chain to be checked against the roots you supply, with the onHandshakeDone event being called, you can then check your pinning public key hashes or Certificate Transparency server in that event. 

     

    Support for Certificate Transparency is on the ICS wish list, it involves checking the SSL certificate received from the server for the URL of the CT server, then doing another HTTPS request to that server and checking the response, so can cause a significant overhead in performance unless a local database is used to cache results, and then periodically updated.  But no-one has ever asked for Certificate Transparency or HPKP support before, so not done anything.

     

    Angus

     

  2. Posted

    I have tested the threaded multi FTP component with hundreds of simultaneous sessions to the ICS FTP server, but not using a tiny pool of 10 ports, as I thought I explained clearly.  I would have been mostly testing SSL which always uses passive mode.  

     

    That tiny port pool and always starting from the beginning of the pool is your design failure in attempting to support multiple sessions.  Sure the component handling of in-use ports could be better, but should rarely be needed in well designed applications.

     

    Angus

     

  3. Posted

    Quote

    at that point it will create an FTPClient object and start the upload.

    Which resets the port pool to the start, it is not necessary to continually recreate components. 

     

    As Remy pointed out, Windows can listen on the same port more than once deliberately, but applications are mostly written to avoid that happening and use unique ports.  Your use of the component is exposing the difficulties in handling these errors.  The solution is to use the component properly so that errors don't happen, as I've tried to explain. 

     

    I will update with FTP client with ExclusiveAddr to stop duplicate listeners, but you should not rely on this.

     

    Angus

  4. Posted · Edited by Angus Robertson

    ICS does have a property ExclusiveAddr which sets SO_EXCLUSIVEADDRUSE, but it's not used by the FTP client, mostly by servers.  FTP client is really half client half, server in active mode.

     

    I've still not had a proper response to how this application is designed, but it sounds like there are multiple instances of the FTP client running in the same thread, sharing a small pool of 10 ports, and each client starts from the beginning of that pool for each transfer getting errors if the port is in-use, then trying the next port.  If it was a single client, the ports would simply increment each time, ditto if Windows was allowed to assign the port.  This scenario really requires a common port pool between clients, but nobody has reported this problem before. 

     

    But I'm not looking at this further until I understand what problem we are trying to fix.

     

    One simply solution is for each FTP client to use a separate range of 10 ports, 21001 to 21010, 21011 to 21020, etc.  That avoids all conflicts and errors. 

     

    Angus

     

  6. Posted

    Quote

    These jungles of compiler IFDEFs are horrible but spread very widely. Necessity of updating the file with each new compiler version (that Emba bakes out each year) is very dumb.

    It would be really cool if Embarcadero broke with tradition of ignoring anything older than one year and included a unified include file that provides all the conditions that numerous third party libraries need to keep supporting multiple compilers, which Embarcadero then updated and included with each new release.  We could even write it for them.

     

    Component vendors would need a one-off update to the new format, and to include the initial version for the existing users, but would no longer need to keep updating their own versions. 

     

    Angus

     

    • Like 3

  8. Posted

    Quote

    I want to avoid dependencies to third-party components

    If you want to avoid third party components, you are stuck with  TNetHTTPClient which is a bare bones solution requiring extra code. 

     

    That is why there are several better third party components, some free, some expensive. 

     

    I support the free ICS which has a forum here, it includes a component that accesses the GMail API with OAuth2, most other Google APIs are probably similar in concept.

     

    Angus

     

    • Thanks 1

  9. Posted

    TNetHTTPClient uses different technologies on different platforms.  Never used it myself since it did not exist 20 years ago when I stated writing internet applications, but understand it uses SChannel for SSL on Windows, so capabilities will vary dependent upon Windows version. 

     

    I have a lot of problems with IIS SSL on Windows Server 2012, each time Windows Update runs and I reboot, I lose all my ECDSA ciphers and the web server does not start since I use ECDSA certificates.  Those are the sort of support headaches you get from using the native functionality and why most people use alternates. 

     

    Angus

     

  11. Posted

    If you want to avoid OpenSSL DLLs, you can buy YuOpenSSL from https://www.yunqa.de/ which is separate compiled DCUs for Delphi 5 to 11.0, offering the two latest OpenSSL releases for 1.1.1 and 3.0.  Available for ICS, Indy and Synapse. 

     

    With ICS, you just change a define, and YuOpenSSL works without any other code changes.

     

    While using the OS HTTP client may seem easier, you may hit limitations or bugs that take years to get fixed, like access to new protocols like TLS/1.3. 

     

    Angus

  15. Posted

    Something is very wrong, OpenSSL can not report 1.0.2 for the 1.1 DLLs.  Make sure you have the latest libcrypto-1_1.dll and libssl-1_1.dll in the same directory as the exe, which can be fun with modern versions of Delphi that use lots of project sub-directories.  Otherwise ICS may open random versions of OpenSSL that are in the Windows path.  

     

    To avoid this, set GSSL_DLL_DIR to your application directory before loading OpenSSL, as happens in the OverbyteIcsHttpsTst sample.

     

    Angus

     

  16. Posted

    All those OpenSSL versions are very old, current versions of ICS don't even support 1.0.2.  The long term support version is 1.1.1.12 which is correctly known as 1.1.1l., there is also 3.3.0 now but that it is very new.  The ICS V8.67 download includes 1.1.1l so I'm guessing you are using old ICS as well.  You really need to get up to date.

     

    The OverbyteIcsHttpsTst sample illustrates version logging, in this case whether we are using OpenSSL DLLs or statically linking a the YuOpenSSL DCU from https://www.yunqa.de/.

     

            LoadSsl;                      

            if NOT GSSLStaticLinked  then begin
                if NOT FileExists (GLIBEAY_DLL_FileName) then
                    DisplayMemo.Lines.Add('SSL/TLS DLL not found: ' + GLIBEAY_DLL_FileName)
                else
                    DisplayMemo.Lines.Add('SSL/TLS DLL: ' + GLIBEAY_DLL_FileName + ', Version: ' + OpenSslVersion);
            end
            else
                DisplayMemo.Lines.Add('SSL/TLS Static Linked, Version: ' + OpenSslVersion);   

     

    Your original problem appears to happen the moment OpenSSL is loaded so something is probably corrupted, changing SSLType should have moved the error elsewhere which is why the logs are important.

     

    Angus

  17. Posted

    What did the log show when you turned off SSL?  The component would no longer be sending the AUTH TLS command where it currently stalls. 

     

    It's best if applications log the OpenSSL version, although the FTP sample doesn't, because often it's not loaded since SSL is not used, and you can not get the version until it's loaded.  But you should know what version you are distributing! 

     

    Angus

     

  19. Posted

    The onCopyEvent is specifically a logging event, with multiple outputs at different levels, fully illustrated in the OverbyteIcsXferTst sample where lots of stuff flashes past on the screen.  All the new high level components I've added in the last two or three years have a single similar logging event, to avoid needing to add logging into different specific events and format the data.

     

    Angus

  20. Posted

    The IcsLogger is primarily for internal SSL development and testing, not for end user applications.  I need to see the component log from the event handler I mentioned which probably has an SSL after the AUTH TLS is sent. 

     

    BTW, it is quite hard to turn off the Windows Defender FIrewall, the service manager does not allow you to stop the service, you have to change a registry setting.  Turning it off in the GUI may be ignored. 

     

    If you email I'll send the firewall unit, the delay is writing documentation and web pages for a few pending components. 

     

    Angus

     

  21. Posted

    You need to implement logging in the component, using onCopyEvent, see the sample application for an example, which logs all the FTP commands, responses and errors, that is only way to diagnose FTP issues. 

     

    As Francois said, almost certainly Windows firewall or a network router blocking the FTP protocol.  I wrote a unit MagFireWall a couple of years ago which lists and adds firewall rules (admin access required), should really put it on my web site, next month.

     

    Angus

     

×