Problems connecting FTPMultiClient with Explicit TLS?

[Nathan Wild 3]

 

I've had this working for ages, but now for some reason at one customer site, I am not able to connect to the server.  I call .FtpLogon() and it just sits there...  Using the exact same code with the exact same parameters works from another computer, so I am confident that this is something to do with this particular customer's network - however when I create an equivalent connection in WinSCP (Protocol: FTP, Encryption: TLS/SSL Explicit encryption; Port: 21) it works perfectly.  The following is what I am calling...  Any idea what I am missing, and why it would only fail on this particular system?  Thanks in advance for any guidance!!

 

    ftpMultiClient.HostName1 := 'myhost';
    ftpMultiClient.UserName := 'user';
    ftpMultiClient.Password := 'password';
    ftpMultiClient.FtpType := FtpTypeAuthSslBoth;
    ftpMultiClient.Port := '21';
    ftpMultiClient.ConnectionType := ftpDirect;
    ftpMultiClient.PassiveX := True;
    ftpMultiClient.XferMode := XferModeBinary;
    ftpMultiClient.Timeout := 600;
    ftpMultiClient.FtpSslPort := '990';
    ftpMultiClient.SslSessCache := True;
    ftpMultiClient.FtpSslVerMethod := ftpSslVerNone;
    ftpMultiClient.FtpSslRevocation := FALSE;
    ftpMultiClient.FtpSslReportChain := FALSE;
    ftpMultiClient.FtpSslCliSecurity := sslCliSecTls1;
    ftpMultiClient.TarDir := '/';
    ftpMultiClient.BulkMode := BulkModeUpload;

 

[FPiette 383]

You application is probably blocked by Windows firewall which blocks by application. Look into the firewall rules (Advanced settings).

[Fr0sT.Brutal 900]

Turn detailed logs on or use sniffer to realize at what stage the issue happens.

[Angus Robertson 574]

You need to implement logging in the component, using onCopyEvent, see the sample application for an example, which logs all the FTP commands, responses and errors, that is only way to diagnose FTP issues. 

 

As Francois said, almost certainly Windows firewall or a network router blocking the FTP protocol.  I wrote a unit MagFireWall a couple of years ago which lists and adds firewall rules (admin access required), should really put it on my web site, next month.

 

Angus

 

[Nathan Wild 3]

Thanks very much to everyone that has responded so far...  I have ensured that the firewall is completely off, and I did set up an icsLogger to see what was going on, and this is what if gives me:

 

08:41:36:067 ! HighLevelAsync 0
08:41:36:115 Control DNS Lookup Done - 35.167.253.97
08:41:36:115 03746150 Socket handle created handle=1488
08:41:36:137 TWSocket will connect to 35.167.253.97:21
08:41:36:169 Control Socket Connect, error=0 to 35.167.253.97:21
08:41:36:169 03746150 TryToSend handle=1488
08:41:36:170 03746150 TriggerDataSent handle=1488
08:41:36:197 >|220 (vsFTPd 3.0.3)|

08:41:36:197 ! HighLevelAsync 0
08:41:36:197 Start command, Req=HostAsync - HOST ftp1.shophero.com
08:41:36:197 03746150 PutDataInSendBuffer handle=1488  len 24 [1]
08:41:36:197 03746150 TryToSend handle=1488
08:41:36:198 03746150 TryToSend handle=1488
08:41:36:198 03746150 TriggerDataSent handle=1488
08:41:36:219 >|530 Please login with USER and PASS.|

08:41:36:219 ! HighLevelAsync 0
08:41:36:219 Start command, Req=AuthAsync - AUTH TLS
08:41:36:219 03746150 PutDataInSendBuffer handle=1488  len 10 [2]
08:41:36:219 03746150 TryToSend handle=1488
08:41:36:219 03746150 TryToSend handle=1488
08:41:36:219 03746150 TriggerDataSent handle=1488

 

Angus: I would love to see that.  Please do post it if you have a chance.

 

 

[Angus Robertson 574]

The IcsLogger is primarily for internal SSL development and testing, not for end user applications.  I need to see the component log from the event handler I mentioned which probably has an SSL after the AUTH TLS is sent. 

 

BTW, it is quite hard to turn off the Windows Defender FIrewall, the service manager does not allow you to stop the service, you have to change a registry setting.  Turning it off in the GUI may be ignored. 

 

If you email I'll send the firewall unit, the delay is writing documentation and web pages for a few pending components. 

 

Angus

 

[Nathan Wild 3]

Thanks.  I was not aware that icsLogger was for this purpose...  Are there any other relevant events that I should be capturing log info from, or the CopyEvent?

[Angus Robertson 574]

The onCopyEvent is specifically a logging event, with multiple outputs at different levels, fully illustrated in the OverbyteIcsXferTst sample where lots of stuff flashes past on the screen.  All the new high level components I've added in the last two or three years have a single similar logging event, to avoid needing to add logging into different specific events and format the data.

 

Angus

[Nathan Wild 3]

Nice, thanks...

 

This is what it logs:

Connect/Logon to FTP Server: [server redacted]:21
< 220 (vsFTPd 3.0.3)
FTP Session Connected OK to: [ip redacted]:21
> HOST [server redacted]
< 530 Please login with USER and PASS.
> AUTH TLS

[Nathan Wild 3]

It does timeout after a bit.  In my original code, I had the timeout set extremely high...

 

2021/10/29 10:28:56 PM: > AUTH SSL
2021/10/29 10:29:11 PM: FTP Session Closed
2021/10/29 10:29:11 PM: Failed to Connect to FTP Server: 426 Timeout

2021/10/29 10:28:56 PM: > AUTH SSL
2021/10/29 10:29:11 PM: FTP Session Closed
2021/10/29 10:29:11 PM: Failed to Connect to FTP Server: 426 Timeout

 

Edited October 30, 2021 by Nathan Wild

[Angus Robertson 574]

It's not a firewall issue, the server is simply failing to negotiate SSL without any errors, so nothing to diagnose.  You could try changing to sslTypeImplicitso it connects to port 990 instead, or reducing the SslCliSecurity level to see if any older protocol works.  Are your other clients connecting to the same server with the same settings?  Which version of OpenSSL?

 

Angus

[Nathan Wild 3]

Thank you again for your assistance...  I had FtpSslCliSecurity set to sslCliSecTls1.  Setting it to sslCliSecIgnore had no effect.

    ftpMultiClient.FtpType := FtpTypeAuthSslBoth;
    ftpMultiClient.FtpSslVerMethod := ftpSslVerNone;
    ftpMultiClient.FtpSslCliSecurity := sslCliSecIgnore; // Have also tried sslCliSecTls1
    ftpMultiClient.SslType := sslTypeImplicit;  // Have also tried sslTypeNone, sslTypeAuthTls

    ftpMultiClient.FtpType := FtpTypeAuthSslBoth;
    ftpMultiClient.FtpSslVerMethod := ftpSslVerNone;
    ftpMultiClient.FtpSslCliSecurity := sslCliSecIgnore; // Have also tried sslCliSecTls1
    ftpMultiClient.SslType := sslTypeImplicit;  // Have also tried sslTypeNone, sslTypeAuthTls

 

Other clients, including this same application with the same setting from another computer on the network in the same location are working fine.

 

How can I determine the SSL version?  I have the same version of the OpenSSL DLL's deployed with the application in all instances.

 

 

[Angus Robertson 574]

What did the log show when you turned off SSL?  The component would no longer be sending the AUTH TLS command where it currently stalls. 

 

It's best if applications log the OpenSSL version, although the FTP sample doesn't, because often it's not loaded since SSL is not used, and you can not get the version until it's loaded.  But you should know what version you are distributing! 

 

Angus

 

[Nathan Wild 3]

Indeed...  I'm pretty new to OpenSSL, so I just grabbed the required DLLs from another installation.DLLs...  That's why I was wondering how to get the application to report what version of SSL *IT* sees?

I have:

libcrypto-1_1.dll v1.1.0.5

libeay32.dll v1.0.2.11

libssl-1_1.dll v1.1.0.5

ssleay32.dll v1.0.2.11

[Angus Robertson 574]

All those OpenSSL versions are very old, current versions of ICS don't even support 1.0.2.  The long term support version is 1.1.1.12 which is correctly known as 1.1.1l., there is also 3.3.0 now but that it is very new.  The ICS V8.67 download includes 1.1.1l so I'm guessing you are using old ICS as well.  You really need to get up to date.

 

The OverbyteIcsHttpsTst sample illustrates version logging, in this case whether we are using OpenSSL DLLs or statically linking a the YuOpenSSL DCU from https://www.yunqa.de/.

 

        LoadSsl;                      

        if NOT GSSLStaticLinked  then begin
            if NOT FileExists (GLIBEAY_DLL_FileName) then
                DisplayMemo.Lines.Add('SSL/TLS DLL not found: ' + GLIBEAY_DLL_FileName)
            else
                DisplayMemo.Lines.Add('SSL/TLS DLL: ' + GLIBEAY_DLL_FileName + ', Version: ' + OpenSslVersion);
        end
        else
            DisplayMemo.Lines.Add('SSL/TLS Static Linked, Version: ' + OpenSslVersion);   

 

Your original problem appears to happen the moment OpenSSL is loaded so something is probably corrupted, changing SSLType should have moved the error elsewhere which is why the logs are important.

 

Angus

[Nathan Wild 3]

Thanks again...  It reports as:

 

SSL/TLS DLL: C:\WebExport\libcrypto-1_1.dll, Version: OpenSSL 1.0.2k  26 Jan 2017

[Angus Robertson 574]

Something is very wrong, OpenSSL can not report 1.0.2 for the 1.1 DLLs.  Make sure you have the latest libcrypto-1_1.dll and libssl-1_1.dll in the same directory as the exe, which can be fun with modern versions of Delphi that use lots of project sub-directories.  Otherwise ICS may open random versions of OpenSSL that are in the Windows path.  

 

To avoid this, set GSSL_DLL_DIR to your application directory before loading OpenSSL, as happens in the OverbyteIcsHttpsTst sample.

 

Angus

 

[Nathan Wild 3]

DLLs were distributed in same folder as my EXE, but I am not setting GSSL_DLL_DIR to same, just to be safe.  I downloaded the latest version of the OpenSSL stuff from the Overbyte site, and updated to ICS v8.67 while I was at it.  Prior, I was using the 8.62 version that was in GetIT (serves me right for trusting that abominable package manager!?)...

 

Now SSL version reports as libcrypto-3.dll Version: OpenSSL 1.0.0k 5 Feb 2013

 

Detailed FTP log is still giving me the same thing though:

2021/11/03 11:07:48 AM: FTP Session Connected OK to: [mysite]:21
2021/11/03 11:07:48 AM: > HOST [mysite]
2021/11/03 11:07:48 AM: < 530 Please login with USER and PASS.
2021/11/03 11:07:48 AM: > AUTH SSL
2021/11/03 11:08:03 AM: FTP Session Closed
2021/11/03 11:08:03 AM: Failed to Connect to FTP Server: 426 Timeout

 

[Angus Robertson 574]

GetIt has ICS V8.67, but only for the last couple of Delphi releases, they don't update for out of support releases. 

 

If you downloaded OpenSSL 3.0 separately, that is the version that would be reported, ICS no longer opens the 1.0 DLLs, so you are still picking up something ancient randomly installed on your PC.

 

Angus

 

[Nathan Wild 3]

Thanks again...  I am on 10.1 Berlin, which must be old enough that GetIT is out of date.  No worries there as installing it from the Overbyte site worked fine.  I am using the two OpenSSL DLLs that ship with 8.67 (in the OpenSSL-Win32 folder).  I have deleted all other DLLs in my application folder.  I am running the following code:

 

  GSSL_DLL_DIR := ExtractFilePath(Application.EXEName);
  LoadSsl();
  DebugLn('SSL/TLS "' + GLIBEAY_DLL_FileName + ' Version: ' + OpenSslVersion);

Which reports:

 

SSL/TLS "C:\WebExport\libcrypto-1_1.dll Version: OpenSSL 1.0.0k 5 Feb 2013

 

Is there any way it could be sucking in the wrong DLL from somewhere even though I am explicitly setting the path?  

 

 

Edited November 3, 2021 by Nathan Wild
Clarification

[FPiette 383]

Use ProcessExplorer to see exactly which (full path) DLL your application has loaded. Then given which one it is, find the reason why that one is picked by the system. You could also remove that old version (Keep it in a safe place in case one of your applications depends on it).

[Nathan Wild 3]

Thank you very much.  This insight helps.  

 

It looks like I am grabbing C:\Program Files (x86)\Intel\iCLS Client\libeay32.dll along with libssl-1_1.dll and libcrypto-1_1.dll from my application directory.  I have renamed this file and when I execute my application it no longer hooks that DLL, nor does it find any other SSL-looking DLLs other than the two in my application folder, which are the ones that ship with ICS.

 

Application still fails to connect as before... 😞

[FPiette 383]

That's a good and a bad news 🙂

 

If you application grab a file from iCLS Client, then it is likely a system path issue. Remove the directory from the path and restore the file so that iCLS Client will work again (If you don't use that product, you could - probably - uninstall it as well).

 

Now that you get the correct DLL from your application directory, you can start again all tests and checks Angus suggested to understand what is wrong with it. But before trying with your own application, try with one of the ICS samples that we are sure the code is correct.

[Nathan Wild 3]

I DID remove this iCLS DLL, and I have confirmed that my application is loading and running without accessing it anything other than the two OpenSSL DLLs that come with the ICS distro.  The same application works properly on another computer on the same network.  Still perplexed 🙂

[Angus Robertson 574]

Suggest you follow the recommendations previously suggested, like reporting logs without SSL, or using the OverbyteIcsXferTst sample instead. 

 

Some of the things you've reported simply can not happen, like opening old OpenSSL DLLs with the current ICS version, so something is seriously wrong.

 

Angus