TIdFTP read time out in FinalizeDataOperation

[Davide Angeli 44]

Hi,

 

I'm using TIdFTP to transfer a 100Mb file on a FTP server on cloud. This process always worked fine.

Now I've upgraded to D10.4.2 (I'm using INDY distribute with Delphi). I don't know what is changed now (if something on Delphi, on Indy or something on the FTP server) but now at the end of the file transfer, in the TIdFTP.FinalizeDataOperation the program raises an EIdReadTimeout executing this instruction:

 

...

GetResponse([226, 225, 250]);

...

 

The file seems transferred regularly. There is only something wrong in the finalizing operation. Attached there is the log file produced with TIdLogFile.

 

There is something that I could do or that I could check to avoid this timeout?

 

Thanks 

Davide

 

 

 

 

installa.exe.log

[Remy Lebeau 1436]

In your log, I don't see a final response being received from the server for the STOR command, or the QUIT command.  That would explain the timeout.  Are you able to perform a similar transfer using another FTP client, like Filezilla, and get a final response after the transfer is finished?

[Davide Angeli 44]

14 hours ago, Remy Lebeau said:

In your log, I don't see a final response being received from the server for the STOR command, or the QUIT command.  That would explain the timeout.  Are you able to perform a similar transfer using another FTP client, like Filezilla, and get a final response after the transfer is finished?

Hi Remy, I've just simulate that transfer with Filezilla (3.52.2) and it works fine (no timeout error)...

 

Just retried TIdFTP and unfortunatly with it the timeout is still present (I was hoping it might be a temporary internet connection problem but it doesn't seem to be the case).

 

Attached is the filezilla log if could help. In it I could see a message "226 transfer complete" after the send data. This message is missing in IdFTP log...

filezillatransfer.log

[Remy Lebeau 1436]

9 hours ago, Davide Angeli said:

Hi Remy, I've just simulate that transfer with Filezilla (3.52.2) and it works fine (no timeout error)...

 

Just retried TIdFTP and unfortunatly with it the timeout is still present (I was hoping it might be a temporary internet connection problem but it doesn't seem to be the case).

I don't know what to tell you, then.  TIdFTP is not receiving a final STOR response from the server.  Since your FTP connection is not encrypted, have you tried using a packet sniffer like Wireshark to verify the response is actually being sent on the wire?

[Davide Angeli 44]

On 6/4/2021 at 5:19 PM, Remy Lebeau said:

I don't know what to tell you, then.  TIdFTP is not receiving a final STOR response from the server.  Since your FTP connection is not encrypted, have you tried using a packet sniffer like Wireshark to verify the response is actually being sent on the wire?

I've monitored with WireShark both Filezilla and TIdFTP transfer. I've attached the final part of the transfer of both. I did these tests one after the other. I really don't know why this is happening. I've done a test with Total Commander too and also with this tool there are non timeout problems. Why TIdFTP doesn't receive the answser and waits for a timeout? I could not attach the 2 wireshark log here because they are about 500Mb (but if it could be usefull I could upload it somewhere).

 

[Remy Lebeau 1436]

Quote

Why TIdFTP doesn't receive the answser and waits for a timeout?

Hard to say for sure without seeing the complete FTP sessions, but if I had to guess, the command connection in TIdFTP is likely being dropped part-way through the transfer, which would explain why the final response is not being received, and also why sending a QUIT request after the transfer timeout is causing a RST packet.  Are you performing these transfers through a router/proxy/firewall, by chance?  If so, they tend to drop idle connections after a few minutes, and thus are likely dropping the command connection during a large transfer since the command connection is sitting idle while the transfer is in progress.  TIdFTP has a NATKeepAlive property to help avoid that issue, try enabling and configuring it as needed.

Quote

I could not attach the 2 wireshark log here because they are about 500Mb (but if it could be usefull I could upload it somewhere).

You should filter the logs to isolate just the FTP command connections before saving them.  You don't need 500Mb to show a handful of packets.

Edited June 7, 2021 by Remy Lebeau

[Davide Angeli 44]

15 hours ago, Remy Lebeau said:

Hard to say for sure without seeing the complete FTP sessions, but if I had to guess, the command connection in TIdFTP is likely being dropped part-way through the transfer, which would explain why the final response is not being received, and also why sending a QUIT request after the transfer timeout is causing a RST packet.  Are you performing these transfers through a router/proxy/firewall, by chance?  If so, they tend to drop idle connections after a few minutes, and thus are likely dropping the command connection during a large transfer since the command connection is sitting idle while the transfer is in progress.  TIdFTP has a NATKeepAlive property to help avoid that issue, try enabling and configuring it as needed.

Thanks you just put me on the right path! IdFTP.NATKeepAlive property was already set in my application but probably with wrong parameters, maybe an IntervalMS too long. I've recalibrated the parameters (IdleTimeMS = 30000 and IntervalMS = 10000) and now it seems working properly (no read time out anymore).

 

15 hours ago, Remy Lebeau said:

You should filter the logs to isolate just the FTP command connections before saving them.  You don't need 500Mb to show a handful of packets.

Sorry I'm not a "Wireshark guru" it's the second time for me using it! I've to say that is not so easy to use. I've found how to isolate FTP commands directed to the FTP server IP address but even if the size has reduced, 100 mb still remains. Perhaps it can be reduced more but it is not necessary.