Can not login to FTP server without password

[DMX78 0]

Hi, 

 

I have a device that I need to access using FTP component, the device not requiring password, so the password should be left empty, but I can't login to the device's FTP because TFtpClient component will raise an "Password Empty" error, I checked and it happen on this method : 

TCustomFtpCli.PassAsync

where the code will check whether FPassword is empty or not and raise error when empty, so the question is why it have to be a non empty password, even though FTP server should define a password, but there is some that need it to be emptied. I already make the changes by remarking the code line, but I don't know whether it will affect the following code where it use the FPassword. Can someone give an idea on this? or have a fixed by having a property to define whether or not to allow empty password

 

Iwan

[FPiette 382]

Are using the sample code provided with ICS to test or do you use your own code ?

[DMX78 0]

I use my own code but based on OverbyteIcsBasFtp1.pas sample.  Is there a design time property that I need to set? I haven't check that, because I created it at runtime

 

Thank you

[Angus Robertson 574]

It is common for servers to bypass authentication, usually by ignoring any authentication information passed. 

 

With FTP, this is usually called anonymous authentication, you pass any user name and the password anonymous.

 

You say 'device not requiring password' which I would interpret as any password used is ignored, so just use xxx and the component will be happy.  

 

If you mean the device actually requires a specific zero length password, null, space, or the PASS command not to be sent, that will require a change to the component.

 

Angus

[DMX78 0]

57 minutes ago, Angus Robertson said:

It is common for servers to bypass authentication, usually by ignoring any authentication information passed. 

 

With FTP, this is usually called anonymous authentication, you pass any user name and the password anonymous.

 

You say 'device not requiring password' which I would interpret as any password used is ignored, so just use xxx and the component will be happy.  

 

If you mean the device actually requires a specific zero length password, null, space, or the PASS command not to be sent, that will require a change to the component.

 

Angus

Here is a log that I capture when login to the FTP server, the admin user is not using any password, so I try to put any password, but it still saying failed to logged in. Even though I don't understand why it still can download the file

 

12 Jul 2022 16:46:14:839 : OnStatus : > USER admin
12 Jul 2022 16:46:14:839 : OnStatus : < 230 User admin logged in.
12 Jul 2022 16:46:14:855 : OnStatus : > PASS 123
12 Jul 2022 16:46:14:855 : OnStatus : < 530 Invalid password, not logged in.
12 Jul 2022 16:46:14:855 : OnStatus : > TYPE A
12 Jul 2022 16:46:14:855 : OnStatus : < 200 Type set to ASCII.
12 Jul 2022 16:46:14:855 : OnStatus : > PORT 192,168,1,12,224,35
12 Jul 2022 16:46:14:871 : OnStatus : < 200 PORT command successful.
12 Jul 2022 16:46:14:871 : OnStatus : > RETR image.jpg
12 Jul 2022 16:46:14:871 : OnStatus : < 150 Opening Direct data connection for 192.168.1.12.
12 Jul 2022 16:46:14:956 : OnStatus : < 226 Data Transfer complete.
12 Jul 2022 16:46:14:962 : OnStatus : ! 68,4Kbytes received/sent in 94 milliseconds
12 Jul 2022 16:46:14:964 : OnStatus : > QUIT
12 Jul 2022 16:46:14:964 : OnStatus : < 221 User admin logged out.
12 Jul 2022 16:46:14:964 : OnStatus : File Downloaded
12 Jul 2022 16:46:14:964 : OnStatus : Done

 

Iwan

[Angus Robertson 574]

That log says your download worked fine, so the authentication was ignored. 

 

Angus

[DMX78 0]

Hi @Angus Robertson,

 

It is true that I can still download the file, so it's like that the authentication is ignored. It just that I need to bypass the Error raised by Empty Password checking, the download can continue because I ignored the error. No problem though, but would be nice to have a property to ignore empty password or not.

 

Iwan

[Remy Lebeau 1393]

5 hours ago, DMX78 said:

12 Jul 2022 16:46:14:839 : OnStatus : > USER admin
12 Jul 2022 16:46:14:839 : OnStatus : < 230 User admin logged in.
12 Jul 2022 16:46:14:855 : OnStatus : > PASS 123
12 Jul 2022 16:46:14:855 : OnStatus : < 530 Invalid password, not logged in.

Reply code 230 on the USER command means the user is fully logged in, so the PASS command should not have been sent at all.  If the server had actually wanted a password, the USER command would have sent back a 331 reply code instead.

[DMX78 0]

Hi @Remy Lebeau,

 

That's maybe correct, I didn't understand how the FTP protocols works exactly, but if you saying that when the return code is 230, which mean it doesn't necesary to sent PASS again, then there might be something missing in the source code, it should also check when the return code is 230, then don't have to send PASS, do you agree on this?

 

Iwan

[Remy Lebeau 1393]

4 minutes ago, DMX78 said:

I didn't understand how the FTP protocols works exactly

Read RFC 959

4 minutes ago, DMX78 said:

if you saying that when the return code is 230, which mean it doesn't necesary to sent PASS again

Yes.

4 minutes ago, DMX78 said:

then there might be something missing in the source code, it should also check when the return code is 230, then don't have to send PASS, do you agree on this?

I just checked ICS's latest code, and (while it is a bit difficult to follow) it does appear to send PASS after USER only if USER returns 331.  So I don't know why your log shows PASS being sent, unless you are calling the client's Pass/Async() method yourself?  It would help if you would show your actual code, and your setup of the client's properties.

[DMX78 0]

3 minutes ago, Remy Lebeau said:

I just checked ICS's latest code, and (while it is a bit difficult to follow) it does appear to send PASS after USER only if USER returns 331.  So I don't know why your log shows PASS being sent, unless you are calling the client's Pass/Async() method yourself?  It would help if you would show your actual code, and your setup of the client's properties.

 

Here is the code to setup the FTP Client

function TCSClass_ICSFTPHelper.GetFile(
  AFileName: string;
  const AFileStream: TMemoryStream
): Boolean;
begin
  Result := False;
  FFTP.HostName := FHostname;
  FFTP.Port := FPort.ToString;
  FFTP.UserName := FUserName;
  FFTP.PassWord := FPassword;
  FFTP.Binary := FBinaryMode;
  FFTP.Passive := FPassiveMode;
  FRemoteFileName := AFileName;
  FFileStream := TMemoryStream.Create;
  FResult := 0;
  FFTP.Open;
  if FResult = 1 then
  begin
    AFileStream.LoadFromStream(FFileStream);
    FFileStream.Free;
    Result := True;
  end
  else
  begin
    ShowStatus('Failed To Get File : %s',[FErrorMessage])
  end;
end;

 

And here is the code that handle the hand shake, this code is based on the example, and just now I found the problem, thanks to your hint, I did actually still sending the PassAsync method, I'm still trying to understand how to use the component, so I just copy and paste the code from the sample.

procedure TCSClass_ICSFTPHelper.HandleOnFTP_RequestDone(
  Sender: TObject;
  RqType: TFtpRequest;
  ErrCode: Word
);
begin
  if ErrCode <> 0 then begin
    FErrorMessage := Format(
      '*** ERROR (%s) : %s ***',
      [IntToStr(ErrCode),
       FFTP.ErrorMessage
      ]
    );
//    if RqType <> ftpOpenAsync then
//      FFTP.QuitAsync
//    else
//      ShowStatus('Done.',[]);
//    Exit;
  end;

  case RqType of
    ftpOpenAsync:
      FFTP.UserAsync;
    ftpUserAsync:
      FFTP.PassAsync;
    ftpPassAsync:
      if FPath = '' then
        FFTP.TypeSetAsync
      else
      begin
        FFTP.HostDirName := FPath;
        FFTP.CwdAsync;
      end;
    ftpCwdAsync:
      FFTP.TypeSetAsync;
    ftpTypeSetAsync:
      begin
        FFTP.HostFileName := FRemoteFileName;
        FFTP.LocalStream := FFileStream;
        FFTP.GetAsync;
      end;
    ftpGetAsync:
      begin
        FResult := -1;
        if FFTP.StatusCode = 226 then
        begin
          FResult := 1;
        end;
        FFTP.QuitAsync;
      end;
    ftpQuitAsync:
      begin
        if FResult = 1 then
        begin
          ShowStatus('File Downloaded',[]);
        end;
        ShowStatus('Done',[]);
      end;
  else
    ShowStatus('*** Unknown RqType %s ***',[IntToStr(Ord(RqType))]);
  end;
end;

But don't you think it still better to make sure not to raise error when passting empty password? so even if we call the PassAsync directly it will try to send the empty password. This at least will not giving me a half day of work to find the culprit 😊

 

Iwan

[Angus Robertson 574]

If ICS allowed to you send an empty password, you'd still get an error from the server. 

 

Your code is specifically sending the password command with FFTP.PassAsync;, remove that, and you won't get an error. 

 

You are using the low level FTP component, the example you should be looking at is OverbyteIcsXferTst.dpr which uses the high level FTP component that hides much of this complexity from you.

 

Angus

[Remy Lebeau 1393]

4 hours ago, DMX78 said:

And here is the code that handle the hand shake, this code is based on the example, and just now I found the problem, thanks to your hint, I did actually still sending the PassAsync method

Yes, you are explicitly calling PassAsync() after UserAsync() is finished, regardless of its outcome.  You need to pay attention the StatusCode of UserAsync() before calling PassAsync(), and make sure your password is not empty if the server requires one.

4 hours ago, DMX78 said:

But don't you think it still better to make sure not to raise error when passting empty password?

No, I don't.  You explicitly (and unconditionally) asked it to send a PASS command, but did not provide it with data to send.  Throwing an error sounds logical to me.  It is an input error on your part.

4 hours ago, DMX78 said:

so even if we call the PassAsync directly it will try to send the empty password.

All that would accomplish is to cause the server to return back a failure StatusCode, as the PASS command can't have an empty value.

4 hours ago, DMX78 said:

This at least will not giving me a half day of work to find the culprit 😊

Well, that is your own fault, for not learning the FTP protocol before making use of it, and not having adequate error handling in your code.  If you had been paying attention to the StatusCodes of your commands, you would not have been calling PassAsync() unnecessarily to begin with.

[DMX78 0]

5 hours ago, Angus Robertson said:

If ICS allowed to you send an empty password, you'd still get an error from the server. 

But what if the server expected an empty password?

 

5 hours ago, Angus Robertson said:

Your code is specifically sending the password command with FFTP.PassAsync;, remove that, and you won't get an error. 

Yes, I will try that

 

5 hours ago, Angus Robertson said:

You are using the low level FTP component, the example you should be looking at is OverbyteIcsXferTst.dpr which uses the high level FTP component that hides much of this complexity from you.

Thank you for the tip, I will look into that sample

 

2 hours ago, Remy Lebeau said:

Yes, you are explicitly calling PassAsync() after UserAsync() is finished, regardless of its outcome.  You need to pay attention the StatusCode of UserAsync() before calling PassAsync(), and make sure your password is not empty if the server requires one.

Yes, this is due to my lack of research on how to do things

 

2 hours ago, Remy Lebeau said:

No, I don't.  You explicitly (and unconditionally) asked it to send a PASS command, but did not provide it with data to send.  Throwing an error sounds logical to me.  It is an input error on your part.

Agreed, my mistake 😊

 

2 hours ago, Remy Lebeau said:

Well, that is your own fault, for not learning the FTP protocol before making use of it, and not having adequate error handling in your code.  If you had been paying attention to the StatusCodes of your commands, you would not have been calling PassAsync() unnecessarily to begin with.

Lesson learned, thank you 😊

 

Just to share a bit of the background on this problem, before using overbyte FTP, I use Indy FTP, it worked without error but that is when I worked on Delphi 10.4. Then I decided to move the project to 11.1, with the same project, just load it in 11.1 and try to recompile, it can recompiled nicely without much effort, but when I try to execute the FTP command to get a file, I got a read time out error, and after struggling with debugging a package project (with the show stopper problem if you notice https://quality.embarcadero.com/browse/RSP-37631) putting log anywhere I know, I can only find out that the component already execute a transfer file method, and it stop with a read time out error. So after trying hard to get to know what the problem, I decided to use overbyte since I also prefer overbyte for communication component and loved it. Then I just choose a random sample in order to get it working, that's why I use the sample and not knowing that I need to check the status code as @Remy Lebeau mention. This project should be just a recompiled and run on 11.1, it is already working on 10.4 and the project need to be alive within a week, that's why I have to do things much faster and skip researching on the component

 

Thank you all for the support, I will try to improved the code, maybe the sample suggested by @Angus Robertson can help me to minimize the long process of getting a file, I see there is a receive command, maybe that is what I'm looking for to make it simpler.

 

Iwan

 

 

[Fr0sT.Brutal 900]

7 hours ago, DMX78 said:

But what if the server expected an empty password?

"Empty password" not exactly equals to "No password" (just like NULL in database is not '') though the settings in the config/UI could seem identical. But it's up to server to decide what "empty" means.

[Angus Robertson 574]

You didn't give any information on the device you are collecting files from, except the file was an image. 

 

If this is a CCTV camera, the OverbyteIcsXferTst sample and the TIcsFtpMulti and TIcsHttpMulti components it uses may be used to download images and movies from many CCTV cameras automatically, with a few lines of code.  The sample is over complex, due to it supporting lots of different features and components. 

 

I have SV3C cameras, which allow downloads using a web server, TIcsHttpMulti indexes the web pages and downloads new images and movies every two hours, about 20GB a day, the cameras also upload motion capture images automatically to the ICS FTP server.  TIcsFtpMulti  will do the same thing for cameras with an FTP server. 

 

Angus

 

[Remy Lebeau 1393]

15 hours ago, DMX78 said:

But what if the server expected an empty password?

The FTP protocol spec does not allow the password (or the username, or the account) to be empty, it must have at least 1 character.

15 hours ago, DMX78 said:

I worked on Delphi 10.4. Then I decided to move the project to 11.1, with the same project, just load it in 11.1 and try to recompile

It is generally not a good idea to simply migrate a project from one major IDE version to another.  Migrations are not always smooth.  It is usually better to just create a new project fresh in the new IDE and then add your existing source files to it as needed.

15 hours ago, DMX78 said:

when I try to execute the FTP command to get a file, I got a read time out error

Did you report that issue?  I don't recall seeing anything about that recently.

[DMX78 0]

On 7/13/2022 at 4:13 PM, Angus Robertson said:

You didn't give any information on the device you are collecting files from, except the file was an image. 

It's a Vision Camera, it has ftp to download last capture image. I will look into other samples you mention, but will read the status code and not sending password if not needed as suggested by @Remy Lebeau

 

20 hours ago, Remy Lebeau said:

The FTP protocol spec does not allow the password (or the username, or the account) to be empty, it must have at least 1 character.

I see, so I will check for the status code before sending PASS

 

20 hours ago, Remy Lebeau said:

It is generally not a good idea to simply migrate a project from one major IDE version to another.  Migrations are not always smooth.  It is usually better to just create a new project fresh in the new IDE and then add your existing source files to it as needed.

This is correct, but it takes more time to create a fresh project and add all file for the project, I will do that when there is a major problem with the project file, usually when upgrading android project. This project actually loaded fine, compiled fine, working fine, but problem only when using the FTP, it is true that migrating project should not be done at last minutes. Will be more careful on this next time

 

20 hours ago, Remy Lebeau said:

Did you report that issue?  I don't recall seeing anything about that recently.

Haven't made report, maybe I will later, even though I now know the problem might be that it is related with the empty password erorr, but no clear error is the one need to be fixed, but perhaps I didn't do adequate logging mechanism

 

Thank you,

 

Iwan

[Remy Lebeau 1393]

On 7/14/2022 at 5:25 AM, DMX78 said:

Haven't made report, maybe I will later, even though I now know the problem might be that it is related with the empty password erorr, but no clear error is the one need to be fixed, but perhaps I didn't do adequate logging mechanism

A read timeout error has nothing to do with authentication.  Sounds more like the data connection is probably being blocked.  Remember, FTP uses multiple TCP connections.  I notice in your earlier log that a PORT command is being issued.  That means your FTP client is acting in ACTIVE mode, where the server opens a data connection to the client.  That is not very friendly to routers/proxies.  You should use PASSIVE mode instead, where the client opens a data connection to the server instead.  That is less likely to cause problems.

[DMX78 0]

On 7/16/2022 at 12:01 AM, Remy Lebeau said:

A read timeout error has nothing to do with authentication.  Sounds more like the data connection is probably being blocked.  Remember, FTP uses multiple TCP connections.  I notice in your earlier log that a PORT command is being issued.  That means your FTP client is acting in ACTIVE mode, where the server opens a data connection to the client.  That is not very friendly to routers/proxies.  You should use PASSIVE mode instead, where the client opens a data connection to the server instead.  That is less likely to cause problems.

Okey, will try to set it passive mode. Thank