Jump to content
dummzeuch

executing a command with ssh-pascal runs into timeout

Recommended Posts

I am using ISshExec.Exec from @pyscripter's ssh-pascal library for running a command that calculates checksums on all files in a directory tree. Since this is potentially a large tree and some of the files are huge, this takes quite a while.

Unfortunately Exec is synchronously waiting for the output of the command which creates two issues:

 

  1. My program has to wait for the command to finish before it can get any output.
  2. For the huge files calculating the checksum takes longer than the standard timeout which raises an exception in the Exec method (actually in ReadStringFromChannel) and stops reading (the command continues running though)

 

Is there any existing way to get the output asynchronously?

 

EDIT: I just found that my ssh-pascal library was outdated, in particular the reading code has changed significantly. Maybe my second point has become moot. I'll check.

EDIT2: No, the timeout still occurs.

EDIT3: Hm, now I can't reproduce the timeout. Maybe I didn't rebuild the executable correctly before trying, so it was still using the old code.

Edited by dummzeuch

Share this post


Link to post

I am intrigued here !,

 

And really want to ask this , Did you asked any AI for solution ? 

Because the solution is very simple unless there is something missing or the ssh library is broken, it could be broken on both client and server, but what concern me is what server SSH is being used, and few numbers will be great to understand this.

 

Anyway, my first suggestion is to hack your way into it, well it is a workaround to guarantee the TCP socket connection is alive and being pinged, just add another channel for port forwarding if that is allowed on server side, this should solve your problem and enforce ping for that channel on that socket to be communicated hence stopping the timeout, well unless the timeout is coming from different cause and has nothing to do with SSH per se.

 

Also what are the setting on server for keepalive interval and max count ?

Share this post


Link to post

ISshExec.Exec was substantially revised and now works in non-blocking mode.

  • Are you using the latest source?
  • Are you using the latest linssh2 binaries?


You can execute ISshExec in a thread.  Note that then it can be cancelled from the main thread using ISshExec.Cancel.

 

Also you memory leak in TSshExec.Exec · Issue #18 · pyscripter/Ssh-Pascal has been fixed.  Can you confirm that?

 

Edited by pyscripter

Share this post


Link to post

My problem is not that the main thread is blocked by the call but that I get the output of the command only after it has finished (which can take 30 minutes). During that time the output is received and stored in a memory stream to be converted to a string. Access to this partial output not available.

I know that I could adapt the source code to change that behavior. But I was asking whether I may just be missing something (maybe a different interface?) in the library that already exists.

 

@pyscripter Yes, I am (now) using the latest sources. No I haven't updated the binaries, but the problem is in the Delphi code.

(Just in case you missed my edits in the original question: The timeout no longer occurs.)

Share this post


Link to post

I'll probably implement something like this (if I get the time to finish this project). I'll of course submit a patch in that case.

  • Like 1

Share this post


Link to post

Here you go:

type
  TOnSshExecDataRecieve = procedure(const AData: TBytes; ADataLen: Int64) of object;

// ...

  { Execute commands on the host and get Output/Errorcode back }
  ISshExec = interface
    ['{CA97A730-667A-4800-AF5D-77D5A4DDB192}']
    procedure SetBufferSize(Size: Int64);
    procedure Cancel;
    // ---- change 
    procedure SetOnStdErrReceive(ACallback: TOnSshExecDataRecieve);
    procedure SetOnStdOutReceive(ACallback: TOnSshExecDataRecieve);
    // ---- endchange 
    procedure Exec(const Command: string; var Output, ErrOutput: string; var ExitCode: Integer);
    property BufferSize: Int64 write SetBufferSize;
  end;

// ...

type
  TSshExec = class(TInterfacedObject, ISshExec)
  private
    FSession : ISshSession;
    FBufferSize: Int64;
    FCancelled: Boolean;
    // ---- change 
    FOnStdErrReceive: TOnSshExecDataRecieve;
    FOnStdOutReceive: TOnSshExecDataRecieve;
    procedure doOnStdErrReceive(const AData: TBytes; ADataLen: Int64);
    procedure doOnStdOutReceive(const AData: TBytes; ADataLen: Int64);
    // ---- end change 
    procedure SetBufferSize(Size: Int64);
    procedure Cancel;
    procedure Exec(const Command: string; var Output, ErrOutput: string; var ExitCode: Integer);
    // ---- change 
    procedure SetOnStdErrReceive(ACallback: TOnSshExecDataRecieve);
    procedure SetOnStdOutReceive(ACallback: TOnSshExecDataRecieve);
    // ---- end change 
  public
    constructor Create(Session: ISshSession);
  end;

// ...

// ---- change 
procedure TSshExec.doOnStdErrReceive(const AData: TBytes; ADataLen: Int64);
begin
  if Assigned(FOnStdErrReceive) then
    FOnStdErrReceive(AData, ADataLen);
end;

procedure TSshExec.doOnStdOutReceive(const AData: TBytes; ADataLen: Int64);
begin
  if Assigned(FOnStdOutReceive) then
    FOnStdOutReceive(AData, ADataLen);
end;
// ---- end change 

procedure TSshExec.Exec(const Command: string; var Output, ErrOutput: string;
  var ExitCode: Integer);
var
  Channel: PLIBSSH2_CHANNEL;
  M: TMarshaller;
  ReadBuffer, OutBuffer, ErrBuffer: TBytes;
  StdStream, ErrStream: TBytesStream;
  TimeVal: TTimeVal;
  ReadFds: TFdSet;
  BytesRead: ssize_t;
  ReturnCode: integer;
  OldBlocking: Boolean;
begin
  if FSession.SessionState <>  session_Authorized then
    raise ESshError.CreateRes(@Err_SessionAuth);

  FCancelled := False;
  Channel := libssh2_channel_open_session(FSession.Addr);
  if Channel = nil then
    CheckLibSsh2Result(libssh2_session_last_errno(FSession.Addr), FSession,
     'libssh2_channel_open_session');

  TimeVal.tv_sec := 1;  // check for cancel every one second
  TimeVal.tv_usec := 0;

  StdStream := TBytesStream.Create(OutBuffer);
  ErrStream := TBytesStream.Create(ErrBuffer);
  SetLength(ReadBuffer, FBufferSize);
  OldBlocking := FSession.Blocking;
  FSession.Blocking := False;
  try
    Repeat
      ReturnCode := libssh2_channel_exec(Channel,
        M.AsAnsi(Command, FSession.CodePage).ToPointer);
      CheckLibSsh2Result(ReturnCode, FSession, 'libssh2_channel_exec');
    Until ReturnCode <> LIBSSH2_ERROR_EAGAIN;

    // Stop waiting if cancelled of Channel is sent EOF
    while not FCancelled do
    begin
      // Wait until there is something to read on the Channel
      Repeat
        FD_ZERO(ReadFds);
        _FD_SET(FSession.Socket, ReadFds);
        ReturnCode := select(0, @ReadFds, nil, nil, @TimeVal);
        if ReturnCode < 0 then CheckSocketResult(WSAGetLastError, 'select');
        if libssh2_channel_eof(Channel) = 1 then Break;
      Until (ReturnCode > 0) or FCancelled;

      try
        // Standard output
        BytesRead :=  libssh2_channel_read(Channel, PAnsiChar(ReadBuffer),
          FBufferSize);
        CheckLibSsh2Result(BytesRead, FSession, 'libssh2_channel_read_ex');
        // ---- change 
        if BytesRead > 0 then begin
          StdStream.WriteBuffer(ReadBuffer, BytesRead);
          DoOnStdOutReceive(ReadBuffer, BytesRead);
        end;
        // ---- end change 

        // Error output
        BytesRead :=  libssh2_channel_read_stderr(Channel,
          PAnsiChar(ReadBuffer), FBufferSize);
        CheckLibSsh2Result(BytesRead, FSession, 'libssh2_channel_read_ex');
        // ---- change 
        if BytesRead > 0 then begin
          ErrStream.WriteBuffer(ReadBuffer, BytesRead);
          DoOnStdErrReceive(ReadBuffer, BytesRead);
        end;
        // ---- end change 
      except
        on E: Exception do
          begin
            OutputDebugString(PChar(E.Message));
            Break;
          end;
      end;

      // BytesRead will be either > 0 or LIBSSH2_ERROR_EAGAIN until
      // the command is processed
      if BytesRead = 0 then Break;
    end;

    Output := AnsiToUnicode(PAnsiChar(StdStream.Memory),
        StdStream.Size, FSession.CodePage);
    ErrOutput := AnsiToUnicode(PAnsiChar(ErrStream.Memory),
        ErrStream.Size, FSession.CodePage);

    // libssh2_channel_close sends SSH_MSG_CLOSE to the host
    libssh2_channel_close(Channel);
    if FCancelled then
      ExitCode := 130 // ^C on Linux
    else
      Exitcode := libssh2_channel_get_exit_status(Channel);
  finally
    StdStream.Free;
    ErrStream.Free;
    libssh2_channel_free(Channel);
    FSession.Blocking := OldBlocking;
  end;
end;

// ---- change 
procedure TSshExec.SetOnStdErrReceive(ACallback: TOnSshExecDataRecieve);
begin
  FOnStdErrReceive := ACallback
end;

procedure TSshExec.SetOnStdOutReceive(ACallback: TOnSshExecDataRecieve);
begin
  FOnStdOutReceive := ACallback;
end;
// ---- end change 

This passes the raw bytes received to the callback. Maybe it is possible to convert them to a string first? Or maybe call every time a line feed is received? Also, maybe the callbacks are enough so collecting the whole output is no longer necessary.

I'll see how far I get with this.

Edited by dummzeuch
  • Like 1

Share this post


Link to post
1 hour ago, dummzeuch said:

This passes the raw bytes received to the callback. Maybe it is possible to convert them to a string first? Or maybe call every time a line feed is received? Also, maybe the callbacks are enough so collecting the whole output is no longer necessary.

This is roughly what I had in mind.

 

Suggestions:

  • Have one event with an Enumerated parameter TExecOutput = (eoStdout, eoStdErr)
  • Pass to the callback only the newly added bytes in a TBytes.  They can be easily converted to strings using the ISshClient encoding.
  • Keep the output as is.  The user has a choice of not providing a callback.  The overhead is small.

 

Edited by pyscripter

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×