Jump to content

merijnb

Members
 View Profile  See their activity

  • Content Count

    35

  • Joined

  • Last visited

Posts posted by merijnb

  3. Posted

    I don't think this is a gexperts problem, but this might be the best place to look for info.

     

    Since upgrading to 11, the shortcut keys to previous and next identifier reference (ctrl alt up and ctrl alt down) keep breaking.

    When I look in the configuration, they're still there and when I reset them (as in, set them again), they work again, after a reboot it's broken again.

     

    Anyone a tip how to solve this?

  5. Posted

    4 minutes ago, Angus Robertson said:

    I'm not sure that this is actually a problem, since ICS already has various derived components like: THttpAppSrv = class(THttpServer) which don't give any errors. 

     

    But I don't see any purpose in setting the name FWSocketServer.Name either, it might have been used for debugging a long time ago, but I've just removed it from my copy so your problem will go away, unless anything thinks the name is needed?

     

    Angus

     

     

     

    Hey Angus,

     

    I have no need for the name (I wondered why it was set), the reason THttpAppSrv doesn't have this error is because the class definition is not part of another class definition.

  7. Posted

    One of the first things the constructor of THttpServer does is setting FName of FWSocketServer, using the current ClassName as a basis:

      

    constructor THttpServer.Create(AOwner: TComponent);
begin
    inherited Create(AOwner);
    CreateSocket;
{$IFDEF NO_ADV_MT}
    FWSocketServer.Name := ClassName + '_SrvSocket' + IntToStr(WSocketGCount);
{$ELSE}
    FWSocketServer.Name := ClassName + '_SrvSocket' + IntToStr(SafeWSocketGCount);

     

    The name of a component can only consist of 'A'..'Z', 'a'..'z', '_', '0'..'9' as can been seen in System.SysUtils.IsValidIdent() (called from System.SysUtils.TComponent.SetName())

     

    If you define a class, overriding form THttpServer as part of another class:


      

    type TMyClass = class(TObject)

private

 type TMyHttpServer = class(THttpServer)

    The ClassName of TMyHttpServer will become TMyClass.TMyHttpServer, so it contains a dot, hence breaking the constructor of THttpServer.

     

    Fix could be to do a StringReplace() to eliminate dots in the constructor:

      

    constructor THttpServer.Create(AOwner: TComponent);
begin
    inherited Create(AOwner);
    CreateSocket;
{$IFDEF NO_ADV_MT}
    FWSocketServer.Name := StringReplace(ClassName + '_SrvSocket' + IntToStr(WSocketGCount), '.', '', [rfReplaceAll]);
{$ELSE}
    FWSocketServer.Name := StringReplace(ClassName + '_SrvSocket' + IntToStr(SafeWSocketGCount), '.', '', [rfReplaceAll]);

     

  9. Posted

    If I see correctly ReuseAddr isn't used, that means that TFtpClient itself cannot be the cause of reusing a port, you agree?

     

    Quote

    The solution is to use the component properly so that errors don't happen, as I've tried to explain. 

    As I said, I have ways around this, this is more to satisfy my curiosity.

     

    When you say I'm not using the component properly, what do you mean exactly? It's now allowed to use multiple instances of TFtpClient at the same time? I understand there are ways around this, but is there a technical reason you say this?

     

  10. Posted · Edited by merijnb

    43 minutes ago, Angus Robertson said:

    I've still not had a proper response to how this application is designed, but it sounds like there are multiple instances of the FTP client running in the same thread, sharing a small pool of 10 ports, and each client starts from the beginning of that pool for each transfer getting errors if the port is in-use, then trying the next port.  If it was a single client, the ports would simply increment each time, ditto if Windows was allowed to assign the port. 

    Apologies, I thought that was clear by now. Your description is about right, this application will at some point need to upload a file to an FTP server, at that point it will create an FTPClient object and start the upload. In a specific case active FTP with a specified port range is required, and then we ran into this interesting behavior.

     

    Quote

    This scenario really requires a common port pool between clients, but nobody has reported this problem before.

    I don't think that will really solve the issue since (at least for me) it's not clear yet how this is happening in the first place. If I debug this I can see clearly that every time one of the clients tries to decide what port to use (at the moment PORT command should be sent), it will try to start listening on the first port in the range, if it fails (since already in use) it will try the next. The principle in code seems to be ok. Yet somehow there seems to be a gap between a socket calling Listen() and that port actually being unavailable for another socket to listen on, at least, that is the only way I can imagine two separate FTP client instances will give the same port to the server to make the data connection on.

     

    To me it seems there are two strange things about this:

     

    1) how is it possible that two clients give the same port to the server for the active connection, that implies that two sockets were able to listen on the same port simultaneously which is not possible.

    2) how is it possible the file transfers (seem) to work (see the wireshark logs). The only thing I can imagine is that FTP client 1 starts his data socket listening on port A, FTP client 2 for some reason (see point above) thinks he's also listening on port A, the server makes two connections back, both on port A, and they both actually connect to the listening socket from FTP client 1 (as far as I can see there is no check how many clients connect to the listening socket started by the FTP client). This kind of makes sense from socket perspective, but makes no sense at all from application perspective, that could never work (how would FTP client 1 be able to handle both data streams).

     

    Now I can work around this in several ways, so this question is more out of interest (I see a learning opportunity here), what the heck is going on here 🙂

  11. Posted

    49 minutes ago, Angus Robertson said:

    So you may have several copies of your FTP application running at the same time, uploading to the same server in parallel  Why not run a queue so the files are uploaded sequentially. 

     

    Angus

     

     

    No, it's one application.

     

    Queuing is a possibility (or a fall back), but why? There is no technical reason to do so.
    Multiple files can be uploaded simultaneously without problems.

  12. Posted

    2 hours ago, Angus Robertson said:

    What is your FTP actually attempting to do?  Download from multiple FTP servers at the same time?   In one thread? 

     

    It's unlikely I'll have time to investigate this or look at your logs or code unless this is a widespread problem. 

     

    Angus

     

    I'm uploading files to an FTP server, these uploads are triggered from outside of my app and handled by a single thread. Each trigger will start up an FTP client to upload a single file.

  13. Posted

    I'm currently in the situation where I'm using an active FTP client while setting a port range for the data connection.

    If a port range is given, the TCustomFtpCli.PortAsync() method in OverbyteIcsFtpCli will see which of the ports is available from given range, simply by trying to start a listening socket on the first port, go to the next if that excepts, etc.

     

    During tests we have seen the strange behavior that when we start multiple FTP connections in rapid succession, it sometimes happens that multiple clients pick the same port for the data connection (so for example client b picks the same port as client a, while client a is still using that port). It's a bit hard to replicate in a consistent way, but the smallest I've been able to make it is by making an application which creates three TFtpClient objects, let them all connection to the same server, set to active FTP with a port range (7000-7010) to upload a small file. This happens from a single thread.

    Something needed for this to happen seems timing related, I've not been able to reproduce this with an FTP server from ICS, but I did with Filezilla server. Obviously it shouldn't matter that much, since the port used for the data connection is chosen by the client, not the server.

     

    I'm quite at a loss how this can happen, for two reasons:

     

    1. The code in PortAsync() tries to setup a listening socket for each connection, apparently it's sometimes able to setup 2 different listening sockets on the same port, how is this possible?
    2. How is it possible that (besides there are two sockets listening on the same port) it's actually possible to get data to the two different listening sockets while they're listening on the same port, how does the tcp stack in Windows know for which of the two sockets the data should be sent to?

     

    I've added a zip file with logging from both the server and clients from my test, in it there is logging from filezilla server, a wireshark capture from the server's perspective, application log from the three clients (things like OnDisplayMessage from the TFtpClient), ICS log from all three clients (ICSLogger) and a wireshark capture from the client machine.

     

    Besides I'm really curious what I'm missing here I wonder if this is a bug in ICS code (how is it possible it's able to start two listening sockets on the same port) and if it's something I should be worried about.

     

    Thanks in advance, I'm really curious what comes out of this 🙂

    ftp test log.zip

  14. Posted

    We've run into a bug with regards the FTP client in ICS, it boils down to the Listen() method of TCustomWSocket in OverbyteIcsWSocket.pas

     

    At the top of the method, fLastError should be set to 0:

      

    procedure TCustomWSocket.Listen;
var
    iStatus        : Integer;
    optval         : Integer;
    optlen         : Integer;
    mreq           : ip_mreq;
    mreqv6         : TIpv6MReq;
    Success        : Boolean;
    FriendlyMsg    : String;
    ErrorCode      : Integer;
{$IFDEF MSWINDOWS}
    dwBufferInLen  : DWORD;
    dwBufferOutLen : DWORD;
    dwDummy        : DWORD;
{$ENDIF}
begin
    fLastError := 0;   // <-- this should be added
    FriendlyMsg := '';

    The reason this is needed is as follows:

    If you start an active FTP connection to a server, while using a data port range the client will try to setup a server for the data connection on port X, if this fails it will try again on X + 1 until it's able to start a server (or runs out of ports to try).

    This is found inTCustomFtpCli.PortAsync() in OverbyteIcsFtpCli:

      

                StartDataPort := DataPort;
            while TRUE do begin
                FDataSocket.Port := IntToStr(DataPort);
                try
                    FDataSocket.Listen;
                    break;                { Found a free port }
                except
                    if FDataSocket.LastError = WSAEADDRINUSE then begin
                        DataPort := DataPort + 1;
                        if DataPort > FDataPortRangeEnd then
                            DataPort := FDataPortRangeStart;
                        if DataPort = StartDataPort then begin
                            HandleError('All ports in DataPortRange are in use');
                            Exit;
                        end;
                    end
                    else begin
                        HandleError('Data connection winsock bind failed - ' +
                                    GetWinsockErr(FDataSocket.LastError));
                        Exit;
                    end;
                end;
            end;

     

    If you make a new FTP connection while one of the ports is already in use, but PortAsync() is able to find another port, the fLastError is still set to something <> 0 due to the earlier failed port. If you then end up in TCustomFtpCli.PutAsync(), it will still see that LastError and raise an exception over that while we're currently happily listening without problems.

      

    procedure TCustomFtpCli.PutAsync;
begin
	DataSocket.LastError := 0; { V2.100 }
    HighLevelAsync(ftpPutAsync,
                   [ftpFctPort, ftpFctPut]);
    if DataSocket.LastError <> 0 then    { V2.100 }
       raise FtpException.Create('Socket Error - ' +
                              GetWinsockErr(DataSocket.LastError)); { V2.100 }
end;

     

     

     

  15. Posted

    Hi all,

     

    Say I have a socket server running, which uses SSL, now if a client connects, but does not send any data, it will take really long before SSL authentication fails, all this time the socket remains open.

    We can work around this by starting a timer when a socket connects, and if we don't have a successful SSL authentication before the timer goes we can close the connection, but is there any way to set such a timeout in ICS / OpenSSL self?
    The only thing I've found with regards to this in OpenSSL seems to be about how long a session stays valid, but not anything with regards to how long a client may take to SSL authenticate.

  16. Posted

    3 hours ago, TigerLilly said:

    1) May not be that easy, because you can only signal to stop. It ma ytalke a while for the async task to realize thios and react.

    2) Is preferred (by me), but not by looping (which will freeze the UI), but starting a timer (so the UI stays responsive) and this waiting until the task has stopped and closing the app is save.

    Starting a timer is basically the same as looping while calling Application.ProcessMessages(), and this is not really an option if you're in the process of stopping the application. I prefer waiting as well (the thread might be writing a file or whatever, which you want to wait for), but how to wait while not having a messageloop for the whole application?

  17. Posted

    Hi, I'm wondering what's the best way to approach a running async / await when the application is closed.

    I see two options:

    1) abort whatever is running async,

    2) wait for what is running async to finish. To wait for it to finish I probably need to have a message loop running (because Await needs to be called which goes through a message?), but I don't  want to run the message loop of the entire application (ie call Application.ProcessMessages()). Is there someway to call a message handler for Omnithread only?
     

  18. Posted · Edited by merijnb

    Hi,

    I'm using the latest release in Delphi 10.4.2When I'm using the open file expert, after opening a file focus changes to the application which had focus before Delphi was focused. This application is also set in front of Delphi.

    I've traced where in code this happens, it seems to be done by the call to ActionServices.OpenFile(FileName) done by GxOtaOpenFile.
    I'm not really into ToolsAPI and stuff, but is this still gexperts scope or is this a bug in the IDE?

      

    function GxOtaOpenFile(const FileName: string): Boolean;

var
  ActionServices: IOTAActionServices;
begin
  ActionServices := BorlandIDEServices as IOTAActionServices;
  Assert(Assigned(ActionServices));

  Result := ActionServices.OpenFile(FileName);
end;

    Can someone confirm they see the same behavior? The same code for me works fine in 10.3.

     

    This workaround seems to fix the issue:

      

    function GxOtaOpenFile(const FileName: string): Boolean;
var
  ActionServices: IOTAActionServices;
begin
  ActionServices := BorlandIDEServices as IOTAActionServices;
  Assert(Assigned(ActionServices));

  var h := GetForegroundWindow();
  Result := ActionServices.OpenFile(FileName);
  SetForegroundWindow(h)
end;

     

  19. Posted · Edited by merijnb

    I've made a patch for this, here a copy/paste from my comment on the issue on sourceforge.

     

    Quote

    I dug into this, it turns out that environment variables actually where unrolled, but couldn't cope with multiple paths in one environment variable. I've patched GX_OtaUtils to enable it to handle this. This solved the issue in both the Open File Expert as well as the Use Clause Manager, and I suspect it will solve the same issue in other places as well.

     

    I've attached the patched file on sourceforge.

    • Thanks 1

  20. Posted

    On 1/18/2021 at 10:01 AM, Primož Gabrijelčič said:

    Variant 1: Use a 'procedure' worker (`procedure Worker(const task: IOmniTask)'). Then you can implement your message loop inside.

     

    Variant 2: Use a `TOmniWorker` descendant (as in my example). It will implement its own message loop.

     

    Variant 2a: Override parts of the TOmniWorker message loop (but I actually don't remember anymore how that is done 😠 ).

     

    You can handle most stuff with Variant 2. In addition to timers you can call Task.RegisterWaitObject to connect code to any waitable handle. See demo 31_WaitableObjects for more information. Read more here: http://www.omnithreadlibrary.com/book/chap07.html#lowlevel-iomnitask-registerwaitobject.

    While playing around with this, I still run into something I don't quite get, please see this code:

      

    
uses
  OtlTaskControl;

type
  TWorker = class(TOmniWorker)
  protected
    fTimer: TTimer; 
  
    procedure OnTimer(Sender: TObject);
    function  Initialize: boolean; override;
  public
    procedure TimerCallback;
  end;

function TWorker.Initialize: boolean;
begin
  Result := inherited Initialize;
  if Result then
  begin
    Task.SetTimer(1, 100, TimerCallback);
	
    fTimer := TTimer.Create(nil);
    fTimer.OnTimer := OnTimer; 
    fTimer.Interval := 1000;
    fTimer.Enabled := true;
 end;
end;

procedure TWorker.TimerCallback;
begin
  // called every 100 ms
end;

procedure TWorker.OnTimer(Sender: TObject);
begin
  // called every 1000 ms
end;

var
  task: IOmniTaskControl;

task := CreateTask(TWorker.Create()).Run;

    In this case, the OnTimer event is fired as expected, so the message loop from TOmniWorker seems to be running fine. However, it only works as long as I also do Task.SetTimer, as soon as I remove that line - Task.SetTimer(1, 100, TimerCallback) -, my OnTimer event isn't fired anymore. It seems SetTimer does something needed to get the message loop going? What can I do to get this working without calling Task.SetTimer()?

    Thanks in advance!

  21. Posted · Edited by merijnb
    found what I did wrong :)

    On 1/16/2021 at 7:24 PM, Primož Gabrijelčič said:

    For a timer, call Task.SetTimer and timer event will be called automatically. 

     

    Something like this: 

    
uses
  OtlTaskControl;

type
  TWorker = class(TOmniWorker)
  protected
    function  Initialize: boolean; override;
  public
    procedure TimerCallback;
  end;

function TWorker.Initialize: boolean;
begin
  Result := inherited Initialize;
  if Result then
    Task.SetTimer(1, 100, TimerCallback);
end;

procedure TWorker.TimerCallback;
begin
  // called every 100 ms
end;

var
  task: IOmniTaskControl;

task := CreateTask(TWorker.Create()).Run;

     

    Hi Primoz,

     

    I've tried this exact code, but the TimerCallback isn't called.
    I also tried to create a normal timer in Initialize, but it's also not called. Is there something missing here which causes the message loop not to run or am I missing something else?

     

    Nvm: I found my brainfart 🙂

     

  23. Posted

    Hey all,

     

    I have some stuff which now works as a descendant from TThread. In the execute method there is a normal message loop.
    If I want to rework this to omni, how can I approach this?

    It boils down to a situation where I can create something like a timer in the omnithread task, then run a message loop and thus will get OnTimer events on scope of the tasks thread.

×