Jump to content

merijnb

Members
  • Content Count

    28
  • Joined

  • Last visited

Everything posted by merijnb

  1. I'm currently in the situation where I'm using an active FTP client while setting a port range for the data connection. If a port range is given, the TCustomFtpCli.PortAsync() method in OverbyteIcsFtpCli will see which of the ports is available from given range, simply by trying to start a listening socket on the first port, go to the next if that excepts, etc. During tests we have seen the strange behavior that when we start multiple FTP connections in rapid succession, it sometimes happens that multiple clients pick the same port for the data connection (so for example client b picks the same port as client a, while client a is still using that port). It's a bit hard to replicate in a consistent way, but the smallest I've been able to make it is by making an application which creates three TFtpClient objects, let them all connection to the same server, set to active FTP with a port range (7000-7010) to upload a small file. This happens from a single thread. Something needed for this to happen seems timing related, I've not been able to reproduce this with an FTP server from ICS, but I did with Filezilla server. Obviously it shouldn't matter that much, since the port used for the data connection is chosen by the client, not the server. I'm quite at a loss how this can happen, for two reasons: The code in PortAsync() tries to setup a listening socket for each connection, apparently it's sometimes able to setup 2 different listening sockets on the same port, how is this possible? How is it possible that (besides there are two sockets listening on the same port) it's actually possible to get data to the two different listening sockets while they're listening on the same port, how does the tcp stack in Windows know for which of the two sockets the data should be sent to? I've added a zip file with logging from both the server and clients from my test, in it there is logging from filezilla server, a wireshark capture from the server's perspective, application log from the three clients (things like OnDisplayMessage from the TFtpClient), ICS log from all three clients (ICSLogger) and a wireshark capture from the client machine. Besides I'm really curious what I'm missing here I wonder if this is a bug in ICS code (how is it possible it's able to start two listening sockets on the same port) and if it's something I should be worried about. Thanks in advance, I'm really curious what comes out of this 🙂 ftp test log.zip
  2. I totally agree, unfortunately in this case I don't have any other option.
  3. If I see correctly ReuseAddr isn't used, that means that TFtpClient itself cannot be the cause of reusing a port, you agree? As I said, I have ways around this, this is more to satisfy my curiosity. When you say I'm not using the component properly, what do you mean exactly? It's now allowed to use multiple instances of TFtpClient at the same time? I understand there are ways around this, but is there a technical reason you say this?
  4. Apologies, I thought that was clear by now. Your description is about right, this application will at some point need to upload a file to an FTP server, at that point it will create an FTPClient object and start the upload. In a specific case active FTP with a specified port range is required, and then we ran into this interesting behavior. I don't think that will really solve the issue since (at least for me) it's not clear yet how this is happening in the first place. If I debug this I can see clearly that every time one of the clients tries to decide what port to use (at the moment PORT command should be sent), it will try to start listening on the first port in the range, if it fails (since already in use) it will try the next. The principle in code seems to be ok. Yet somehow there seems to be a gap between a socket calling Listen() and that port actually being unavailable for another socket to listen on, at least, that is the only way I can imagine two separate FTP client instances will give the same port to the server to make the data connection on. To me it seems there are two strange things about this: 1) how is it possible that two clients give the same port to the server for the active connection, that implies that two sockets were able to listen on the same port simultaneously which is not possible. 2) how is it possible the file transfers (seem) to work (see the wireshark logs). The only thing I can imagine is that FTP client 1 starts his data socket listening on port A, FTP client 2 for some reason (see point above) thinks he's also listening on port A, the server makes two connections back, both on port A, and they both actually connect to the listening socket from FTP client 1 (as far as I can see there is no check how many clients connect to the listening socket started by the FTP client). This kind of makes sense from socket perspective, but makes no sense at all from application perspective, that could never work (how would FTP client 1 be able to handle both data streams). Now I can work around this in several ways, so this question is more out of interest (I see a learning opportunity here), what the heck is going on here 🙂
  5. No, it's one application. Queuing is a possibility (or a fall back), but why? There is no technical reason to do so. Multiple files can be uploaded simultaneously without problems.
  6. I'm uploading files to an FTP server, these uploads are triggered from outside of my app and handled by a single thread. Each trigger will start up an FTP client to upload a single file.
  7. We've run into a bug with regards the FTP client in ICS, it boils down to the Listen() method of TCustomWSocket in OverbyteIcsWSocket.pas At the top of the method, fLastError should be set to 0: procedure TCustomWSocket.Listen; var iStatus : Integer; optval : Integer; optlen : Integer; mreq : ip_mreq; mreqv6 : TIpv6MReq; Success : Boolean; FriendlyMsg : String; ErrorCode : Integer; {$IFDEF MSWINDOWS} dwBufferInLen : DWORD; dwBufferOutLen : DWORD; dwDummy : DWORD; {$ENDIF} begin fLastError := 0; // <-- this should be added FriendlyMsg := ''; The reason this is needed is as follows: If you start an active FTP connection to a server, while using a data port range the client will try to setup a server for the data connection on port X, if this fails it will try again on X + 1 until it's able to start a server (or runs out of ports to try). This is found inTCustomFtpCli.PortAsync() in OverbyteIcsFtpCli: StartDataPort := DataPort; while TRUE do begin FDataSocket.Port := IntToStr(DataPort); try FDataSocket.Listen; break; { Found a free port } except if FDataSocket.LastError = WSAEADDRINUSE then begin DataPort := DataPort + 1; if DataPort > FDataPortRangeEnd then DataPort := FDataPortRangeStart; if DataPort = StartDataPort then begin HandleError('All ports in DataPortRange are in use'); Exit; end; end else begin HandleError('Data connection winsock bind failed - ' + GetWinsockErr(FDataSocket.LastError)); Exit; end; end; end; If you make a new FTP connection while one of the ports is already in use, but PortAsync() is able to find another port, the fLastError is still set to something <> 0 due to the earlier failed port. If you then end up in TCustomFtpCli.PutAsync(), it will still see that LastError and raise an exception over that while we're currently happily listening without problems. procedure TCustomFtpCli.PutAsync; begin DataSocket.LastError := 0; { V2.100 } HighLevelAsync(ftpPutAsync, [ftpFctPort, ftpFctPut]); if DataSocket.LastError <> 0 then { V2.100 } raise FtpException.Create('Socket Error - ' + GetWinsockErr(DataSocket.LastError)); { V2.100 } end;
  8. Hi, I'm wondering what's the best way to approach a running async / await when the application is closed. I see two options: 1) abort whatever is running async, 2) wait for what is running async to finish. To wait for it to finish I probably need to have a message loop running (because Await needs to be called which goes through a message?), but I don't want to run the message loop of the entire application (ie call Application.ProcessMessages()). Is there someway to call a message handler for Omnithread only?
  9. Hi all, Say I have a socket server running, which uses SSL, now if a client connects, but does not send any data, it will take really long before SSL authentication fails, all this time the socket remains open. We can work around this by starting a timer when a socket connects, and if we don't have a successful SSL authentication before the timer goes we can close the connection, but is there any way to set such a timeout in ICS / OpenSSL self? The only thing I've found with regards to this in OpenSSL seems to be about how long a session stays valid, but not anything with regards to how long a client may take to SSL authenticate.
  10. Starting a timer is basically the same as looping while calling Application.ProcessMessages(), and this is not really an option if you're in the process of stopping the application. I prefer waiting as well (the thread might be writing a file or whatever, which you want to wait for), but how to wait while not having a messageloop for the whole application?
  11. Hi, I'm using the latest release in Delphi 10.4.2When I'm using the open file expert, after opening a file focus changes to the application which had focus before Delphi was focused. This application is also set in front of Delphi. I've traced where in code this happens, it seems to be done by the call to ActionServices.OpenFile(FileName) done by GxOtaOpenFile. I'm not really into ToolsAPI and stuff, but is this still gexperts scope or is this a bug in the IDE? function GxOtaOpenFile(const FileName: string): Boolean; var ActionServices: IOTAActionServices; begin ActionServices := BorlandIDEServices as IOTAActionServices; Assert(Assigned(ActionServices)); Result := ActionServices.OpenFile(FileName); end; Can someone confirm they see the same behavior? The same code for me works fine in 10.3. This workaround seems to fix the issue: function GxOtaOpenFile(const FileName: string): Boolean; var ActionServices: IOTAActionServices; begin ActionServices := BorlandIDEServices as IOTAActionServices; Assert(Assigned(ActionServices)); var h := GetForegroundWindow(); Result := ActionServices.OpenFile(FileName); SetForegroundWindow(h) end;
  12. We have many projects, all with a huge list of search paths defined. Many of these search paths are identical. Once a path needs to be added to this, we need to update the search path on many projects, which is a pita. I'm now experimenting with putting all these paths in an environment variable, this way, I only need to update the environment variable, use that environment variable in the search path of the project and it's done. This all works well and looks like this: set MY_DELPHI_PATHS = 'lots and lots of paths' in Delphi set this in the search path of the project configuration: $(MY_DELPHI_PATHS) but unfortunately both the Open File expert and the Uses Clause Manager seem unable to cope with this. The uses clause manager shows the files in these search paths in the 'project' tab, but not in the 'search path' tab, the Open Files Expert doesn't show the files in these search paths anywhere. Can these editors be made environment variable aware? Thanks! I've just noticed the feature request list on sourceforge, so I've added this there as well
  13. I've made a patch for this, here a copy/paste from my comment on the issue on sourceforge. I've attached the patched file on sourceforge.
  14. Hey all, I have some stuff which now works as a descendant from TThread. In the execute method there is a normal message loop. If I want to rework this to omni, how can I approach this? It boils down to a situation where I can create something like a timer in the omnithread task, then run a message loop and thus will get OnTimer events on scope of the tasks thread.
  15. merijnb

    Messageloop in Omnithread task

    While playing around with this, I still run into something I don't quite get, please see this code: uses OtlTaskControl; type TWorker = class(TOmniWorker) protected fTimer: TTimer; procedure OnTimer(Sender: TObject); function Initialize: boolean; override; public procedure TimerCallback; end; function TWorker.Initialize: boolean; begin Result := inherited Initialize; if Result then begin Task.SetTimer(1, 100, TimerCallback); fTimer := TTimer.Create(nil); fTimer.OnTimer := OnTimer; fTimer.Interval := 1000; fTimer.Enabled := true; end; end; procedure TWorker.TimerCallback; begin // called every 100 ms end; procedure TWorker.OnTimer(Sender: TObject); begin // called every 1000 ms end; var task: IOmniTaskControl; task := CreateTask(TWorker.Create()).Run; In this case, the OnTimer event is fired as expected, so the message loop from TOmniWorker seems to be running fine. However, it only works as long as I also do Task.SetTimer, as soon as I remove that line - Task.SetTimer(1, 100, TimerCallback) -, my OnTimer event isn't fired anymore. It seems SetTimer does something needed to get the message loop going? What can I do to get this working without calling Task.SetTimer()? Thanks in advance!
  16. merijnb

    Messageloop in Omnithread task

    Hi Primoz, I've tried this exact code, but the TimerCallback isn't called. I also tried to create a normal timer in Initialize, but it's also not called. Is there something missing here which causes the message loop not to run or am I missing something else? Nvm: I found my brainfart 🙂
  17. merijnb

    Messageloop in Omnithread task

    Thanks Primoz, The timer was just an example though, I need a message loop in a thread for several async stuff which is going on there.
  18. In that feature request there is a link to this one: https://sourceforge.net/p/gexperts/bugs/148/ And that one says resolved 🙂 @dummzeuch need any more info on this?
  19. Is this still open source? I might take look myself if you can't find the time.
  20. note: We have seen these issues with multiple versions of the OpenSSL binaries, including the ones bundled with ICS 8.64. Please consider the following code snippet: var s := TSslContext.Create(nil); try s.SslCALines.Text := sslRootCACertsBundle(); try s.InitContext(); except on e: exception do ShowMessage(e.Message); end; finally s.Free(); end; This snippet works on most machines, but we have a few machines at client sites, where this code raises an exception: 'Error reading info file "Lines"'. (We haven't figured out yet why these specific machines have this issue). This exception is raised on OverbyteIcsWSocket.pas:14667 (ICS 8.64). InfoStack := PStack(f_PEM_X509_INFO_read_bio(InBIO, nil, nil, nil)); if not Assigned(InfoStack) then raise ESslContextException.CreateFmt('Error reading info file "%s"', [FileName]); Apparently the call to f_PEM_X509_INFO_read_bio() failed, but no information is give why, so we've added some calls to f_ERR_get_error() to see what is going wrong, and then it turns out the call to f_PEM_X509_INFO_read_bio() sets the error code 151580774 (get_header_and_data). To find out which of the hard coded certificates returned by sslRootCACertsBundle() were making this error happening, we've passed each of the certificates (sslRootCACerts001, sslRootCACerts002, etc) one by one to an TSSLContext to see which ones failed. This showed 2 interesting things: There is a 'typo' in sslRootCACerts009 and sslRootCACerts011: 'AGAT/3B+XxFNSRuzFVJ7yVTav52Vr2ua2J7p8eRDjeIRRDq/r72DQnNSi6q7pynP' + #13#10 + '9WQcCk3RvKqsnyrQ/39/2n3qse0wJcGE2jTSW3iDVuycNsMm4hH2Z0kdkquM++v/' + #13#10 + 'eu6FSqdQgPCnXEqULl8FmTxSQeDNtGPPAUO6nIPcj2A781q0tHuu2guQOHXvgR1m' + #13#10 + '0vdXcDazv/wor3ElhVsT/h5/WrQ8' + #13#10; sslRootCACerts010 = '-----END CERTIFICATE-----' + #13#10 + '# X509 SSL Certificate' + #13#10 + and 'S0Zj/gA0QHDBw7mh3aZw4gSzQbzpgJHqZJx64SIDqZxubw5lT2yHh17zbqD5daWb' + #13#10 + 'QOhTsiedSrnAdyGN/4fy3ryM7xfft0kL0fJuMAsaDk527RH89elWsn2/x20Kk4yl' + #13#10 + '0MC2Hb46TpSi125sC8KKfPog88Tk5c0NqMuRkrF8hey1FGlmDoLnzc7ILaZRfyHB' + #13#10 + 'NVOFBkpdn627G190' + #13#10; sslRootCACerts012 = '-----END CERTIFICATE-----' + #13#10 + '# X509 SSL Certificate' + #13#10 + (the END CERTIFICATE line is part of the wrong constant), which isn't a problem when you call sslRootCACertsBundle() since all constants are concatenated, but which shows up if you try to use these certificates separately. More interestingly, one these specific machines there seems to be an issue with sslRootCACerts033 and sslRootCACerts034. This is sslRootCACerts033: # X509 SSL Certificate # Subject Common Name: GlobalSign # Subject Organisation: GlobalSign # Subject Organisation Unit: GlobalSign Root CA - R2 # GlobalSign Root R2 SHA1 • RSA • 2048 # Issuer: Self Signed # Expires: 15/12/2021 -----BEGIN CERTIFICATE----- MIIDujCCAqKgAwIBAgILBAAAAAABD4Ym5g0wDQYJKoZIhvcNAQEFBQAwTDEgMB4G A1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjIxEzARBgNVBAoTCkdsb2JhbFNp Z24xEzARBgNVBAMTCkdsb2JhbFNpZ24wHhcNMDYxMjE1MDgwMDAwWhcNMjExMjE1 MDgwMDAwWjBMMSAwHgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEG A1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjCCASIwDQYJKoZI hvcNAQEBBQADggEPADCCAQoCggEBAKbPJA6+Lm8omUVCxKs+IVSbC9N/hHD6ErPL v4dfxn+G07IwXNb9rfF73OX4YJYJkhD10FPe+3t+c4isUoh7SqbKSaZeqKeMWhG8 eoLrvozps6yWJQeXSpkqBy+0Hne/ig+1AnwblrjFuTosvNYSuetZfeLQBoZfXklq tTleiDTsvHgMCJiEbKjNS7SgfQx5TfC4LcshytVsW33hoCmEofnTlEnLJGKRILzd C9XZzPnqJworc5HGnRusyMvo4KD0L5CLTfuwNhv2GXqF4G3yYROIXJ/gkwpRl4pa zq+r1feqCapgvdzZX99yqWATXgAByUr6P6TqBwMhAo6CygPCm48CAwEAAaOBnDCB mTAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUm+IH V2ccHsBqBt5ZtJot39wZhi4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDovL2NybC5n bG9iYWxzaWduLm5ldC9yb290LXIyLmNybDAfBgNVHSMEGDAWgBSb4gdXZxwewGoG 3lm0mi3f3BmGLjANBgkqhkiG9w0BAQUFAAOCAQEAmYFThxxol4aR7OBKuEQLq4Gs J0/WwbgcQ3izDJr86iw8bmEbTUsp9Z8FHSbBuOmDAGJFtqkIk7mpM0sYmsL4h4hO 291xNBrBVNpGP+DTKqttVCL1OmLNIG+6KYnX3ZHu01yiPqFbQfXf5WRDLenVOavS ot+3i9DAgBkcRcAtjOj4LaR0VknFBbVPFd5uRHg5h6h+u/N5GJG79G+dwfCMNYxd AfvDbbnvRG15RjF+Cv6pgsH/76tuIMRQyV+dTZsXjAzlAcmgQWpzU/qlULRuJQ/7 TBj0/VLZjmmx6BEP3ojY+x1J96relc8geMJgEtslQIxq/H5COEBkEveegeGTLg== -----END CERTIFICATE----- When processing this certificate on these specific machines OpenSSL gives this error, unless: - I remove one of the • in the 5th line from the top. (As long as there is only 1 • in the file it's ok, as soon as I add a 2nd • anywhere in the header the error returns) - I add a CRLF at the bottom of the file The same thing applies to sslRootCACerts034. I realize that this is probably a bug in OpenSSL itself, but it might be wise to remove the •'s from the hard coded certificates to work around this issue. Curious if anyone can shed some light on the cause of this 🙂 Merijn
  21. Or just convert it to ascii of unicode gives the issues
  22. I can confirm that this checkbox was checked, after unchecking and rebooting the problem is gone.
  23. I'm trying to receive UDP packets which are sent to a multicast group. I can see the packets being received in WireShark (dest ip = 225.255.255.255 dest port = 1120), and can receive the data using an example python script I found on the net, meaning that UDP multicast group and port are correct. I'm creating an UDP server like this: var UDPServer: TWSocketServer; begin UDPServer := TWSocketServer.Create(Self); UDPServer.Proto := 'udp'; UDPServer.Addr := '0.0.0.0'; UDPServer.Port := '1120'; UDPServer.MultiCast := true; UDPServer.ReuseAddr := true; UDPServer.MultiCastAddrStr := '225.255.255.255'; UDPServer.OnDataAvailable := OnUDPDataAvailable; UDPServer.Listen(); end; The OnUDPDataAvailable is never called, am I missing something here in setting the server up properly to listen for this group?
  24. merijnb

    UDP multicast issues

    The issue currently is not listening on multiple ports and / or addresses, receiving multicast data on a single ip and port doesnt' work either. It seems to be with the fact that you need to register with the kernel what unicast groups you want to listen to, otherwise the datagrams aren't even passed to your process.
  25. merijnb

    UDP multicast issues

    Hey Dave, I'm not going to switch from ICS to Indy, but with your experience with getting receiving multicast, can you see if there is anything wrong with the snippet from ICS above, or can you see if there is anything missing? Is calling setsocktopt with IP_ADD_MEMBERSHIP what needs to be done?
×