Jump to content

Angus Robertson

Members
 View Profile  See their activity

  • Content Count

    1731

  • Joined

  • Last visited

  • Days Won

    33

Posts posted by Angus Robertson

  2. Posted

    ICS is an event driven component, you don't appear to have any events, so you have no idea what happens after you start the connection attempt.  The logger is designed for debugging the internals of the components, not end user applications.    You should be reporting the onConnect event which tells you whether connection worked or failed, also onHandshakeDone.

     

    If you are using HTTPS, you should be using the TSslHttpRest which does not require an SslContext.  If this is a custom protocol, use TIcsIpStrmLog which doesn't need SslConrext either.  Both components are detailed in the Getting Started with ICS in readme8.txt, which explains the best components to use for new applications., also http://wiki.overbyte.eu/wiki/index.php/ICS_Getting_Started 

     

    Unless your application is using an SSL client  certificate (which is rare) you don't need a private key or certfile.

     

    Angus

     

     

     

  3. Posted

    Sorry, insufficient information to make any real diagnosis, you don't say what OpenSSL you are using, or which protocols, or what errors the client sees.  The client may have closed the connection and the problem is not server related. 

     

    SslHandshakeDone is called for both success and failure, you are not logging what happened. 

     

    I suggest you try configuring the OverbyteIcsSslMultiFtpServ.dpr sample with your certificate and IP address and try to connect to that, it has better logging.  Maybe use V8.66 as well.  The latest versions of ICS will start without SSL certificate, it creates a self signed certificate to start-up, and will then order a free certificate from Let';s Encrypt so you don't need to buy manually from RapidSSL.

     

    Angus

     

  7. Posted

    You don't say what response Gmail is providing when you send email

     

    Google Gmail accounts have two levels of security, historically they worked with clear authentication under SSL, and one of mine still does.  But if you choose a Google prompt to set 'higher' security on your account that means they will only support OAuth2 so the account password is not stored locally or sent in SMTP commands.  Instead the password is entered through a browser window.  

     

    So for the ICS mail queue component, specify authentication as XOAuth2 and the password field will be greyed.  When you start the mail queue, a browser window will appear in which you login with the same account as the user name here.   Google returns two tokens, one is used to access Gmail and remains valid for a short time, the second refresh token is saved by the sample as the password and can be used in the background to get a new access token without a browser window. 

     

    The refresh token usually remains valid for months, unless the account details are changed, but if you are sending unattended emails you need some scheme to allow interaction to login again if new tokens are needed.

     

    Before OAuth2 will work, you need a Google application account which is explained in notes at the top of OverbyteIcsSslHttpRest.pas. 

     

    Angus

     

     

     

  8. Posted

    I thought I fixed all the incompatibilities with Linux, I was certainly able to build packages for V8.65. 

     

    But I am not planning any further work to actually test it or make ICS work in Linux, unless someone pays me, I have no plans to write any Linux applications myself. 

     

    So it's up to those that actually want to use ICS with Linux to make it work, I will then update SVN (with complete source files, not patches). 

     

    Angus

     

  10. Posted

    Sorry, the internet hosting centre has been suffering a series of DDOS attacks since the weekend, usually only half an hour at a time before they block it, please just keep trying.  The servers themselves are fine, I can access them using my internet connection directly to the hosting centre. 

     

    Angus

     

  12. Posted

    Yes, paths are meant to be encoded within the path delimiters, not / itself, unless after ?  So I did the simple fix of only handling space.  About 20 links failed my own tester with full encoding.  So a new version is in SVN.  That Canadian site also broke the ICS proxy due to not supporting absolute URLs used by proxies, that's been on my list to fix for a year, so got done as well. Testing is always useful, gets me to fix things. 

     

    Angus

     

     

     

    • Like 1

  14. Posted

    You can check the original location header by keeping it in the onHeaderData event before the relocation actually happens. 

     

    I'm only auto URL encoding the redirection URL that the user can not change, not a URL passed to the component, that needs careful consideration.  Auto URL encoding is effectively what Firefox and Edge/Chrome do.

     

    < HTTP/1.1 302 Redirect
    < Content-Type: text/html; charset=UTF-8
    < Location: http://ec.gc.ca/Error 404.html

    < Content-Length: 153
    > GET /Error%20404.html HTTP/1.1

     

    Angus

     

  15. Posted

    I found the problem with HEAD and some redirections, it failed to start the redirected request until close was called, a bug that seems to have been there for many years.  Looked at my own link checker and I use GET not HEAD which is why I've never seen it.  Your 404 error page returns content even for 200 and HEAD.  It will be in SVN tonight.

     

    Angus

     

    • Like 1

  17. Posted

    I have investigated your URL and fixed it by correcting the bad relocation URL containing a space, at least for GET, but not yet for HEAD so another problem somewhere.  In my tests, the server closes the page as soon as a path with a space is found.  Testing with Firefox and Edge/Chrome suggests they correct the location path, only Edge displays it corrected. 

     

    Even after correcting the space the Error 404.html page is returned with a 200 response, despite the page saying HTTP Error 404 - Not Found in English and French, so certainly my link checker would assume the link was okay, I don't parse the page text, maybe you do?

     

    This also raises the issue of whether ICS should correct bad URLs, which browsers seem to do.  However, such correction is not trivial except for the simplest case of spaces, since we don't want to double encode / for instance. 

     

    Angus

     

     

     

  21. Posted

    I agree that UDP should send whole packets, but when they arrive two or more may be buffered before they are read in the DataAvailable event.  Also, they may not be sent as complete packets, for instance a record may be sent in one send/packet then a CRLF as the next send/packet, so if the application is waiting for that CRLF as a record separator, it needs two packets.  So best to treat UDP as a stream. 

     

    Angus

     

  22. Posted

    The most important issue about the DataAvailable event is not the size of your Receive/ReceiveFrom buffer, but that you should loop within the event continually reading all waiting data into a larger public receive buffer or stream until Receive/ReceiveFrom returns 0 or less.  If you don't do that, the event will be called again immediately you exit it to empty the internal receive buffers. 

     

    There is no guarantee about the length of data any call to Receive/ReceiveFrom will return even for UDP, it might take several events for a full packet to be assembled, rarely, but it can happen. 

     

    As I said before, all this is done for you in the OverbyteIcsIpStreamLog component

    .

    Angus

     

  23. Posted

    ICS V8.66 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download

     

    ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney, and C++ Builder 2006 to XE3, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. The distribution zip includes the latest OpenSSL 1.1.1k win32, with other versions of OpenSSL being available from the download page.

     

    Changes in ICS V8.66 include:

     

    1 - Added a new TIcsInetAlive component to check for IPv4 and/or IPv6 internet connectivity, using Ping and/or HTTP, defaulting to  www.msftconnecttest.com run by Microsoft for Windows 10 alive checking. The online and offline check intervals may be set, and event fires when online state changes.  Sample OverbyteIcsHttpRestTst has a demo for new component.

     

    2 - OpenSSL 1.0.2 and 1.1.0 ceased security fix support over 12 months ago so ICS now only supports 1.1.1, with 3.0 support due in the next few months. This removes a lot of legacy code and functions, and several old ICS functions needed for backward compatibility.  Two components only used for 1.0.2 have been removed, TSslStaticLock and TSslDynamicLock which may give an error loading forms, until they are removed.  SslContext SslOptions is now ignored, use SslOptions2 instead which has more modern options.  Can not remove SslOptions because it's saved on too many DFMs. Most modern applications should not be using SslConext, but IcsHosts for servers for high level client components like TSslHttpRest.

     

    3 - Undertook a major clean-up of OpenSSL functions that may require end user application changes if low level OpenSSL functions have been used, hopefully very rarely. All OpenSSL functions have been renamed to their original names removing ICS f_ prefix for commonality with other Delphi applications.  Renamed literals BIO_CTRL_PENDING_, CRYPTO_NUM_LOCKS_, CRYPTO_LOCK_, OPENSSL_VERSION_ with underscore to avoid conflicts with functions of the  same case sensitive names. Removed unit OverbyteIcsLibeayEx not used for a while. Most OpenSSL type pointers to Dummy array are now
    simple Pointer type. Changed most OpenSSL var parameters to pointers for commonality.  New TSslWSocket method SslOK is TLS/SSL negotiated OK. LoadVerifyLocations no longer lets OpenSSL use default CA folder locations which is potentially a security risk.

     

    4 - Bring server SSL/TLS security levels up to latest Mozilla recommendations. In 2021 Mozilla now recommends TLSv1.3 as modern ciphers and TLSv1.2/1.3 as Intermediate supporting all browsers from last five years, so IcsHosts now use sslCiphersMozillaSrvTLS12 as Intermediate level, also Mozilla recommends no cipher server preference so changed that. Updated SslSrvSecurity levels: sslSrvSecInter, sslSrvSecInterFS and sslSrvSecHigh now all the same TLSv1.2 or 1.3, sslSrvSecTls12Less now TLSv1.2 only, sslSrvSecSsl3 not supported, only sslSrvSecBack supports TLSv1 and 1.1, sslSrvSecTls13Only  unchanged TLSv1.3 only.

     

    5 - Added support for YuOpenSSL which provides OpenSSL in a pre-built DCU statically linked into applications, rather than using external OpenSSL DLLs. This make application distribution more reliable since it can not fail by users deleting the DLLs or copying incompatible versions into the directory.   YuOpenSSL is a commercial product from https://www.yunqa.de/ and is supplied as separate compiled DCUs for Delphi 5 to 10.4. DEFINE YuOpenSSL in Include\OverbyteIcsDefs.inc determines whether the DCU is linked or the external DLLs.  Note only one version of OpenSSL can be linked with  YuOpenSSL, whereas different DLLs can be supported.  Apart from setting the define and adding a path to YuOpenSSL.dcu, no other application code changes are needed unless you check or report the DLL directory, when GSSLStaticLinked can be checked to see if OpenSSL is statically linked. Also worth noting that YuOpenSSL provides access to the all OpenSSL APIs, while ICS only accesses those DLL APIs needed for functionality we support.   

     

    6 - Added a better way of configuring Socks proxy and HTTP Tunnel proxy settings in TWSocket, similarly to that added for THttpCli in V8.62. The new ProxyURL property sets SOCKS or HTTP Tunnel proxy settings using a single URL, ie proto://[user:password@]host:port where proto is socks5 or http. Also added new methods SetSocks and SetHTTPTunnel to set proxy properties in a single call. Socks proxy and HTTP tunnelling again support host names, broken in V8.56. The sample OverbyteIcsSocksTst has been updated to demonstrate use of the new ProxyURL property and also now supports HTTP
    Tunnel proxy.

     

    7 - TIcsRestEmail has new OAAuthType property and event for OAuth2 browser URL perhaps via email for servers. TIcsRestEmail now sets OAuth2 errors, and clears old tokens. Increased default timeout for TIcsRestEmail AOuth2 login to 120 seconds since several web page warnings may be seen while attempting to login through the web page.

     

    8 - TIcsHttpMulti has a new ParseLevels property that follow links on a parsed web page to lower level pages, which are also parsed, looking for files to download with a specific file extension, ie .avi.  It ignores parent links without a file extension, and external links. This allows the component to download files indexed using HTTP from CCTV camera memory cards.

     

    9 - In TSslHttpRest ensure SSL is initialised for non-REST HTTP requests like GET, POST and PUT.

     

    10 - In TSslWSocketServer, added better error handling when IcsHosts SslCert is blank, so SslContext is still created for automatically created self signed certificates.

     

    11 - In TSslWSocketServer, TSslHttpAppSrv and TSslHttpSrv, added new onBeforeCtxInit event which if set is called once before each IcsHosts SslContext is initialised to allow the context parameters to be adjusted for special ciphers or protocols.

     

    12 - In THttpCli, AddrResolvedStr is now still available after the connection closes to report the IP address used for the connection.  When using an HTTPS proxy with the CONNECT command don't send host and port which some servers are unable to handle, probably broken with V8.64.

     

    13 - In TIcsMailQueue, added a new method CheckOAuthLogins that should be called before the queue starts when using OAuth2 email logins, which allows interactively checking with a browser of any OAuth2 login credentials, avoids asking for login first time email sent maybe days later. 14 - In TSslX509Certs, When starting the local web server for file challenges,  check the server IP address still exists or change to 0.0.0.0, may not work if multiple IPs exist.

     

    15 - Updated to OpenSSL 1.1.1k with two high security fixes, one introduced in the last version relating to a certificate chain validation that ICS does not use, the other relating to servers using TLSv1.2 and renegotiation, and that feature is removed in V8.66.

     

     

    • Like 7
    • Thanks 3

  25. Posted

    You can use the OverbyteIcsUdpLstn sample to see how to receive UDP data.  Your won't have megabytes of data waiting when that event is called, in fact you will never know how much UDP data is being sent since it only arrives one packet at a time.  So typically you use ReceiveFrom to receive a maximum of say 4K, and then add that to a large receive buffer. 

     

    Much easier to use the new OverbyteIcsIpStreamLog component which does all this for you, look at the OverbyteIcsIpStmLogTst sample.

     

    Angus

     

×