Jump to content

Angus Robertson

Members
 View Profile  See their activity

  • Content Count

    2069

  • Joined

  • Last visited

  • Days Won

    38

Posts posted by Angus Robertson

  1. Posted

    Please stop sending your comments as private messages as well, I do read this forum, when I'm in the office.

     

    Why do you specifically want to use OnDataSendTar and OnDataRevcTar events, they are very low level.

     

    If you want to modify headers and/or body, you should be using onHttpReqBody, onHttpRespBody, onHttpReqHdr, onHttpRespHdr, which have a simple String property you can update.  

     

    If you change the body length, you map also need to change header fields.

     

    Angus

     

  2. Posted

    Also, our recent OpenSSL DLLs no longer work on Windows XP either, and ICS has removed support for unsupported OpenSSL versions that might still work on XP.   We also digitally sign the OpenSSL DLLs and older versions of Windows XP do not recognise the root certificate used today. If you want to support the latest security standards, you need Windows 10.

     

    Angus

     

  3. Posted

    530 5.7.0 Must issue a STARTTLS command first. - this simply means the server requires an SSL/TLS connection, and you have not sent the command to start it.  You are using the old sample that does not support SSL, you should be using OverbyteIcsSslMailSnd.dpr or OverbyteIcsMailQuTst.dpr (but that may not be in old versions).

     

    The ncrypt.dll missing error means new versions of ICS are no longer supported on Windows XP, sorry.  I recently added some functions to access SSL certificates and private keys that needed newer Windows APIs not in Windows XP.  ICS V8.66 is probably the last that supported Windows XP, I'll update the documentation.

     

    Angus

  4. Posted

    Since XP has been obsolete for several years, we don't test ICS against it, only Windows 7 and later, and that will be dropped soon.  But there shouldn't be anything specific in ICS to stop it installing. 

     

    What specific errors did you get with V8.58, and with V8.68?

     

    Angus

     

  6. Posted

    ICS V8.68 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0 and C++ Builder 2006 to XE3, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. The distribution zip includes the latest OpenSSL 3.0.1 win32, with other versions of OpenSSL being available from the download page.


    Major Changes in ICS V8.68 include:

     

    1 - V8.68 is a minor release, mainly HTTP client and server improvements including new request and response headers to assist browser caching and conditional requests, improved error reporting to help diagnose failed HTTP requests, and improvements in the HTTP REST component allowing it to save files including resuming failed downloads and download files of any size.  There are minor fixes in various components, and updates to installing on MacOS and C++ Builder. There is also a new File Clean-Up demo ideal for deleting the old log files that many applications (including some ICS samples) leave on systems.

     

    2 -The HTTP client TSslHttpCli now keeps the Etag response header as ResponseEtag, allowing applications to save it with the page content, and when requesting a refresh to add the request method ReqIfNoneMatch or ReqIfMatch to avoid downloading the page again. Improved RequestDoneError to give more information than a simple abort, added httperrOutOfMemory and httperrBgException which happen while receiving and processing data in the OnDocData event and httperrSslHandShake. For range downloads, added the ContentIfRange request header which can send an RFC1123 date or Etag so a partial download only happens if the file is unchanged.

     

    3 - Previously, the HTTPS REST client TSslHttpRest always downloaded content to a TMemoryStream with content size being limited to MaxBodySize (default 100 MByte), and generally restricted by memory to less than 250 MByte.  To remove this limitation, added the HttpMemStrategy property with THttpMemStrategy on how to handle downloads: HttpStratMem only TMemoryStream; HttpStratTemp uses a work file in the system temporary directory for sizes larger than MaxBodySize; HttpStratFile always writes a named file HttpDownFileName (with .part extension during download); HttpStratResume is similar to HttpStratFile but supports resume of failed partial downloads (with .http extension for resume information). Property ResumeMinSize defines the minimum sized partial file that should be resumed, rather than start again (default 64K). Added ShowProgress property that causes download information to be sent to the OnHttpRestProg event.

     

    4 -The HTTP server TSslHttpServer now supports the If-Range, If-Match and If-None-Match request headers for conditional pages using Etag or last modified date, and sends 304 not modified for  matches, to help with caching.  Added an ETag header to responses in AnswerStream and AnswerPage when we can create one from a file modification date and size (base64 CRC32), or if the EntityTag property is specified in the client onGetDocument event before using hgSendDoc or hgSendStream, perhaps a CRC32 of the entire content from a cache. The 304 not modified response now includes more recommended headers. Also add a Date: header to AnswerStream and AnswerPage responses to help with caching.

     

    5 - The Multi HTTPS client TIcsHttpMulti has been updated to a fix a problem that meant large files failed download with only an abort error if too large for TMemoryStream, now downloaded to TFileStream with .part extension and renamed up successful completion.

     

    6 - The previous ICS release added support for the OpenSSL 3.0 release. Now it's been available for three months it has been added to the main distribution, the samples SslInternet directory now has both OpenSSL 1.1.1m and 3.0.1. ICS now supports YuOpenSSL 3.0 and 1.1.1 versions as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs.

     

    7 - In TIcsFileCopy, the DeleteFiles method now supports a new Zipped property so files are zipped with .zip extension before being deleted, useful for cleaning up old logs to save space (requires VCLZip).  The OverbyteIcsXferTst,dpr sample has a new File Clean-Up tab to test the multiple DeleteFiles method, it allow files older than x days or a specific date range to be archived/zipped (to save space) or deleted, ideal for deleting the old log files that many applications (including some ICS samples) leave on systems.

     

    8 - In the FTP client TSslFtpClient, using the PORT command to set Active mode now prevents other FTP clients sharing the same port number and address, usually only a problem with a small port pool (which is not recommended).

     

    9 - Updated the trusted root certificate bundle files, lots of changes from Microsoft since June, Google is now issuing it's own certificates. Updated the build-in sslRootCACertsBundle, few gone, now total 59 certificates commonly used.

     

    10 - ICS should now build with C++ for RAD Studio 10.4 and 11.0, fixed the 11.0 packages and various Windows API related units, including for  Win64.

     

    More detailed release notes are at http://wiki.overbyte.eu/wiki/index.php/ICS_V8.68

     

    • Like 2
    • Thanks 3

  7. Posted

    Did a quick Google search for the error, seems the developer has messing with this 'security feature' for years causing a lot of problems, and recently with TLSv1.3  that changed how sessions are created. and in fixing 1.3 probably broke 1.2.  What is really needed is the tick box to turn off the feature, which seems to have gone. 

     

    I'm removing the port from the session cache name anyway, since conceptually it is wrong even if it does not fix this particular bug, it should save one TLS session set-up.

     

    Angus

     

  9. Posted

    Okay, I can reproduce it by forcing ICS to use TLSv1.2.  So not sure if this is really a FileZilla bug given it works on the older versions that did not support TLSv1.3. 

     

    The old version also has a configuration option 'require TLS session resumption of data connection when using PROT P' which sound like the error message, but I have that ticked on the old beta, and that setting and lots of other interesting and useful ones have disappeared from the new version. 

     

    Seems like users should go back to the reliable beta rather than the 'release' version <g>

     

    Angus

  10. Posted

    I can find no problem accessing my own FileZilla servers.  My hosted server had v0.9.60 beta from a year ago:

     

    > PASV
    < 227 Entering Passive Mode (217,146,102,143,82,95)
    ! Passive connection requested to: 217.146.102.143:21087, control channel: 217.146.102.143
    > MLSD /webapps/telerest/templates/testing/
    Check for Old SSL Session
    Old SSL Session Found Cached
    < 150 Opening data channel for directory listing of "/webapps/telerest/templates/testing"
    ! SSL Connected OK with TLSv1.2, cipher ECDHE-ECDSA-AES256-GCM-SHA384, key auth ECDSA, key exchange ECDH, encryption AESGCM(256), message auth AEAD
    filezilla.ftptest.org SSL Connected OK with TLSv1.2, cipher ECDHE-ECDSA-AES256-GCM-SHA384, key auth ECDSA, key exchange ECDH, encryption AESGCM(256), message auth AEAD
    < 226 Successfully transferred "/webapps/telerest/templates/testing"

     

    It seems after a decade of beta releases, v1 finally came out this summer, so I installed v1.2.0 on my hosted server, unfortunately Windows Firewall blocks it, despite it being added manually, so installed it locally, and it also works. 

    15:05:41:693 > PASV
    15:05:41:693 Starting SSL Session
    15:05:41:693 Cache SSL Session: New
    15:05:41:693 < 227 Entering Passive Mode (192,168,1,105,251,19)
    15:05:41:693 ! Passive connection requested to: 192.168.1.105:64275, control channel: 192.168.1.105
    15:05:41:693 > LIST
    15:05:41:694 Check for Old SSL Session
    15:05:41:695 Old SSL Session Found Cached
    15:05:41:695 < 150 Starting data transfer.
    15:05:41:697 ! SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD
    15:05:41:697 pc21-web5.magenta SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD

     

    So no idea why you are seeing error 425, is there something more useful in the FileZilla server log? 

     

    If this is something to do with re-using SSL sessions, when ICS caches a session it adds the port number to the IP address when saving it, to prevent different services being accessed by the same session, but this is effectively what happens with the FTP data channel.  So in TIcsFtpMulti you could try removing  FtpCli.ControlSocket.PeerPort from xxNewSession and xxGetSession and see that improves matters.  I'm not going to change this until I find out how FileZilla is configured to cause the error.

     

    Angus

     

     

  14. Posted

    You should never use TDateTime for duration calculations, users can change the system time, and summer time saving changes it twice a year (unless you use UTC time). 

     

    Always use the difference between two GetTickCount64 Int64 values.

     

    Angus

     

     

     

    • Like 1
    • Thanks 1

  16. Posted

    Can you please try and install the latest ICS from SVN or the overnight zip, we've made various C++ package changes (CPP 11.0 only) this week that should resolve the lib files not being updated, and fixes some CPP warnings.   

     

    Same applies for any other CPP users, V8.68 is finished and will be released next week, so now is the time to test it installs correctly. 

     

    V8.68 is a minor release, mainly install problems, added OpenSSL 3.0.1, and support for new HTTP request and response methods to help caching (Entity Tags), the REST component will now download files of any size, including resuming failed partial downloads, and HTTP error reporting is improved, as illustrated in the  previous post here that previously would have said just Abort without any explanation. 

     

    Angus

     

  17. Posted

    Probably unrelated, but just been reading an article in PC Pro magazine about the new Intel 12th generation processors, which have two types of core, performance and efficiency, something ARM has had for a while. 

     

    Only Windows 11 has the extra code to receive telemetry from the Intel Thread Director in the CPU to negotiate on which cores processes should run.  Windows 10 application performance may be more random, and different each time you run it.  Which is slightly frightening. 

     

    So just saying modern CPUs may affect applications in ways you have not considered. VMs are even worse. 

     

    Angus

     

  19. Posted

    BTW, for C++ you should be using the early version of V8.68 from SVN or the overnight zip (same wiki page as the stable version), I fixed a load of C++ package and sample issues back in October so that 10.4 and 11.0  build again, including the ones you mention, sorry only just remembered. 

     

    Angus

     

     

  20. Posted

    Sorry, I don't support C++, I just distribute files updated by other ICS C++ users.  Hopefully one of them will be along shortly to help.

     

    OverbyteIcsSslThrdLock.pas has long gone, just remove any reference to it.  ICS builds with Delphi 11.0 without any warning, if C++ gives warnings just ignore them.

     

    Angus

     

  21. Posted

    All that happens internally within the ICS TRestOAuth component, it holds the expiry date so knows when to refresh the access token. 

     

    Getting a new refresh token has various options since many applications are used unattended, not just Windows services, so it will notify an administrator that a new OAuth2 login us required, but API access will fail until it happens.  It is unfortunate that OAuth2 was designed without a refresh expiry date being known, so things could be planned better.

     

    Angus

     

  22. Posted

    Provided you have set the RefrToken or RefreshToken property, you won't see a login window. 

     

    It;'s expiry is unknown to the application, but can happen if the account secrets are changed, if you invalidate it through the online console, or at the whim of Google or Microsoft.  So applications need to handle token failure, as I mentioned a couple of days ago. 

     

    The same refresh token can be used on multiple computers, at least for Gmail which is the service I use regularly as a backup when my own SMTP server is down.   This why the ICS MailQueue component handles multiple email servers, with OAuth2.

     

    Angus

  23. Posted

    I think you are confusing the two tokens that OAuth2 should return. 

     

    In ICS, AccToken property is the short lived token used for HTTP requests.  The RefreshToken property is a long lived token that you should store safely like a password, and may be used repeatedly by the component to refresh AccToken, in background without any interaction.  I tested GMail yesterday, and the saved refresh token meant it just worked without a new login. 

     

    So you need to check if Azure is returning a refresh token and whether you are saving it for re-use.   You may need to provide specific scope settings to get a refresh token, see the constant OAuthUriMSRest. 

     

    Angus

     

  24. Posted

    Quote

    I wonder what prevents them to make it FMX compatible right away, on all platforms, I expect not much VCL code inside ?

    The ICS packages all build okay with FMX and YuOpenSSL, why do you think it is incompatible?  Or at least only with lots of deprecated warnings from OpenSSL, that we never see when using the DLLs.  Just built one of the FMX samples and that works fine, once I'd commented out a couple of old lines.  Need to update those old samples.

     

    Angus

     

  25. Posted

    OpenSSL has released quarterly updates for the two supported branches, 3.0.1 and 1.1.1m, Windows binaries are available from

    http://wiki.overbyte.eu/wiki/index.php/ICS_Download or https://www.magsys.co.uk/delphi/magics.asp .

     

    OpenSSL 3.0.1 fixes a medium security risk relating to clients verifying X509 certificates from the server, a malicious server could potentially send a bad certificate that caused the client to hang or misbehave during verify.  https://www.openssl.org/news/secadv/20211214.txt

     

    Now OpenSSL 3.0 has been available for three months, updated the main supported OpenSSL release to 3.0.1.  The samples SslInternet directory now has both OpenSSL 1.1.1m and 3.0.1, ICS will try and load OpenSSL 3.0 first, then 1.1.1 if not found, unless the global variable GSSLEAY_DLL_IgnoreNew is set true before OpenSSL is loaded. Likewise GSSLEAY_DLL_IgnoreOld may be set true to ignore 1.1.1 and fail unless 3.0 is available. This is available from SVN and the overnight zip. 

     

    Note the binaries are now digitally signed by 'Magenta Systems Ltd' instead of 'Open Source Developer, François PIETTE' due to the massive cost of renewing the open source certificate.  Developers can always resign the DLLs with their own signing certificate to remove the Magenta name.

     

    Separately YuOpenSSL has released both these versions as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs.

     

    Angus

     

    • Thanks 3
×