Angus Robertson

The IcsShellExec function moved units in V8.69, it is now in OAuth, but perhaps you have an old HttpRest unit with it still in? Or an old DCU. When installing a new version of ICS, it's always safest to use the Build Group pane to build all four versions of all packages, just wish it installed them as well. Angus

ICS has never been available from GitHub, someone must have stolen it and reposted it there without permission. We use a private SVN server, https://svn.overbyte.be/svn/ use the ICS branch. But I prefer to add patches using WinMerge comparing complete old and new units (no diffs), so please email complete units with your new code commented so I know why it is there. My email is in various ICS units and the readme. I suggest you look at the OverbyteIcsHttpProt,pas unit in SVN today, there were a many changes yesterday from another contributor. Angus

I've started converting ICS VCL sample applications to FMX, using VCLToFireMonkeyFormConvtr.exe from: http://www.innovasolutions.com.au/delphistuf/ADUGVCLtoFMXConv.htm or https://github.com/rogerinnova/Delphi-Firemonkey-Dunit It is very basic, search and replace using an INI file, not looked at the Github version, but the zip download was for XE5 and the INI file needed several changes to bring to cope with the latest FMX forms. I mapped TListView to TStringGrid, TRadioGroup to TListBox, TBitBtn to TButton, TComboBox to TComboEdit, TRichEdit to TMemo, TPageControl to TTabControl, and a few other things. TGrid would be better than TStringGrid, except it uses virtual data (which the help fails to mention) which needs a lot of rewriting. TRadioGroup is the biggest problem, visually you can do it, but there is no simple ItemIndex to use, so TListBox with or without a separate caption is easier. I can share my INI file is anyone wants it, but it's a work in progress, several more samples to convert. Angus

The wiki server hardware has been rebuilt recently so has been off-line on and off, but ICS can also be downloaded from my server at: https://www.magsys.co.uk/delphi/magics.asp Angus

ICS will shortly have a new websocket (WSS) client component. One of our long term users has kindly contributed code extracted from a working application and a new test application which works, it will take me a few days to check throughj the source and test it thoroughly. I will then also look at our existing websocket server which is very old code, and try to bring that up to date with SSL/TLS. Angus

Server sockets tend to only raise errors when they are processing data, not sitting idle listening. There are really only two solutions, send wake up data periodically to the server socket, or check a last received time stamp if you expect data every five minutes or something, or perhaps a combination of the two. Wake up testing can be added with a few lines of code using TSslHttpRest for HTTP or TIcsIpStrmLog for simple ASCII which I believe HL7 uses. I had a problem with one of my web servers not listening after restart and added this in half an hour. Then discovered I was incorrectly setting a firewall rule on startup. If TWSocket has stopped listening, you can try closing and re-opening, but it's safer to restart the application, which is easy with a Windows service, just set service properties to restart after five seconds, and stop it. My servers send an email when this happens so I know to check the logs for why, the OverbyteIcsDDWebService sample illustrates this. Angus

There are simple ways to download a file with zero error handling, and proper ways. Without error handling, unexpected things can happen. My PC crashed out of disk space a few weeks ago. Eventually I discovered the Sage accounting software had downloaded 11,000 copies of a tmp file, totalling over 400 gigs, over a couple of days, in background. No error handling and a lousy design, expected better from a company that charges hundreds of pounds to get a new software version that supports TLS/1.2. Angus

Originally it was considered allowable for native applications to display an embedded browser window in the application to capture the Authorization Code during redirect. But that potentially means the application can also capture the login as well so is no longer best practice, see RFC8252, and some apps will block the embedded window. The preferred authorization method is for the native application to launch the standard browser and redirect to localhost where a small web server runs to capture the Authorization Code. When OAuth2 was originally added to ICS, the only embedded browser available in Delphi was TWebBrowser using Internet Explorer, which Microsoft had announced was being removed from Windows and Google was ceasing to support. So ICS initially only supported the standard browser for authentication using a local web server. Since then Delphi 10.4 added the embedded TEdgeBrowser Chromium based browser and despite MSIE disappearing TWebBrowser still seems to work, so ICS now also supports both as Embedded Browsers, to provide a better user experience during authentication, with the window closing automatically and not needing a local web server (that may be blocked by a firewall). Earlier Delphi versions will support TWebBrowser but this no longer works with Google, so applications should still allow the standard browser to be used. Edge Chromium can be installed on Windows 7 and later. The form checks for Edge in the registry and for the WebView2Loader.dll, otherwise uses TWebBrowser. Officially the Microsoft.Web.WebView2 runtime (from GetIt) must be installed for Edge Chromium to work, but in practice copying WebView2Loader.dll into the same directory as the executable seems to work, there are Win32 and Win64 versions of this DLL with the same name, you need the correct version for the build! SVN and the overnight zip have a new ICS beta with the new window, it is currently only supported for Delphi 10.4 and 11, VCL only, later betas will add TWebBrowser for old Delphi versions and FMX. Any feedback on the new window cosmetics and it's operation would be appreciated before this is finally released. All three SSL samples for sending and receiving email have been updated with the new window. Angus

Don't often look at map files, but you are correct, the TRestOAuth object and new OAuthForm are being dragged into the ICS web server sample. I don't believe OAuth is referenced anywhere, but all units using TSslSocketServer bring in TSimpleWebSrv to handle automatic SSL/TLS certificate ordering, and that that is in the same unit as OAuth. Unless you un-define AUTO_X509_CERTS. I've been under the impression for 20 years that Delphi only linked in objects that are actually referenced and ignores unused ones in shared units, but perhaps that has changed? And maybe that is why applications built with new versions of Delphi become bloated in size. If linking is broken, I could move TSimpleWebSrv back to the REST unit (where OAuth used to be) or to a separate unit. I try to avoid lots of small units to simplify maintenance and needing to change over 100 runtime packages each time I add a new unit, but that has to happen for the next ICS release to add the new OAuth forms. Update: looking at some code, the OAuth stuff is being brought in to support automatic SSL/TLS certificate ordering from a German company CertCentre who sell Sectigo and DigiCert certificates, whereas most people probably only need Let's Encrypt. I'll look at making CertCentre conditional. Angus

The OAuth unit uses TSimpleWebSrv which is a minimal overhead web server, not THttpServer. Angus

Looking into the strange host name ws.twelvedata .com posted here, in UTF-8 it is \x77\x73\x2E\x74\x77\x65\x6C\x76\x65\x64\x61\x74\x61\xEF\xBB\xBF\x2E\x63\x6F\x6D which includes a special symbol before the second dot, in UTF-8 xEF\xBB\xBF\ or #65279, which is the non-printing reserved Unicode symbol range. So you can not see it, but it copies and converts into an international domain name (like Chinese). Potentially a risk for phishing, but no idea how it got into the original wss URL. Angus

I suspect your URL contains non-ASCII characters that look like ASCII, often used by phishers to hide fake URLs, but can happen for other reasons. When I first tried ws.twelvedata.com ICS gave the error 'Cannot convert host address 'ws.xn--twelvedata-lb99c.com' - Host not found ' (as your screen short) which means the host name contains illegal characters and has been translated into ASCII, note the xn- part which is the giveaway. When I retyped it as ws.twelvedata.com I can connect okay on port 80. Angus

If you don't want to update your code to support OpenSSL 3, then build an older version instead. These functions have not been used for years. Angus

While ICS will connect to a Websockets server using simple TCP, ICS does not have a Websockets client, only a Websockets server, So you are on your own if any handshaking is required, sorry never looked at the WSS protocol. Angus

Sorry, VCLZipPack seems to slipped back into OverbyteIcsD2007Run.dpr/dproj with the changes to remove it in V8.70 from more recent compilers. Just remove it and the package should rebuild, You may have to also fix a missing variable in another old unit. I've now disabled VclZip from my master copy since I'm no longer using it now ICS supports TZipFile, so this should not happen again... These changes are now in SVN. Angus

THttpCli and TSslHttpRest have a property RcvdHeader: TStrings which after a request completes has all the received headers, so just search it for your X headers. Angus

I've changed LmCompatLevel from public to published, so it becomes more obvious. The notes suggest setting it from the registry, but the LSA\LMCompatibilityLevel key doesn't exist on my Windows 11 PC, so no idea how you find the settings. Angus

But the function you changed also builds the time zone of emails and HTTP headers, where a colon is breaking, probably, Unfortunately I do break ICS occasionally, but take a lot of care to avoid doing so. Angus

I will be looking at the time zone issue, but fixing it in a non-breaking way. Angus

Sorry, I don't have the time to enhance your code, unless there is a specific application for it. Nor do I change anything for which there are no tests. Angus

Date/time handling in ISuperObject was lacking, so I added it myself, from the changes at the top of the OverbyteIcsSuperObject unit: Added new datetime get/set type for Delphi TDateTime, saves as ISO 8601/RFC3339 string: obj.AsDateTime, obj.AsObject.DT['foo'], obj.DT['foo'], obj.AsArray.DT[0] I also improved the parser to return error information upon failure instead of a nil object. Angus

OK, YuOpenSSL does solve the DLL problem, although your application installer should ensure it's not really a problem anyway. I'm aware there are trial and commercial versions of YuOpenSSL, and that the trial version of other components expire in spectacular manner, so I would check the trial lasts more than a few weeks, and you can get new trial versions with security fixes. Angus

The latest OpenSSL DLL files can be downloaded from the same site you found ICS V8.70, they are also included in that zip, in the Samples\Delphi\SslInternet\ directory and in OpenSSL-Win32\. You need to copy the DLLs into the same directory as your EXE and leave GSSL_DLL_DIR blank, no idea what path you are trying to set. YuOpenSSL is an alternative commercial product that avoids needing DLLs, you don't need it to use ICS. Angus

Your solution may be easier than a pool, but it does not seem to work. ICS was not designed for cross thread use, use of threads within ICS is very rare, only for long ZLIB operations and a background mail queue. Angus

So effectively you want to use an external pool of TSslHttpCli objects from within your threads, rather than creating them as needed within the threads? So why use TSslHttpCli in the thread, why not just use one of the pool objects asynchronously, waiting in the thread for a semaphore to be set on competition? Windows actually creates a thread for async winsock operations, so there is no reason to use TSslHttpCli in the thread with all the messy stuff that goes with it. One of the ICS samples uses a pool of components and a queue to download all the elements on a web page. Angus