Angus Robertson

Thanks for your help, you've sent some updated C++ units already, look forward to the rest.

Angus

As has been mentioned once or twice, no-one supporting ICS does C++. 

So when I make changes, I try to keep C++ working, but without being able to test anything error occur.   

10,4 and later only use 'New' packages which have $auto suffiux so work with all new compilers, avoiding all this messing with versions.  So all those 110 references should be New. 

When you have it working, please zip and email me any changed or new files, so I can update SVN.  Probably best to discuss problems by email rather than here, my email is in many source files.

Angus

I often have two or three bds.exe running, but different versions, and all on the task bar.  The problem is when they are not obviously running...

Angus

Just had this IDE problem again, D11.3, edit a source file, unable to rename to history file. 

After having the problem with one file and fixing it by renaming the source file, had it with a second file, despite closing and reopening the IDE, three times. 

Finally discovered there was bds.exe instance still running holding open several source files, so I guess it did not close cleanly during one previous restart, yet allowed another instance to open. 

Angus

Sorry, skimmed the message...

Angus

ICS used to use self signed SSL/TLS certificates when there was nothing better, but issuing your own CA signed certificates is almost as easy and more friendly.

ICS has a sample that does it manually, or the servers do it automatically, there is also a function to install the ICS root CA into the Windows store for clients. 

Angus

Thank you for your Linux contribution @wright.

But how did you test it.  Running any ICS FMX samples in Linux using the WSocket unit just freezes the application when TIcsEventQueue is created, and I can not trace into it with the debugger.

I have made progress with Linux support for Ubuntu 22.04, OpenSSL now loads correctly, albeit Ubuntu has a two year old version 3.0.2 and seems not to want to upgrade to newer versions. 

I have a new PemTest sample that views and creates SSL/TLS certificates on Linux, not in SVN yet. 

Unfortunately, older Ubuntu releases don't include OpenSSL 3 so can not be used with ICS.

Angus

Component, Import Component wizard, Import Type Library. 

Angus

The best solution is for the web server to provide valid SSL certificates, Let's Encrypt for public hosts, your own CA for internal domains.   For your own CA, the root certificate needs to be installed on client devices, easy for Windows, less easy for mobiles. 

The ICS component library does all this automatically for ICS web servers, but can also generate certificates for other web servers.

Angus

REST server is just another name for a web server that support API type requests as well as web pages and media.  

Start with the sample OverbyteIcsSslMultiWebServ which is effectively a complete web server, and add code for your specific APIs and probably authentication.   Most servers are run as Windows services, so OverbyteIcsDDWebService is almost the same but can be run as a GUI or installed as a service. 

Both samples also include websockets which can be useful for simple API requests with less overhead than HTTP. 

Angus

It should only take a few minutes to update Midware to the latest ICS, read my notes carefully, fix the compiler errors and correct the uses in the packages.

Angus

Yes, there were massive changes in ICS V9.1 that meant many projects and derived components need updating, mostly relating to SSL/TLS, but also some unit split in two, and new package names for D10.4 and later, the important stuff is at:

https://wiki.overbyte.eu/wiki/index.php/Updating_projects_to_V9.1

but you should also read the release notes for V9.1 and readme9.

I only maintain and support ICS, Midware is supported by François Piette and I guess it has not been updated for V9.1, his web site says last release was in 2020. 

Angus

ICS V9.2 is now also available from GetIt for Delphi 11 and 12, thanks Embarcadero.

Angus

These problems all came down to a missing line in the configuration file, so the ICS proxy behaved as a TCP proxy instead of an HTTP proxy, and did not modify HTTP headings.

I'll look at writing a configuration GUI for the IcsHosts files, to make it all easier and self documenting, it's not obvious which settings relate to the several protocols IcsHosts servers support. 

Angus

Please take this problem to email, it seems specific to your application rather than a general ICS problem.  I've replied.

Angus

To sum up, you are unable to make the ICS proxy work with a specific web server or web broker, really no idea what that is.

How do you know this is an ICS problem?  Have you tried using the ICS proxy with other more modern web servers like Apache, as I have it, 

Your service example is undocumented, no setup instructions, I've no idea what is supposed to do, and I don't have time to debug it for you.

Angus

I've tested the ICS proxy as a reverse HTTP proxy for HTTPS<>HTTP, I've emailed you the URL, everything seems to work fine with a browser, with web pages of varying sizes.

The servicetest zip you posted seems to use web broker, never used that, and nothing to do with ICS.  I'm afraid I don't have the spare time to build, install and test such applications that are unconnected to ICS. 

Angus

I only managed to tick a few boxes, guess I've managed without all those tools for 25 years. My experience is the IDE is more stable to less third party stuff you install.

Angus

There are no simple solutions to supporting multiple cloud providers.

The free ICS component library has the HTTP REST and OAuth2 support you need, and has components to access cloud email at Google and MS, so should not be hard to add support for cloud storage, but someone will have to dedicate a few days to supporting and testing each cloud service. 

Angus

Each of these cloud providers has its own REST API to access storage services.  

You can sign up to each of them, read loads of API documentation and write code using various REST components for each one, and hope they don't change the API too often.

Or search GetIt for commercial components that have done all this hard work, and hope the authors keep them up to date as APIs change, TMS Software and /n Software come to mind, never used them myself. 

Angus

I'll be testing using my own public proxy server, remembered I do have one http server, svn.overbyte.be.  But I have some stuff to catch up with first.

Angus

Thanks, I'll look at reproducing it on my servers, main problem is all my web sites are https only and force http to https, so needs something set-up for testing.

Angus

To fix a bug I must be able to reproduce it.  Sure the logs sure some different numbers, but I've no idea what data or requests you are sending, no headers logged or anything useful for debugging. 

Forward proxy is intended to direct to another web site, such as this one that generates a lot of data, very useful for testing in a live environment, the internals of the proxy moving data are the same for forward and reverse HTTP proxy. 

Angus

ICS V9.2 has been released at: https://wiki.overbyte.eu/wiki/index.php/ICS_Download

ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10, 10.1, 10.2, 10.3, 10.4, 11 and 12 and C++ Builder 10.4, 11 and 12. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. Beware Mac OS-X and C++ have not been tested recently due to lack of support from such users.

The distribution zip includes the latest OpenSSL 3.0.14. 3.2.2 and 3.3.1, for Win32 and Win64.

Changes in ICS V9.2 include:

1 - V9.2 is a minor release, fixing a few issues introduced in the last major release, and other bugs located since. There are no breaking or installation changes from V9.1, but if updating from earlier releases please read https://wiki.overbyte.eu/wiki/index.php/Updating_projects_to_V9.1

2 - TIcsMailQueue can now queue a prepared EML file created by another application, or perhaps received by the SMTP Server.  Added optional SkipEmpty argument to StartMailQu method so queue is not started unless there are pending emails waiting to be sent. The sample has 'Send Prepared EML File' to queue an existing EML file rather then preparing email with properties.

3 - Improved email MIME decoding by supporting embedded boundaries, usually for multipart/alternative parts, within a multipart/mixed message, using code written 20 years ago but suppressed for some reason.  Previously these parts were sometimes left encoded within a part. There is a new property LooseRFC to allow decoding if the boundaries in the body are missing the two required hyphens, usually because the boundary also begins with hyphens. TMimeDecodeEx should now always return the body if no MIME parts are found, and TPartInfo has PLevel which is Part Level, and PInfo which is displayable part information for logs.  The MimeDDemo sample has various improvements to test these features.

4 - Fixed a nasty Win64 problem reading EC certificates from the Windows Store, which may have caused server crashes, also reproducible in the PemTool listing the Windows Store.  This was due to Win64 bad initialisation of a buffer used for a Crypto API call that failed.

5 - The HTTP client now checks the URL always has / at start of the path, ie add it for test.com?query.  In the REST client, added a sanity check for RawParameters to encode any spaces, which can break the HTTP request.  After a file download completed, check actual file size against response size. The multipart/form-data MimeBoundary no longer includes extra -- at start that are required preceding boundaries within parts, some web servers may have  been unable to decode our MIME encoding.

6 - The HTTP server has a new method AnswerRedirect for various redirection responses to a new URL. When accessing the default document in a path without a trailing path delimited /, redirect using 301 to the correct path with delimiter instead of adding it locally and displaying the document which will then incorrectly link to pages in a higher level directory. Using the THttpOption hoAddMissPath redirects if the default document is missing perhaps a template or virtual document. Fixed a bug where authenticated POST/PUT requests always returned a 404 error. Added AnswerBodyTB client response with TBytes binary, similar to AnswerString, tested in the sample by supporting favicon.ico request.   Check if the request HTTP version gets corrupted due to spaces in the URL, which are not allowed.  The SslMultiWebServ sample has new web pages to test POST/PUT and template authentication.

7 - TSslX509Certs has a new function CertResetDomain to reset a certificate order state to None, if the order process stalls or gets confused due to errors. If AcmeV2StartChallgs fails because there are no pending challenges, reset to order to None so it starts again next time and does not loop.

8 - ICS not longer tries to load OpenSSL RAND_screen function that may be missing from recent DLLs.

9 - TIcsHttpMulti fixes a bug introduced in V8.66 that stopped the application setting authentication, rather than adding it to the URL, and a Win64 free stream bug.

10 - TIcsIpStrmLog correctly counts failed client connection attempts if ping is not used first to check the remote IP address. The sample has a new client Retry Attempts box to test this.

11 - Updated the Snippets sample to use authentication to access some the hardcoded URLs, which started failing after authentication was added to test web server bugs (see above).

12 - Added OverbyteIcsHttpThrd sample to show how to use TSslHttpRest component in a multi-threaded program.

13 - Improved Posix support for Linux and Android, not tested or supported yet.  Beware SSL does not correctly load for Posix at the moment.

14 - Added support for a new feature release of OpenSSL 3.3 with {$DEFINE OpenSSL_33} in the Defs.inc file, ICS includes  new versions of the active versions, 3.3.1, 3.2.2 and 3.0.14, but no longer includes 3.1 since there are two newer feature versions.

15 - Updated the 'ICS Intermediate Short' SSL certificates, used by ICS to generate temporary server certificates to allow SSL servers to run until a Let's Encrypt or commercial certificate is installed. It now expires after 200 days, 21st December 2024, after which self signed certificates will be used instead, unless a newer 'short' is installed.

16 - Only Delphi 10.41  and 10.42 (10.4 with updates 1 or 2) will install correctly with the new install packages, the original RTM version does not support the package LIB suffix: $(Auto) so you must change it manually for each package to 21.0.

The release notes for V9.2 are at https://wiki.overbyte.eu/wiki/index.php/ICS_V9.2

All ICS active samples are available as prebuilt executables, to allow ease of testing without needing to install ICS and build them all. There are four separate zip files split into clients, servers, tools and miscellaneous samples which can be downloaded from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples

Angus

Nothing obvious missing, why did you create a certificate in the Windows store instead of a PEM/PFX that ICS would create for you?  I find it better to test with real IP addresses, your server won't start if there are any web servers running on 80/443. 

The two logs look the same except for a couple of values, not sure what they are supposed to show. 

All my mail travels through TIcsProxy, and I use the Forward Proxy on my public servers to access my local servers remotely, so the component is well tested, and rarely touched to break it

Angus