Angus Robertson

Can not recall what I said in my previous posts here, but can only reiterate that for serving files, threads are not necessary, there is minimal blocking activity.  There is a limit to how many new SSL/TLS connections can be accepted each second due to the processing, but once the connection is open, hundreds of connections can run in a single thread. 

Threads are necessary to make use of more than one CPU.   ICS does have a threaded server component, but it does not support SSL/TLS or the web server and has not been tested for 10 years. 

If you use threads in ICS, each thread has its own message queue and handler, so you can not send from a different thread.

Angus

As I wrote, ICS decodes your file correctly into two parts, how your application is supposed to understand such an encoded attachment is a different issue. 

Angus

The ICS components are event driven, so have various state variables for control, they will need careful manipulation if opening a connection is skipped.  Basically, test, fix, repeat.

Angus

It would be possible to assign a socket to THttpCli and then call SocketSessionConnected that triggers the first request, perhaps in a descended component. 

But testing it without your clever device would be painful, it would need a modified web server that made the original connection instead of accepting it, which is where we cross over into proxies.

Angus

If you are listening for a connection on port 80, that is a web server, not sure why you then respond with a web command.  Or is the embedded device trying to connect to a proxy server? 

ICS has a proxy component, also TSimpleWebServer and TIcsIpStrmLog that will listen for connections.  But none will pass a connection to THttpCli.

Angus

I used the OverbyteIcsMimeDemo sample for your EML file, Decode File Extended button which uses TMimeDecodeEx.DecodeFileEx, it seems to work OK:

MIME-Version: 1.0
Content-Type: application/octet-stream; name="image.jpg"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="image.jpg"
Part 0, Content: application/octet-stream, Size: 0, Name: image.jpg, FileName: image.jpg, Encoding: base64, Charset: us-ascii, ApplType: , Content Id: , Reg Extn:
Part 1, Content: text/plain, Size: 41, Name: , FileName: , Encoding: , Charset: UTF-8, ApplType: , Content Id: , Reg Extn: .txt
Part 2, Content: image/jpeg, Size: 32563, Name: image.jpg, FileName: image.jpg, Encoding: base64, Charset: , ApplType: , Content Id: , Reg Extn: .jpg

That sample does not write parts to files, so I can not view the image easily.  I tried the same EML file on two online web sites,, and both failed to decode the file, one created an 46K image file containing the raw MIME content. 

If you want me to look at the other issues, I'll need similar files to run through the MimeDemo sample to reproduce the errors.

Angus

Quote

Well, Midware is much, much more than a simple stream.

I appreciate that, but some of the low level components like TAppSrvClient could be massively simplified if descended from newer ICS components, rather than being complicated to add SSL/TLS.

Angus

Looking at the Midware client code, it is very similar to the ICS TIcsIpStrmLog component that can be configured as a client or server, to send textual or binary information between instances, look at the OverbyteIcsIpStmLogTst sample.

Midware has some special processing escaping binary that would need adding, but TIcsIpStrmLog could replace much of the low level stuff while adding SSL/TLS.

Angus

I will test your file with the MIME sample application next week.   Not looked at the code for a long time, it just works, tested with 40GB files last year.

Angus

The first question is are all your examples correctly encoded according to the RFCs, and decoded correctly by other MIME implementations.

ICS can not be expected to handle non-compliant MIME. 

Looking at your test file, the MIME has 995 long lines which is allowed in the 30 year old RFC, but 76 is rather more common and recommended.  Not sure if long lines are supported.

Angus

The StatusCode is a web server response, although sometimes it is generated by the client if the server can not be reached for any reason, there is no official code for DNS failure.

You should also be checking LastResponse, ReasonPhase, RequestDoneError and RequestDoneErrorStr for more information after a request.

Angus

ICS V9.1 has major structural changes, hundreds of files are disappearing.  You should read the svn log in the zip for full details.  The release is almost finished and the readme will be done then. 

The relevant SVN comment:

If you are using Delphi 10.4 or later, all existing version specific install groups and packages have gone, since that version of Delphi added a new package $auto suffix allowing the same packages to be used for future Delphi versions. So uninstall all ICS packages, and use one of the new install groups, IcsInstallFmx, IcsInstallVcl, IcsInstallVclFmx or CBIcsInstallVclFmx.  These use packages IcsCommonNewRun/Design, IcsVclNewRun/Design and IcsFmxNewRun/Design, and build with the compiler version instead of a Delphi version, i.e. IcsCommonNewRun290.bpl.

Angus

OpenSSL has released new versions of the three active versions, 3.2.1, 3.1.5 and 3.0.13 which have three low priority security fixes.

Windows binaries are available in SVN and the overnight zip file and separately from https://wiki.overbyte.eu/wiki/index.php/ICS_Download or https://www.magsys.co.uk/delphi/magics.asp

In addition to the three DLL files, the zips include compiled RES resource files that contain the same DLLs, text files and version information, see the RC file. The RES file may be linked into application EXE files and code then used to extract the DLLs from the resource to a temporary directory to avoid distributing them separately.

ICS V9.1 and later optionally support loading the resource file, currently in SVN and the overnight zip. Beware V9.1 has a lot of other changes that may need application changes, please read the SVN change log very carefully.  There will be a lot of new documentation about V9.1 over the next two weeks.

Separately, YuOpenSSL has released 3.0.13 as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs.

Angus

35 minutes ago, Silver Black said:

My CE works perfectly, it's just GetIt that it's unreachable, but the IDE works

Because you installed from an ISO image or before the servers went down, GetIt servers provide much of the platform installation, Delphi, C++. etc.  Had the same problem myself.

Angus

37 minutes ago, dummzeuch said:

Until recently Japan used floppy disks in large numbers for submitting some documents

https://www.theregister.com/2024/01/29/japan_government_floppy_disks/

https://www.floppydisk.com/ has bought up large stocks of floppies for those really in need.   I used to carry boxes of 8in and 5in floppies around the world.

Angus

I maintain and use DDService, but not Eurekalog. 

I've been using madExecept for many years, but only for logging errors, none of the restart stuff or emails. 

Strangely, my main web server has crashed twice during the last two nights, restarted within a few seconds, not happened since November and that was development bugs in ICS. Error was  C0000005 buffer overrun. Using YUOpenSSL.  

But it could be hackers trying to exploit several low priority exploits in OpenSSL due to be fixed this week.

Angus

Thanks, I'll look at Application Recovery and Restart, if it applies to Windows Services. 

Mine are all set to restart if the application stops, and my services go to a lot of trouble to try and save logs and terminate cleanly on any unexpected errors, including emailing me, although I've delayed that now until the service restarts, since corruption sometimes meant the service locked solid instead of stopping and restarting meaning manual intervention.  Keep meaning to write a second monitoring application, but these problems are so rare I never get around to it.

Angus

To be honest, I've not really thought about this problem in a long time, my original comment was about the FTP server, but I transfer thousands of files daily to and from my servers, so it's not an issue that needs much of my time.

I do get a web server crash maybe once every couple of months, but I also get continual attacks on those servers, there are usually 100 or more IPs blocked for attacks.  Sometimes they can be so heavy that even the firewall gives up, had to replace that.  So hard to say if the problems are Delphi or OpenSSL related.  All my servers have heavy logging, but only flushed to disk every few seconds, so usually lost during a crash.  These are live, not experimental servers, if they stop my phone rings within a few minutes. 

Angus

I've not seen the heap corruption error for a long time, but the four ICS servers applications (web, rest, ftp, proxy) installed on my public servers are all built with YuOpenSSL Win64.   All my servers are single thread, although some FTP commands use a thread for lengthy commands. 

My Windows Server 2018 in particular has one Win64 server that won't start with the OpenSSL DLLs, yet other servers work, and it works on Server 2022, but I've not looked at that for 18 months since YuOpenSSL works fine. 

There are major OpenSSL changes in V9.1 that are almost ready for release, once that is out I'll do some more testing with the OpenSSL DLLs.   And long term I do plan a new threaded web server, to allow more than one CPU to be used. 

Angus

I always install Delphi from ISOs, I have DVDs and CDs in a box going back to Delphi 1. 

The version I need today does not have an ISO...

Angus

Sorry, I think this topic should now be closed. 

He is going around in circles installing random packages not designed for his compiler, then wasting our time repeatedly asking why they don't work.  And ignoring all our efforts to install ICS properly.

He needs professional help, but this is not the correct place. I will no longer be responding. 

Angus

This may be a silly question, but did any Embarcadero.com servers ever come back? Two weeks ago...

Trying to install a new version, the online installer complains get-it-olympus.embarcadero.com timeout.

Angus

Sorry, insufficient information to make any real comment for the errors. 

You need proper logging to see when you are making requests and when those responses arrive, which will help explain what is wrong.

This is all much easier using the TSslHttpRest component that removes over half your code, and includes logging.  You should try accessing the server using the OverbyteIcsHttpRestTst sample and see if that works better.

Angus

12 hours ago, FearDC said:

if we conditionally replace kqueue with epoll, l

Another ICS user has already tried to do that with ICS V9, he sent me the modified WSocket unit but not the EpollTypes and EpollAPI units so I can not add it ICS V9.  V10 does not use it.

Angus

The OverByteIcsDXERun package built OK, but a post build command failed, probably because you have not extracted the ICS files into our directory structure, as required.

Building OverbyteIcsPingTst.dpr says Success, so why post it?  We don't need minute by minute progress on your development attempts.  This is not a teaching forum. 

If you only want to run the samples, they are all available pre-built from https://wiki.overbyte.eu/wiki/index.php/ICS_Samples

Angus