Angus Robertson

I've updated the ICS MQTT client and server components to support protocol 3.1.1, previously we only supported 3.1. Not finished testing, not checked all packets yet. The client will connect to a v5 server by ignoring dozens of new options, but needs a lot more work, much more complicated than v3.1.1. 

Also added a lot more logging so we see the packets being sent and received, and added more options to the sample.

In SVN now, the overnight zip will be updated this evening.

Angus

I always return REST Json responses with a results wrapper, success/fail, to avoid the client needing to parse for data. 

But your change is made, and will be in SVN this week.

Angus

The REST component does a simple check for {}, but would miss Json that comprises only array elements without any objects, which I assumed was very rare.  

But adding such a check is trival, meanwhile you can just parse ResponseRaw, albeit with the risk of failed parsing from non-Json text. 

Angus

No, OpenSSL is widely used with threaded programs, provided those programs are written correctly. 

Angus

Windows itself uses a thread for each async socket.

Angus

On the general issue of using threads, remember that the ICS web and FTP servers support hundreds of simultaneous clients without using threads, I've never understood why so many people assume that threads are necessary to do two or more client connections at once.  Unless they are migrating from using blocking components... when the code should be rewritten properly.

At some point, there will be a new ICS threaded server component, since threads are necessary to use more than one CPU, but the server will generally only need one thread per CPU, with multiple clients in each thread.

Angus

Using threads to handle multiple components in parallel is unnecessary, provided you are using async methods, so fully event driven.  I've done tests with hundreds of components running in parallel, the SSL negotiation takes a while so eventually limits the number in parallel.

The FTP component should usually be created and destroyed within the thread. 

There is a specific component for your purpose, TIcsFtpMultiThread in OverbyteIcsFtpMult.pas with a sample OverbyteIcsXferTst.

Angus

I've added a new event OnWSFramesDone to the Websocket client, called when the queue of frames has been sent, for flow control when sending a lot of data. Previously, the OnWSFreameSent was called after each frame, but it was not easy to tell if this was the last queued frame.  

I've only tested it with a few frames of ASCII, the HttpRest sample really needs a new button to send a queue of files, which is on my list, but also needs a server able to accept those files.  But I'm sure you'll tell me if it does not work.  In SVN now, will be zipped overnight.

Angus

24 minutes ago, Lajos Juhász said:

I need badly Konopka Signature Controls.

I've just installed Bonus KSVC 7.0 OK. 

Angus

ICS V9.4 is ready for RAD Studio 12 Update 3, aka RAD Studio 12.3, and specifically the new 64-bit IDE.  The ICS Delphi V9.4 design packages already have Windows 64-bit as a target, allowing them to be installed into the 64-bit IDE.  The 32-bit IDE allows design packages to be built with target Windows 64-bit, but not installed.

Note the 64-bit IDE does not support Windows 32-bit targets, only Windows 64-bit.  If you use the IcsInstallVclFmx.groupproj Build Group, all design packages will be built for Windows 64-bit.

All the main samples build OK in the 64-bit IDE, with a Windows 64-bit target being automatically added, unfortunately even when deliberately missing such as the OverbyteIcsNetMon sample that does work in Windows 64-bit.

Currently, GetIt only works in read-only mode in the 64-bit IDE, so you will need to install ICS using GetIt from the 32-bit IDE, then start the 64-bit IDE and manually install the three design packages.

The ICS V9.4 C++ Builder packages can be currently be installed successfully into C++ Builder 12.3 32-bit IDE for the Windows 32-bit target only.  The 32-bit IDE should be able to build for Windows 64-bit and Win64x (Modern) as well, all the package paths are correct, but both targets currently give compiler errors.

The C++ Windows 64-bit errors are four unresolved external, and the same error happens in Delphi 11, so should be fixable by someone with C++ knowledge.  Building for Win64x (Modern) gives lots of undefined symbol errors due to RAD Studio being unable to import other packages compiled built with C++ Win64x, this error was reported six months ago for C++ 12.2 but is not fixed in 12.3.  So ICS can not be used in the C++ 64-bit IDE that only supports Win64x (Modern), not Windows 64-bit.

Separately, there is a new version of the DD Service Application Framework used by two of the ICS samples, adding support for 64-bit IDE Wizards.  Available shortly from https://www.magsys.co.uk/delphi/ddservice.asp

Angus

What is the benefit of keeping the internal TLS keys?  Wireshark?

Angus

Thanks, I'll add it shortly.

Angus

I'm updating the ICS MQTT client for v3.1.1 at the moment, should be this week.  Not planning v5 at the moment, that is totally different.  Most servers still support 3.1.1.

Angus

I've installed the Mosquitto Broker server locally, and it works OK against the ICS MQTT client.  

Our sample currently always adds a default username/password, and this was upsetting test.mosquitto.org on the unauthenticated port, so the sample now allows them to be set specifically, and leaving them blank connects OK to test.mosquitto.org, which is also now on a ComboBox dropdown to make testing easier.

But this does not explain why PicoMQTT fails (unless it's also authentication).  Mosquitto supports 3.1, 3.1.1 and 5, I'll see if I can disable 3.1.  There is a comment in the conf file about clients connecting with 3.1.1 and a zero length client Id so the server allocates one, perhaps this explains the difference. 

Angus

I get Connection NOT AUTHORISED connecting to test.mosquitto.org on ports that say unauthenticated, 

I'll spend a few minutes looking to see why, but my knowledge of MQTT is zero, and I don't propose to learn the protocol, maintenance of the ICS MQTT component really needs to be done my users that understand and use MQTT. 

Angus

I would assume the difference between 3.1 and 3.1.1 is relatively small, so it should be easy to change the ICS component.  But we'd need to know what that difference is, and a server to test against.

Angus

I am not going to change ICS to check for Windows XP automatically.  But if you follow my earlier instructions, ICS can be built with SSL, but not loaded automatically, so you can choose whether to use SSL for each application, as happened prior to V9.1. 

Angus

OpenSSL 3.0 and 3.4 both use ConvertThreadToFiberEx so I guess you were using an older OpenSSL with v8.  Never even heard of Fiber APIs before. 

Not easy to check, Dependency Walker took 18 minutes of CPU time to fill it's treeview for those two DLLs, slowing my desktop to a crawl. 

Angus

The readme9,txt file explains how to change defines to stop OpenSSL being loaded automatically, some components also have a NoSSL property.  Generally, we make it easy to use SSL, not to not use it.

Angus

Sorry, we ceased support for Windows XP several years ago, the OpenSSL DLLs do not support it.  The oldest OS supported is Windows 7, and that is rarely tested. 

ICS does not use ConvertThreadToFiberEx, perhaps OpenSSL does.  So you may be able to disable all SSL functionality to keep XP working, but you can not use very old OpenSSL DLLs. 

Or stick with old unsupported code for an unsupported OS. 

Angus

Sorry, no simple answer, the MQTT component was contributed, and only tested against itself.  I don't have any other local MQTT servers to test against, although it was tested against some public servers.  I'm aware there are various versions of MQTT and some servers are non-standard, questionable whether ICS should break the protocol to work with devices that don't follow it, please search back in this forum, I think it's been discussed before.

Otherwise, you'll need to use Wireshark or something to see how the other clients are communicating with the server, and change the component to match that.

Angus

I'd make the general point that ICS is an async library, generally you should never use Sleep(), but events.  If you want to delay something, use triggers within a timer.  Having multiple ProcessMeesages everywhere is also bad design.  

Your code should be packaged into an object with events, called before a single message loop.   In fact, this design would mean you can test and debug your WS code in a simple GUI before using it in a DLL. 

From your various comments, I gather you are writing a Websocket client DLL that sends large binary blocks of data to a server, All my testing and the ICS samples are server to client communication, although in theory the code is two-way and similar in client and server.  But I simply don't have a way to easily test your requirement.

Angus

Sorry, no idea off hand, not used Websockets for sending large binary blocks, only simple ANSI/HTML packets. 

Reproducing your scenario is not trivial and would take some effort, I'll put it on my list, but it might take a while.  Are both server and client ICS WS apps?

Angus

I've seen problems with Cloudflare and ICS, it can be sensitive to the User-Agent or strange request headers, it tries to be too clever and fails. Using a real browser User-Agent might help. 

Angus

Try Sleep(0) or MsgWaitForMultipleObjects before calling the pump, that is what the ICS synchronous methods do.  BTW, Snippets is simple examples. 

Angus