Angus Robertson

Technically, you can use the event you posted to get a new token from the desktop through interaction, ICS V8.71 now uses an internal browser window that makes it seamless. 

But it's not really necessary, using the same secrets in service and desktop applications, get a refresh token using the desktop and manually paste that to the configuration file for the service, use the IcsLoadRestEmailFromIni function to load it from an INI file per the ICS server samples.  The refresh token rarely expires so only needs to be updated if you change the secrets or deliberately invalidate it online. 

I've been doing this with the IcsMailQueue in my web, rest and FTP servers on all my different servers, all with the same refresh token, for two or three years.

Angus

Indeed, the OverbyteIcsIpStreamLog unit builds text lines by parsing a TByte buffer one character at a time.  But many quick and dirty programs accept that TCP sends full packets, like my new WebSocket sample. 

The other advantage of TByte is avoidance of pointer handling, all those @ and ^ symbols that really have no place in modern applications since they can be abused so easily.  Ditto the Move function.  ICS simply offers alternatives, choose the easiest to use.

Angus 

From the notes for ICS V8.70 TWSocket:

Added ReceiveTB(var Data : TBytes; MaxLen : Integer = -1) : Integer; where MaxLen is optional, to receive TCP data into a TBytes dynamic array of bytes. Also ReceiveFromTB and ReceiveFrom6TB for UDP datagrams.

Receiving into a fixed size buffer is more slightly more efficient, but TBytes has more RTL support for converting to strings, etc.  ReceiveTB will resize the variable up to MaxLen. 

The new functions are used in the OverbyteIcsIpStreamLog unit which how they were tested.

Angus

I accept the package structure for ICS is out of control, with over 280 dpk, dproj and cbproj files I have to update when I add a unit, even updating a single Delphi release is 10 or more packages.  But for the same reason, changing it will take days of effort if we continue to support all versions of Delphi back to 7. 

Ideally an application should have been written years ago to create packages from a master file, as I did for the SSL/TLS root certificate packages, but it never happened.

So changing it for ICS V8 is highly unlikely, but very probable for ICS V9 that will only support recent compilers so we can use new language features.  Vincent's suggested package rules all make sense, for new projects. 

I used to create separate packages for my own ICS components without problems, but they are nearly all now part of the ICS distribution. 

Your MQTT project appears to be branched from the 10 year old https://github.com/pjde/delphi-mqtt and the simple way to avoid package problems in the future would be for the units and sample to be included with the ICS distribution, as I did with the new WebSocket unit recently. 

Angus

29 minutes ago, Sid D said:

Looks like something to do with TIdServerIOHandlerSSLOpenSSL component?

Highly unlikely, it will be the way you have configured the component, probably the wrong certificates or protocols, but no-one can guess what you have done wrong. 

Ssllabs will keep the host secret if you tick the correct box, and will almost certainly give you clues. 

Angus

A PCI scan will be for a public server, so why are you hiding the public host name?  

SSLLabs rates SSL sites and offers extensive advice about certificate errors, server misconfiguration and chain errors. 

The error you describe sound like the certificate is issued by an untrusted CA, but no-one can tell without seeing it. 

Angus

ICS has a proxy server sample that includes logging and headers and optionally bodies, I've used it for debugging SSL connections.  You can even fake a server SSL certificate (and some anti-malware packages do, to intercept SSL sessions if you can not originate in HTTP.

Angus

WinPCap development ceased a few years again, it was taken over by Npcap which is used by most network sniffing software, including Wireshark which is the leader. 

I wrote a Delphi sample packet sniffer using WinPCap and Npcap many years ago, very basic really needs better filtering and IPv6 support, must spend some time on it.

Angus

I've built the ICS V8.70 and now V8.71 packages on Delphi 11.2 dozens of times since September when it was released.  The new WebSocket units are only in SVN and the overnight zip, not V8.70. 

The package says 11.0 because Embarcadero changed the naming scheme when releasing the first update to 11, but works with all versions of 11.

Sorry, no idea why it wants to rebuild the RTL package. 

Angus

There was a new WebSocket server in SVN last week, sharing much code with the new client since WebSocket is a symmetrical service, and built into the existing web server samples running on port 443, rather than a separate port.  Most of the methods and events are the same for client and server.

The OverbyteIcsSslMultiWebServ and OverbyteIcsDDWebService samples should respond to these WebSocket URLs:

wss://localhost/WebSocket/Echo     (echoes messages received)
wss://localhost/WebSocket/EchoPing (echo and send keep alive pings)
wss://localhost/WebSocket/Chat?MyName (multi user chat server)

There is a new websocketclient.html page listed on the main demo.html  page that allows testing these WebSocket servers.  The new client component will also access these URLs.  I also have similar URLs on my public web server and anyone wants to test the client alone, but email for the full URL, I don't want it indexed.

Angus

Quote

Will there be in the near future any update to support macOS 64 bit?

Only if a Apple user contributes something, neither Francois or I have any suitable hardware so can not test anything.

Angus

ICS ships with four different root certificate stores to allow chain verification, three as files, one as linked unit. 

How they are used depends on the component you are using, older and low level components need extra code to check the certificate chain, if you use the TSslHttpRest component it is all built in, you decide whether to use a root store or the Windows store, look at the OverbyteIcsHttpRestTst.dpr sample to see it all working.

Angus

It is not mention in the documentation, but the Magenta Hardware Components package includes a unit MagService.pas written almost 20 years ago that says:

MagService is a unit of functions to control, install and remove Windows Service applications.  These functions all require the application to have administrative access rights, which can be checked by IsProgAdmin.  Services can be started and stopped, check if running, installed to run with startup options, an account and dependencies, have a service description set, or removed from the service database.

My old WMI unit can be downloaded from the same place, although the latest version is part of ICS.  But WMI is much slower than direct windows APIs. 

Angus

I suspect the old source file was still held open by Delphi after it had said 'another application has updated the file, reload it'. 

Tried deleting all historic files, still failed, restarted 11.2 and sanity was restored.

Angus

Delphi 11.2, after editing a unit and trying to save it, I get an error: 

Unable to rename 'C:\DelphiComp\ics\Source\OverbyteIcsWebSocketSrv.pas' to 'C:\DelphiComp\ics\Source\__history\OverbyteIcsWebSocketSrv.pas.~74~'.

and can do nothing else than abandon changes.  The version 74 does not exist, only 73. 

Is there a clever way to save the changes, other than copy/paste to a new file? 

Suspect it happens if I edit the same file in another version of Delphi on the same PC, a couple of days ago.

Angus

The long planned new web server will use a configurable mix of threads and parallel components to make proper use of multiple CPUs, allowing heavier loads than can be supported by a single thread as at present.  For instance, there could be four threads, each with 500 clients, or 500 threads each with one client. 

ICS already has a TSslWSocketThrdServer component (one per client), but there has never been a web server using it. I've never heard of anyone using it. 

Angus

As I said, SSL/TLS is always selectable for each connection, that will never change, except when QUIC becomes an alternative. 

Quote

I'm curious what exactly troubles do you experience with USE_SSL defines?

TSslWSocketServer is a relatively simple component, but descends through eight different classes with 24 USE_SSL defines, all of which makes improvements difficult, the new version will be much simpler.

The SChannel wrapper for ICS is an interesting project, but YuOpenSSL allows ICS to be used without external DLLs, and SChannel is different with every operating system, fine if you only want to support the latest and greatest, useless if you want applications to continue to run on Windows 7 or 8 where each time Windows Update is run, half the ciphers disappear so causing ECDSA key certificates to be ignored, for instance. 

Angus

SSL./TLS is nearly always optional for the SSL enabled components and that will never change.

I'm planning a new web server component, and there will be no non-SSL version, as there is no non-SSL proxy component.  The existing web server without SSL/TLS could then be abandoned.

Angus

Quote

SSL/TLS remaining optional is valuable to me.

Why?

Angus

I would also argue that using ICS without USE_SSL is risky since I do zero application testing without that define, before a new release I simply ensure the packages will build without error without that define.  So testing is dependent on end users that don't use SSL/TLS.  Thus the recommendation to use older tested versions.

Angus

Since 95% of new features in ICS relate to SSL/TLS, those that only need limited features and no SSL/TLS may as well stick with old versions. 

Sure there are applications that don't need SSL/TLS so it is optional in most components, the issue is whether those components should be compilable without the SSL./TLS code to save space, at the expense of extra complexity and riskier maintenance, thousands of conditional statements.  Simply, it wastes my time. 

Angus

Thanks, I'll look at the various package issues before the final V8.71 release when I update them all with new components.

Done the defs file, but it just missed an SVN update, next time. 

I rarely test without USE_SSL, it will disappear from the next major release since very little of the internet works without SSL/TLS nowadays and all the conditionals and extra classes make maintenance and upgrades a nightmare.

Angus

This particular web site is aimed at the UK market and has little interest for those outside the UK, it contains information about UK telephone and broadband networks. Some information is free but limited, businesses pay for access. 

But some users try multiple IP addresses to get around my counting accesses against IP addresses, and Tor is easy for them (I assume).  I just went through the 50,000 accesses sequentially to my database to find Tor exit nodes.

I did accidentally block access to British Telecom at the same time, got a phone call from their network people asking why their staff could no longer access my site (for free)...

Angus

My Delphi web servers have an IP blocked list which hackers get automatically added to based on several criteria, any path with filers ,php, .pl., /,,, ;echo, mysel, etc,since there is no CGI or Perl or MySQL on the site, also if they access pages by IP address instead of the correct domain name using SSL.  I also block Tor exit nodes, to try and stop IP blocked users using alternate access. 

Blocked IP addresses are cleared from the list 24 hours after their last access, and any accesses have a response delayed by one minute, so try and slow down the avalanche of requests hackers make, often hundreds a minute.  Currently 179 IP addresses blocked on my main site.

Angus

You need to request a download password on the TMagRas page, and then go to https://www.magsys.co.uk/delphi/dfiles/default.asp

Sorry about password nonsense, it's to stop Google scanning my files and falsely black listing my site again.

Angus