Indy & OpenSSL 1.1.1 & TLS 1.3

[mezen 13]

Hi,

when you read the title, you probably already had the standard answer in your head: "Does not work with Indy, supports only OpenSSL 1.0.2 at most and thus no TLS 1.3".
I can reassure you, that is not my point. Or actually even more precisely: That is exactly my point 😉

 

I've spent "a little" time writing the Indy support for OpenSSL 1.1.1 and TLS 1.3, and added a push request to Indy:  #299. At the same time I have fixed a few issues that have been posted to GitHub (see PR).

I wrote 2 new IO handlers ( one for server and one for client), the old ones are unchanged to avoid conflicts.

Everything was written and tested in Delphi Berlin 10.1.2 on Win32 and Win64. I have neither macOS nor iOS nor Linux nor Android, nor FreePascal, nor older (or newer) Delphi versions. I have tried to keep older Delphi versions in mind to ensure that it will run on them, but there have been no tests on my part.

I have tested it extensively in small test applications with other servers/clients. In addition, I built it into a large Real World program with TCP server/client, SMTP/IMAP/POP clients, FTP client, HTTP client, and it also ran there without problems.

 

Unfortunately the nice man, who has provided new binary files of OpenSSL under indy.fulgan.com has said that he does not offer versions > 1.0.2 anymore. So I used the versions of slWebPro in the beginning (they even still work on WinXP), later I used the versions of Overbyte, because they do not have any external dependencies (and are digitally signed, but no XP anymore^^). But both worked without problems.

 

All files are located in the subfolder "Lib/Protocols/OpenSSL". There are also subfolders "static" and "dynamic" which offers quite extensive imports of the OpenSSL API, once for static linking, once for dynamic loading/unloading. For dynamic loading there are also possibilities in the "IdOpenSSLLoader.pas" to trigger the loading/unloading itself, if you need the API outside of the IO handler (e.g. for your own x509 generation).
To save me the double writing of the imports, I wrote a kind of intermediate code in the folder "Intermediate", which I let generate with "GenerateCode" to the two variants. The tool "GenerateCode" is only a simple string customization and actually only designed for Berlin, so I didn't bother about downward compatibility. As a normal user of the IO handlers you don't need them, only if you make changes to the API implementation.

So and now comes your. It would be nice if one or the other would test this, so that it is not only WOMM certified, but can withstand more real situations.
For me it also works with the Indy, which comes with Delphi Berlin, when I create another unit, which provides some new Indy types and functions. Of course some units have to be adapted locally to use the new unit.

Edited May 4, 2020 by mezen

[Remy Lebeau 1387]

9 hours ago, mezen said:

I've spent "a little" time writing the Indy support for OpenSSL 1.1.1 and TLS 1.3, and added a push request to Indy:  #299. At the same time I have fixed a few issues that have been posted to GitHub (see PR).

And I do appreciate that work, and I will review it when I have time to do so (I wasn't able to get to it this past weekend, like I had hoped to).

[Lars Fosdal 1789]

Any news on TLS 1.3 support?

We will be needing it this autumn as the APIs we access will make it a mandatory requirement.

[Remy Lebeau 1387]

15 hours ago, Lars Fosdal said:

Any news on TLS 1.3 support?

Still a work in progress.  Mezen's pull request for a new SSLIOHandler for OpenSSL 1.1.x is still pending review, I just have not had time lately to review it yet, but you can download and try it for yourself.

 

Edited June 5, 2020 by Remy Lebeau

[alexandreDEV 0]

Guys, you do a really good job, seriously who works for free in this world? I see less people doing it every day :( I have some program downloaded +100 000 without one donate

I can't donate but i can say THANKS YOU!!!

[Jason Wharton 0]

On 6/4/2020 at 9:15 PM, Remy Lebeau said:

Still a work in progress.  Mezen's pull request for a new SSLIOHandler for OpenSSL 1.1.x is still pending review, I just have not had time lately to review it yet, but you can download and try it for yourself.

 

How can I help?  I need to get higher than TLS 1.2.  My websites are being flagged as having too weak of encryption.

[Angus Robertson 574]

Quote

I need to get higher than TLS 1.2.  My websites are being flagged as having too weak of encryption.

Flagged by whom?  TLS 1.2 is perfectly good provided you disable a lot of weak ciphers and hashes. 

 

Most IIS sites are still only TLS 1.2, Microsoft does not support TLS 1.3 until Windows Server 2022. 

 

Angus

 

[Jason Wharton 0]

On 7/27/2021 at 1:50 AM, Angus Robertson said:

Flagged by whom?  TLS 1.2 is perfectly good provided you disable a lot of weak ciphers and hashes. 

 

Most IIS sites are still only TLS 1.2, Microsoft does not support TLS 1.3 until Windows Server 2022. 

 

Angus

 

Sorry, I mistyped.  I need to get to TLS 1.2.

[Remy Lebeau 1387]

On 7/30/2021 at 4:45 PM, Jason Wharton said:

Sorry, I mistyped.  I need to get to TLS 1.2.

Then you should be fine using the latest Indy with OpenSSL 1.0.2, as it supports TLS 1.2

[Jason Wharton 0]

On 8/2/2021 at 11:25 AM, Remy Lebeau said:

Then you should be fine using the latest Indy with OpenSSL 1.0.2, as it supports TLS 1.2

Yes, correct.  I was mistaken.

 

I finally discovered that my path to the Open SSL binaries was not configured correctly and so Indy was resorting to TLS 1.0.

[Lars Fosdal 1789]

I am getting questions about supporting TLS 1.3 on our https interfaces again...

[Stefan Glienke 1998]

Given the activity on Indy, I am tempted to call that project dead.

[Remy Lebeau 1387]

5 hours ago, Lars Fosdal said:

I am getting questions about supporting TLS 1.3 on our https interfaces again...

Status hasn't changed since the last time you asked.   The new support code for OpenSSL 1.1.x and TLS 1.3 is still in Pull Request #299 on GitHub awaiting review and merge into the main codebase.  AFAIK, it works in general, but I offer no guarantees about it yet as there are some open issues with it, cross platform support hasn't been tested, etc.

4 hours ago, Stefan Glienke said:

Given the activity on Indy, I am tempted to call that project dead.

It is not dead.  I still work on it pretty frequently, but I'm pretty much the only developer working on it (aside from the occasional contribution from users or Embarcadero), and I just don't have the kind of free time that I used to have.  But I do what I can with the time I can spare for it.

 

Plus, it doesn't help that I don't have a working IDE installed anymore.  Many years ago (I just checked my blog, and wow, I didn't realize just how many years have past now!), a series of multiple back-to-back system crashes took out my entire development environment - all of my VMs, NAS backups, laptop, everything gone.  To this day, I still haven't recovered any of my old files yet.  In the case of Indy specifically, fortunately the main code was stored online, so I can continue to checkin updates/fixes as needed, but I lost a lot of internal dev code I was working on at the time.  As for the IDE itself, I just haven't committed time to reinstall it yet.  I don't use RAD Studio in my day job anymore.  Not to mention I haven't been very happy with all the problems I see people report in various online forums about the past handful of releases, so I've been hesitant to reinstall a new IDE for awhile.  But, last year I did finally buy a new laptop for future dev work, and plan on installing RAD Studio onto it (eventually).

 

I know, it sounds like I'm just making excuses.  Maybe I am.  I need to pick up the slack ...

Edited March 25, 2022 by Remy Lebeau

[Stefan Glienke 1998]

As a solo maintainer of an open-source library myself I can say this: if that person does not have a working dev environment installed anymore and does not have time to work on it - I personally consider that project being de facto dead. That is not an offense but there has to be taken action. Nobody can blame you for not working on a spare time project if you got no time or the mood to. But if that project is of fundamental importance - and I consider Indy as a crucial piece of the 3rd party ecosystem for Delphi - there needs to be something done about it.

 

Not having an up-to-date networking library available (yes I know there are several others, but Indy always has been out of the box and freely available) is one of the many reasons that make Delphi unappealing for decision-makers.

[corneliusdavid 213]

First of all, THANK YOU @Remy Lebeaufor your contributions over the years to the Indy project, both in development and in support on this and many other forums.

23 minutes ago, Stefan Glienke said:

I consider Indy as a crucial piece of the 3rd party ecosystem for Delphi

So do I and to learn there's only one main developer on this project who doesn't even have a working IDE at the moment is quite concerning. I wish I had the time/knowledge to help. 

[KenR 29]

I didn't realise Indy was not being updated. I have used it since it first became available. I feel I need to move away, but what to?

[Remy Lebeau 1387]

1 hour ago, Stefan Glienke said:

there has to be taken action. Nobody can blame you for not working on a spare time project if you got no time or the mood to. But if that project is of fundamental importance - and I consider Indy as a crucial piece of the 3rd party ecosystem for Delphi - there needs to be something done about it.

I totally agree.  And I have been meaning to reach out to the community for a long time looking for volunteers to join the dev team on a more regular basis, I just haven't gotten around to it.

21 minutes ago, KenR said:

I didn't realise Indy was not being updated.

Well, it is still being actively maintained, as in it receives updates for fixes, and adding minor features that are easy to add to the existing code.  But major updates have been delayed for a long time.  Namely, releasing Indy 11 (maintenance release to drop pre-Unicode compilers, restructure the runtime packages, etc) and starting on Indy 12 (major new features, logic rewrites, etc) has been on the back-burner for many years now.  Actually, Indy 11 is almost ready, but without an IDE I haven't been able to test the new structure and finalize it.

Edited March 25, 2022 by Remy Lebeau

[Mike Torrettinni 198]

I only use Indy for software update check, so very minimal. And I was looking into Indy status the other day and it seems pretty active, with recent bug fixes:

 

https://github.com/IndySockets/Indy/issues?q=is%3Aissue+is%3Aclosed

 

 

Didn't know about any new features implemented, but recent bug fixes are definitely a sign this is an active project.

 

But I guess if I needed TLS 1.3 support, would probably be more worried about when and if it will be available.

 

[Remy Lebeau 1387]

4 hours ago, Mike Torrettinni said:

But I guess if I needed TLS 1.3 support, would probably be more worried about when and if it will be available.

The IF is certainly a yes, but I couldn't tell you the WHEN for sure.  But you can try out the pending code for yourself at https://github.com/IndySockets/Indy/pull/299 and see if it works for you.

[KenR 29]

Good to hear as I use indy a lot and it works perfectly for me. Thanks Remy for your continued involvement.

[Fr0sT.Brutal 899]

On 3/25/2022 at 8:42 PM, Stefan Glienke said:

Not having an up-to-date networking library available (yes I know there are several others, but Indy always has been out of the box and freely available) is one of the many reasons that make Delphi unappealing for decision-makers.

They made x-platform sockets and http clients that use system facilities for TLS so the most requirements are covered. Even Go and Node don't provide much more

[chmichael 12]

@Remy LebeauI'm also using Indy Openssl 1.1.1 patch to download the upgrades with not problems and it works quite well on very slow connections compared to 1.0.2

The code is there so i guess make a poll for the merge or something ?

 

 

[Remy Lebeau 1387]

On 4/2/2022 at 10:22 PM, chmichael said:

The code is there so i guess make a poll for the merge or something ?

The main reason it hasn't been merged yet is because I just haven't had any time to review it - I don't question that it "works" in general (I've seen enough people say it does), but I still need to see how consistent it fits with the rest of the library, if all of the necessary package/IDE support is in place, how it handles the multiple platforms Indy runs on, etc.  The occasional issues being reported in the PR with regards to compiler/runtime errors, etc.  As well as this is kind of a big feature to maybe warrant pushing Indy into a new versioning scheme that is long overdue.  So, a lot of behind-the-scenes stuff that has made me hesitant to just merge it blindly.

[Todd Grigsby 1]

What would it take to get TLS 1.3 support "unstuck"?  I had a major project I was working on that blew up completely because we could get Indy's latest main branch on github to get past the handshake.  From this thread, it appears the last time anyone paid attention to this PR was over a year ago.  I understand that pull requests require due diligence, but what would it take to get this PR pulled in?

Edited September 12, 2023 by Todd Grigsby

[mezen 13]

I wrote on the PR a comment some days ago.

tl;dr: Stopped on that PR, currently working on OpenSSL 3, will create a new PR when its done.