HTTPS problem

[HeikoG 1]

Hello,

 

I'm using ICS 8.68 with Delphi 2007, and the latest 32Bit OpenSSL DLLS from the Overbyte website (http://wiki.overbyte.eu/arch/openssl-3.0.2-win32.zip).

 

I've written a short test program (Just a TSslHttpCli, TSslContext and TIcsLogger component on a form), using the default settings other than connecting them.

I can access google without problem (https://www.google.com), but when I try to access the entry page of the german patent office (https://depatisnet.dpma.de/) I get an error message:

 

SSL handshake failed - No error returned, State: SSLv3/TLS write client hello, connection closed unexpectedly

 

I have attached the log file created.

 

How can I proceed from here? How can I get more information about The error? I can reach this website using firefox, and it seems to use TLS 1.2 then. Internet Explorer on Windows 10 is unable to connect, even with TLS 1.3 enabled.

 

 

Depatisnet.log

[Angus Robertson 577]

In technical terms, depatisnet.dpma.de is a crap web site! 

 

Just run an https://www.ssllabs.com/ test and it says 'Assessment failed: No secure protocols supported' which is very rare, normally poor sites scrape through with a D or E assessment for old protocols. 

 

As you say, normal browsers seem to display the page OK, perhaps there is some scripted magic relocation going on or the server does not like our Agent string, but that should only be checked once SSL is negotiated. 

 

Rather than using TSslHttpCli, you should start new projects with TSslHttpRest which handles all the SSL and logging for you, look at the OverbyteIcsHttpRestTst.dpr sample.  Unfortunately OpenSSL does not provide explanation for protocol errors, they just fail. 

 

Angus

[Angus Robertson 577]

Done a little more research.  SSL Labs tests www.dpma.de okay, but it warns does not support Secure Renegotiation. 

 

Using ICS and OpenSSL 3.0.2 I get a different error to depatisnet.dpma.de, 'unsafe legacy renegotiation disabled', but I can connect OK using OpenSSL 1.1.1.  There is an OpenSSL issue about this, but it seems OpenSSL is removing support for unsafe negotiations which some older servers try to use. The ICS REST component specifically disables renegotiation, but even enabling it does now allow connection to either web site. 

 

So I'm afraid my earlier comment stands, misconfigured web server.

 

Angus

 

[HeikoG 1]

Thanks a lot for you checking, I think over the weekend I'll install the latest ics version using OpenSSL 1.1.1 (8.57), and give that a try.

I also switched my test program to the ICS REST component, but of course that did not fix the problem.

 

Most other options i am aware of are not working either (Indy, Curl, Wget, CleverComponents, WinInet), only Wget2 works.

 

I hope the website maintainers will fix things, but since that's a federal german website this may take a while.

 

Thank you again, if I find out more I'll post it here so others with similar problems may find this info.

 

Heiko

Edited April 1, 2022 by HeikoG

[Angus Robertson 577]

ICS V8.68 supports both OpenSSL 1.1.1 and 3.0, you don't need to use an old version.   By default it looks for the 3.0 DLLs, then 1.1.1, or GSSLEAY_DLL_IgnoreOld and GSSLEAY_DLL_IgnoreNew control which is loaded. 

 

Angus

 

[HeikoG 1]

Hello Angus,

sorry I have to bother you again. I did follow your suggestion and used ICS V8.68 with OpenSSL 1.1.1 (I did run tests with all versions I hav available here, 1.1.1f,g,h,k,l,m,n, all from the Overbyte website).

But I still had no success connecting to https://depatisnet.dpma.de. Are there some other Options I need to set in the SslRestclient or SssContext, or did you get it to work with OpenSsl 1.1.1 and some other tool?

 

Here ist the test procedure I did use, to make sure I did not oversee something when testing different property settings I now create all objects at runtime. I did check GLIBEAY_DLL_FileName to make sure that indeed the OpenSsl 1.1.1 dlls are used.

 

Else I will have to wait if they fix their website, they did implement the ssl change only on march 30th.

 

Thank you

 

Heiko
 

procedure TMainForm.GetPage(Url : string); // Problem with https://depatisnet.dpma.de/
var
  Logger : TIcsLogger;
  SSLContext: TSslContext;
  SslRest: TSslHttpRest;
  Data : TStringStream;
  S : string;
  DateTimstr : String;
begin
  mmoPage.Lines.Clear;
  mmoLog.Lines.Clear;

  GSSLEAY_DLL_IgnoreNew := True;
  GSSLEAY_DLL_IgnoreOld := False;

  Logger     := nil;
  SSLContext := nil;
  SslRest    := nil;
  try
    Logger     := TIcsLogger.Create(Self);
    SSLContext := TSSLContext.Create(Self);
    SslRest    := TSslHttpRest.Create(Self);

    DateTimeToString(DateTimstr,'"-"yymmdd"-"hhnnss',Now);
    Logger.LogFileOption := lfoOverwrite;
    Logger.OnIcsLogEvent := Ilogger1IcsLogEvent;
    logger.LogFilename := ChangeFileExt(Application.ExeName,DateTimstr + '.log');
    logger.LogOptions := [loDestEvent, loDestFile, loDestOutDebug, loAddStamp, loWsockErr, loWsockInfo, loWsockDump,
                          loSslErr, loSslInfo, loSslDevel, loSslDump, loProtSpecErr, loProtSpecInfo, loProtSpecDump];

    SSLContext.IcsLogger := Logger;
    SSLContext.SSLOptions2 := [sslOpt2_ALLOW_UNSAFE_LEGACY_RENEGOTIATION];

    SslRest.SslContext := SSLContext;
    SslRest.IcsLogger := Logger;
    SslRest.DebugLevel := DebugSslLow;


    try
      Data := TStringStream.Create('');
      try
        SslRest.Agent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36 Edg/99.0.1150.36'; // Edge
        SslRest.RcvdStream := Data;
        SslRest.Url := Url;
        SslRest.Get;
        S := Data.DataString;
        mmoPage.Text := S;
        mmoLog.Lines.Add('received bytes ' + IntToStr(Length(S)));
      finally
        Data.Free;
      end;
    except
      on E : Exception do mmoLog.Lines.Add('Exception ' + E.Message);
    end;

  finally
    Logger.Free;
    SSLContext.Free;
    SslRest.Free;
  end;

  mmoLog.Lines.Add('');
  mmoLog.Lines.Add('SSL library path '    + GLIBEAY_DLL_FileName);
  mmoLog.Lines.Add('SSL library version ' + IntToStr(ICS_OPENSSL_VERSION_NUMBER) + '/' + IntToHex(ICS_OPENSSL_VERSION_NUMBER,8));
end;

 

 

[Angus Robertson 577]

Sorry, I did say I was unable to connect to depatisnet.dpma.de with either version of OpenSSL, it was www.dpma.de that connects with 1.1.1 but not 3.0. 

 

Not sure what this means, could be OpenSSL has improved security that now breaks the site.  depatisnet.dpma.de appears to be configured differently, so always fails. 

 

Someone using Wireshark may be able to interpret the handshaking, but that won't fix anything.  It's frustrating that the browsers work, why?

 

You could try looking for a proxy that works, but they are often based on OpenSSL.

 

Angus

 

[HeikoG 1]

Hello again,

 

just for information, by now Depatisnet probably did notice that they have a problem, and did update their site once more. And now ICS works as expected, without any problem.

 

Thanks again for your support.

 

Heiko

[Angus Robertson 577]

I'm now seeing the same 'unsafe legacy renegotiation disabled' error on both sites with OpenSSL 3.0, but that error did not happen with 1.1.1.  So better, but not really fixed unless it really is an OpenSSL error that few other sites show up.

 

Angus

 

[Angus Robertson 577]

Done a little more research, to connect to this site needs literal SslOpt2_LEGACY_SERVER_CONNECT adding to SslContext.SslOptions2. With OpenSSL 1.1.1 this defaulted set, but with 3.0 is not set due to:

Quote

Setting the option SSL_OP_LEGACY_SERVER_CONNECT has security implications; clients that are willing to connect to servers that do not implement RFC 5746 secure renegotiation are subject to attacks such as CVE-2009-3555.

 

Angus