How to validate the public key

[sfrazor 3]

ICS 8.70 and Delphi 11.3

Using TNetHTTPClient there is an event handler that allows verification of key data and setting the accept to true/false.   OnValidateServerCertiicate(const Sender:  TObject; const ARequest: TURLRequest; const certificate; var Accepted: boolean).

 

I am currently working with TSslHttpRest.  I am unsure how to accomplish this same validation check in ICS.  If there is a demo that illustrates this, or some direction, I would  appreciate the feedback.

 

 

[Angus Robertson 574]

The OverbyteIcsHttpRestTst sample illustrates SSL/TLS certificate validation, if that is what you mean by key data. 

 

You don't have to write code or make decisions, it's all handled automatically by ICS, if you set property CertVerMethod to CertVerBundle or CertVerWinStore.  ICS has built in root bundles for certificate validation.  Property SslReportChain will report the chain for your log, while SslRevocation will cause an OCSP server to be checked as well. 

 

Angus

 

[sfrazor 3]

42 minutes ago, Angus Robertson said:

The OverbyteIcsHttpRestTst sample illustrates SSL/TLS certificate validation, if that is what you mean by key data. 

 

You don't have to write code or make decisions, it's all handled automatically by ICS, if you set property CertVerMethod to CertVerBundle or CertVerWinStore.  ICS has built in root bundles for certificate validation.  Property SslReportChain will report the chain for your log, while SslRevocation will cause an OCSP server to be checked as well. 

 

Angus

 

Thank you. I'll look at that example.  Working strictly with self signed keys in a closed environment, I need to specifically validate (compare) a compiled in public key in the client to the public Key sent by the server and accept or reject based on that compare.

[Angus Robertson 574]

Checking the public key is exactly how chain verification works.

 

Your self signed certificates should really be signed by your own certificate authority, or ideally an intermediate issued by your own CA.  You distribute the CA certificate to your PCs and install it in the Windows and/or PEM store, and all normal certificate chain validation just works, for any application. 

 

The OverbyteIcsX509CertsTst sample will create certificates signed by your own CA, or intermediate, I use them for testing on my LAN. 

 

If you want to check the server certificate chain yourself, use the OnSslHandshakeDone event.

 

Angus

 

[sfrazor 3]

unit Unit6;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants,
  System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, OverbyteIcsWndControl,
  OverbyteIcsHttpProt, OverbyteIcsWSocket;

type
  TForm6 = class(TForm)
    Memo1: TMemo;
    Button1: TButton;
    SslHttpCli1: TSslHttpCli;
    SslContext1: TSslContext;
    procedure Button1Click(Sender: TObject);
    procedure SslHttpCli1SslVerifyPeer(Sender: TObject; var Ok: Integer;
      Cert: TX509Base);
    procedure SslHttpCli1SslHandshakeDone(Sender: TObject; ErrCode: Word;
      PeerCert: TX509Base; var Disconnect: Boolean);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form6: TForm6;

implementation

{$R *.dfm}

procedure TForm6.Button1Click(Sender: TObject);
begin
  SslHttpCli1.URL := 'https://www.google.com'; //choose any https server
  SslHttpCli1.ResponseNoException := True;
  SslHttpCli1.Get;
end;

procedure TForm6.SslHttpCli1SslHandshakeDone(Sender: TObject; ErrCode: Word;
  PeerCert: TX509Base; var Disconnect: Boolean);
var
  MyKey: string;
begin
  Memo1.Lines.Add('HandshakeDone: ' + IntToStr(ErrCode));
  Memo1.Lines.Add(PeerCert.GetPKeyRawText(True));
  if PeerCert.GetPKeyRawText(True) = MyKey then <<<-- this is wrong and is probably the private key
    Disconnect := False
  else
    Disconnect := True;

end;

procedure TForm6.SslHttpCli1SslVerifyPeer(Sender: TObject; var Ok: Integer;
  Cert: TX509Base);
var
  MyKey: string;
begin
  Memo1.Lines.Add('VerifyPeer');
end;

end.

 

I think the question is, in either VerifyPeer or HandshakeDone, how do I retrieve the peer pubkey for validation?  A hex string would be optimal.

Also in the above is GetPKeyRawText(True)); for retrieving the public key or private key?  

Edited August 30, 2023 by sfrazor

[Angus Robertson 574]

Sorry, no ICS applications need to use or display a raw public key, so there are no methods available to get it as a string. 

 

The TX509Base property X509PublicKey returns a pointer to the internal OpenSSL key, but there are no ICS functions to convert this to a string.  There are some Jose functions for JSON Web Keys but these need private keys, not public. 

 

You can see the use of GetPKeyRawText in the OverbyteIcsPemtool sample, it prints all fields from a certificate, including the public key in hex, but you would have to parse the result to get the hex only. 
 

As I said before, applications normally check certificates, not keys.

 

Angus

 

[sfrazor 3]

Thank you Angus for looking in to this.  Unfortunately it seems I'm stuck with TNETHttpClient since it surfaces all aspects of the key for manual validation including the public key.   

CertName
SerialNum
Subject
Issuer
ProtocolName
AlgSignature
AlgEncrytpion
Cert Start Date
Cert Exp Date
Keysize
publicKey (RSA Modulus (n) in hex)

Seems all it is missing is fingerprint.

 

When SSL has determined the keys are a problem and the callback allows override, publickey is very popular for manual validation because its small and easy to embed.

I'll investigate pulling the X509PublicKey but I don't think I'll be allowed the time.  Please consider surfacing publickey in the future.  🙂

 

  

[Kas Ob. 121]

@sfrazor You are defeating the whole purpose of using X509 certificates with SSL/TLS by comparing the certificate elements.

 

X509 infrastructure is there to establish a chain of trust, means you don't need to ..... really long story....!!!

 

I will try to simplify the process in few steps to utilize X509 certificates the right way.

1) Create self signed certificate and lets call it root, you must not disclose its private key to anyone or anything, means even your client and server must not have its private key, but the client must have it, the certificate its self.

2) Put that root certificate in place to accessed by the client, either install it in Windows store or even better use the ICS bundle store.

3) Create Another certificate and lets call it SC ( server certificate) and sign it by the root !, this one should be with your server with its private key.

4) That is it, we are done preparing, the client on handshaking will verify the server certificate (SC) against his included trusted root !

 

also you can add middle certificates between the Root and the SC , and those called CA , these can be included with client bundle or sent with the SC as a reply for the Client establishing a handshake.

 

Please try searching and researching this process more in detail, and if you need better or easier way to generate certificates then try this

https://hohnstaedt.de/xca/

Also you can use as a safe place for your certificates and their keys.

 

 

[sfrazor 3]

2) Put that root certificate in place to accessed by the client, either install it in Windows store or even better use the ICS bundle store.

Unfortunately this is not an option.

 

Thanks for the input Kass Ob.  I completely understand.  If this were a normal client app I would do it exactly as you mentioned.   In this case we're dealing with some imbedded code and size matters.  Pushing the entire cert in the binary is not preferred and there is no place to write the pem(cert) to on the client.  So in order to validate the cert belongs to a specific server, the publickey is imbedded and used for validation.  We could imbed the key fingerprint instead.  But none of the comms components I've seen  surface the fingerprint.

[Kas Ob. 121]

19 minutes ago, sfrazor said:

In this case we're dealing with some imbedded code and size matters.  Pushing the entire cert in the binary is not preferred and there is no place to write the pem(cert) to on the client.  So in order to validate the cert belongs to a specific server, the publickey is imbedded and used for validation.  We could imbed the key fingerprint instead.  But none of the comms components I've seen  surface the fingerprint.

Well, if you insist on going this then i suggest to just hash the X509 certificate file in whole and check the hash, and call it done, this will be identical to the thumbprint but with different value.

[Angus Robertson 574]

ICS exposes the two hashes for the certificate, which are small and easy to check, but this would only work if there is only a single self signed certificate on the network, if there are more than one all signed by the same private key, like a CA, then checking the public key is the only solution.  But no-one else has ever needed it.   It is only a couple of lines of code using OpenSSL functions, but not this week.

 

Angus

 

[sfrazor 3]

50 minutes ago, Angus Robertson said:

ICS exposes the two hashes for the certificate, which are small and easy to check, but this would only work if there is only a single self signed certificate on the network...

There is 2 servers on the network both using the same CA for keygen during the CI/CD pipeline builds.  That process is how the client apps get the pubkey imbedded when they are built.  Its a one-to-many between the server and app. No other servers besides the primary and failover servers will have that pubkey and the apps will only rollover to a secondary that has the cert signed by the same CA.   This effort is a progression from a previous post where DLL plugins loaded with memorymodule cause bad results when exception handling is used for comms state instead of error codes during a failover.  In the case NetHttpClient, that's a show-stopper now.  ICS offers a way to disable that and rely on error codes as you pointed out in a previous post plus it supports YuOpenSSL. Thus the ICS investigation.

 

I'd have to mod our pipeline builds to use a hash instead of publickey.  That also means I'd have to redistribute the current 2 rest apps (soon to be 3) to validate against a hash.   

 

I'm still investigating pulling the X509PublicKey via the pointer that is passed in via peerverify.  

 

[Angus Robertson 574]

The X509PublicKey property returns a PEVP_PKEY pointer which can be passed for processing to numerous OpenSSL functions, PEM_write_bio_PUBKEY might do what you need, it is used in the PublicKeySaveToText method, but that needs a private key which you don't have.

 

PEM_write_bio_PUBKEY returns base64 ASCII text, the same as you'd find in a PEM file for a public or private key, which is probably what you have already, it probably has the top and tail headers...

 

Angus

 

Edited August 31, 2023 by Angus Robertson

[Angus Robertson 574]

It seems the industry has come up with RFC7250 Raw Public Key to allow authentication of TLS connections without using X509 certificates and trusted certificate chains, which seems aimed at your type of LAN application.  You are really doing the same thing, but still sending a self signed certificate you ignore.  

 

RPK uses TLS extensions, which are supported by the next OpenSSL release 3.2 currently in alpha testing.  I'll look at supporting RPK when it arrives.

 

Meanwhile, I'll add a TX509Base method to get a public key this week.

 

Angus

 

 

[Angus Robertson 574]

This fix is now in SVN and the overnight zip, property X509PubKeyTB in TX509Base will get the certificate public in DER binary format as TBytes, from where it may be converted to hex or base64, and used for Raw Public Key certificate validation. 

 

At least assuming the raw key distributed is the DER format.  It seems comparing a SHA256 digest is common, rather than a long string. 

 

Angus

[sfrazor 3]

Thanks Angus.  I had to have a look at our pipeline.  Its producing a PEM formatted cert(s).  We have a server and a couple of apps that are using it.  I'll look in to changing to DER and see how far I get.  I seem to recall being able to convert to-from PEM/DER.  Not something I'm familiar with.  I really appreciate the work!!

 

 

Edited September 14, 2023 by sfrazor

[Angus Robertson 574]

A PEM certificate is base64 encoded lines of the binary DER, with a header and trailer added, although ICS often adds comments to the files for identification purposes.  So could just strip off the headers, remove line endings and do a comparison of the base64 text.

 

Is your server also ICS?  So it could be updated for Raw Public Key TLS support?

 

Angus

 

 

 

[sfrazor 3]

On 9/15/2023 at 2:49 AM, Angus Robertson said:

A PEM certificate is base64 encoded lines of the binary DER, with a header and trailer added, although ICS often adds comments to the files for identification purposes.  So could just strip off the headers, remove line endings and do a comparison of the base64 text.

 

Is your server also ICS?  So it could be updated for Raw Public Key TLS support?

 

Angus

 

 

 

Thank you.  managing certs is all magic to me. This is definitely a learning experience.  So, decode and clean up the text...

 

 

Its a fairly simple server in python (tornado).  However. we currently have a ticket in our backlog to rewrite with additional features.  It will be ICS at some point as we've been asked to move away from tornado/python and toward RAD Studio solutions  I added your Raw Public Key TLS support to the comments in the ticket for consideration.   All clients ( 2 and soon to be 3) are currently TNETHttpClient based dll's and as mentioned, failover has become a priority.  As also noted above, the dll plugins are memory loaded and I'm stuck with that architecture for the time being.   TNetHttpClient uses exceptions to determine connection failures and rollover causes the dll's to crash when an exception is hit/raised.  I am working on a replacement that uses error codes in comms instead.

 

Edited September 18, 2023 by sfrazor