New security requirements for code signing, disruptive ?

[A.M. Hoornweg 144]

Hello all,

 

On 23 june 2023, new security requirements for storing code signing certificates will come into force.  I quote the most relevant change:

 

"Effective 1 June 2023, the code signing certificate key pair must be generated and stored in a hardware crypto module that meets or exceeds the requirements of FIPS 140-2 level 2 or Common Criteria EAL 4+. This means the key pair will be generated in a device, where the private key cannot be exported. This will help to minimize the private key compromise."

 

https://securityboulevard.com/2022/05/ca-browser-forum-updates-requirements-for-code-signing-certificate-private-keys/

 

 

 

As far as I can judge, this is highly disruptive. Our company has multiple developers that work remotely much of the time and we need to be able to rollout software updates in a timely fashion. It would very much interfere with our deployment processes if we are no longer allowed to store the keys on our development machines. 

 

Your opinions on this?

 

 

 

 

 

[Uwe Raabe 2057]

My signing process is part of the build process running at the build server being a VM on an ESXi machine. While I have no problem to connect the security device to the physical machine, I still need to see if and how it can be accessed from the build server VM.

[Lars Fosdal 1792]

That is a conundrum.  Our build VM is on an Azure host - so we don't even have access to the physical VM host. 

Edit: Slightly more informative: https://www.entrust.com/blog/2022/09/ca-browser-forum-updates-requirements-for-code-signing-certificate-private-keys/

[Clément 148]

26 minutes ago, Lars Fosdal said:

That is a conundrum.  Our build VM is on an Azure host - so we don't even have access to the physical VM host. 

Edit: Slightly more informative: https://www.entrust.com/blog/2022/09/ca-browser-forum-updates-requirements-for-code-signing-certificate-private-keys/

Not sure I understood correctly, but
" Effective 1 June 2023, the code signing certificate key pair must be generated and stored in a hardware crypto module that meets or exceeds the requirements of FIPS 140-2 level 2 or Common Criteria EAL 4+. This means the key pair will be generated in a device, where the private key cannot be exported. This will help to minimize the private key compromise. "

This means every signed software in a machine will have a corresponding key stored in crypto module. "Out of memory" comes to my mind.

and

 

" In addition, the CA must verify or ensure the private key was generated in a hardware crypto module using one of the following methods: "

So I need to have a crypto modulo compatible with the one of my customer. I have a bad feeling about this.

Hopefully someone can clarify this

[Lars Fosdal 1792]

I assume the documentation will improve as the support question queue grows 😛

[Bill Meyer 337]

3 hours ago, Lars Fosdal said:

I assume the documentation will improve as the support question queue grows 😛

Documentation? That's so 20th century. 😉

[Angus Robertson 574]

The date for this change was 15 November 2022 but has now been postponed until June 2023.

 

One way to put off the inconvenience of buying a $200 dongle and shipping it around the world is to buy a multiyear certificate now. 

 

I've got a three year certificate from K Software for $209, SSL.com would be $330 for three years, they sell durations up to 10 years, not sure that is allowed now.

 

Angus

[Vincent Parrett 750]

I have been researching this for some time, since it will impact most of our customers. I really do wonder if they have considered the impact this will have on build automation. 

One thing I would point out is do not get a certificate that is issued on a Yubikey - there is absolutely zero way to automate signing - the yubikey uses the windows smart card api and absolutely prompts on every file you sign! So far in my research, it seems only SSL.com (also the cheapest) use Yubikeys - everyone else I have looked at so far uses SafeNet tokens, which do have a work around.  If anyone has a token other than SafeNet or Yuibkey please let me know (brand and where it was issued). I did buy a Yubikey (without a cert) to experiment with but didn't get far with that yet. 

 

https://www.finalbuilder.com/forums/t/signtool-with-ev-certificate-fails/6535/22

 

I have yet to confirm if this work around works from a windows service (which is typically how CI servers run). 

 

More testing to be done in the next few days before my EV cert expires (we are still using our OV cert that expires next year in production).  

[Vincent Parrett 750]

3 hours ago, Angus Robertson said:

they sell durations up to 10 years, not sure that is allowed now.

I suspect this would be wasted money - my guess is OV certificates will eventually be discontinued. 

[Vandrovnik 214]

10 hours ago, Vincent Parrett said:

One thing I would point out is do not get a certificate that is issued on a Yubikey - there is absolutely zero way to automate signing - the yubikey uses the windows smart card api and absolutely prompts on every file you sign!

 

Where do you have to enter the password - on your computer, or directly on the device? On the computer, we could use AutoHotKey, which would enter the password instead of us 🙂

[Vincent Parrett 750]

43 minutes ago, Vandrovnik said:

On the computer, we could use AutoHotKey, which would enter the password instead of us

Good luck doing that when your build process runs from a windows service where you are not logged in. 

[Tommi Prami 130]

I tried to read the articles linked, and it was not very clear will this affect we signing our apps, or is this just when it is done in Azure or so.

And where actually requirement is coming from, what actually will enforce this for us, if will. MEaning how resulted signed app will differ from what we have now?

 

Articles , to me, are written in way that you need lot of background information to actually understand what this all means.

 

-Tee-

[Vincent Parrett 750]

8 minutes ago, Tommi Prami said:

Articles , to me, are written in way that you need lot of background information to actually understand what this all means.

Yup, have to a gree, and it doesn't help that the sites that sell certificates are just plain sh1te - I've never seem so much unhelpful content in one place.

 

Working on a blog post about this, hoping to get it out tonight (it's almost 8pm here) or tomorrow.  I'm trying to edit it down to a reasonable size and reduce the jargon where possible. 

[Angus Robertson 574]

I signed the OpenSSL binaries with a Certum dongle for a while, that prompted for a password for each file which was a pain, then Certum told us the dongle was obsolete and we'd have to buy a new one, so gave up and now using my Comodo/Sectigo certificate.

 

One possible solution is to use OpenSSL to sign Windows PE files, there is project Osslsigncode intended to sign on Linux but can be built for Windows. Another project has a TPM2 provider for OpenSSL that should be able to read keys from dongles, although not checked if the Yubikey is a TPM2.  Not a trivial project, but might be a good solution.

 

Angus

 

[Vandrovnik 214]

1 hour ago, Vincent Parrett said:

Good luck doing that when your build process runs from a windows service where you are not logged in. 

Well, this is not my case, I build locally.

[Vincent Parrett 750]

22 minutes ago, Angus Robertson said:

I signed the OpenSSL binaries with a Certum dongle for a while, that prompted for a password for each file which was a pain, then Certum told us the dongle was obsolete and we'd have to buy a new one, so gave up and now using my Comodo/Sectigo certificate

 

I hadn't seen that one, and based on what I have found out about it so far I'm just going to ignore it.

 

22 minutes ago, Angus Robertson said:

One possible solution is to use OpenSSL to sign Windows PE files, there is project Osslsigncode intended to sign on Linux but can be built for Windows. Another project has a TPM2 provider for OpenSSL that should be able to read keys from dongles, although not checked if the Yubikey is a TPM2.  Not a trivial project, but might be a good solution.

 

 

Nice idea but will likely fail with hardware tokens - the private key stays on the token - so you need client software to allow the code signing tool to access it. 

 

[Angus Robertson 574]

25 minutes ago, Vincent Parrett said:

Nice idea but will likely fail with hardware tokens - the private key stays on the token - so you need client software to allow the code signing tool to access it.

 

Yubico has a SDK for the ubiHSM 2 FIPS dongle which can be used with older OpenSSL releases using engines, then Osslsigncode updates the PE file authenticode signature.

 

TPM2 might be easier, not sure if Yubico supports it.

 

Angus

[Uwe Raabe 2057]

I wonder how the option 

Quote

There is flexibility where the code signing certificate subscriber may use a hardware crypto module which is operated by:

...

• A cloud service, such as AWS or Azure

...

is going to work. Do we all have to send our dongles to Amazon or Microsoft so they can plug them into their servers?

 

There must be a system that allows to deploy the certificate to these services in some way. As they mention a "such as" for AWS or Azure, there should exist something like an API or guide to implement this for other cloud services, too.

[Vincent Parrett 750]

Just now, Uwe Raabe said:

Do we all have to send our dongles to Amazon or Microsoft so they can plug them into their servers?

LOL - no.

1 minute ago, Uwe Raabe said:

There must be a system that allows to deploy the certificate to these services in some way.

There is - you create a certificate request which you send to the CA - they send back a certificate that can only be installed on the HSM that generated the certificate request. The CA websites do a really bad job at explaining things.

[Vincent Parrett 750]

Blogged - https://www.finalbuilder.com/resources/blogs/code-signing-with-usb-tokens

[Angus Robertson 574]

6 minutes ago, Vincent Parrett said:

you create a certificate request which you send to the CA - they send back a certificate that can only be installed on the HSM that generated the certificate request.

That concept I can live with for a local dongle, the major problem I have is with shipping physical dongles around the world each year to be updated. customs do not like USB keys.

 

Angus

 

[Uwe Raabe 2057]

As far as I understand, the dongle is for the system that does the signing, not the one checking the signed exe.

 

The problem I see is that this completely counterfeits the idea of virtualization when the dongle (or HSM) is hardware bound. I can move around my build server and agents, which are all realized as VM. If my build machine breaks, I can easily set up a new one and continue. In case of a dongle I may be able to move it too, as long as it did survive the fire, flood, earth  quake or bomb attack that destroyed the old machine. It creates a new hardware dependency and a single point of failure.

[timfrost 78]

Excellent blog posting from Vincent - many thanks.  It seems that VirtualHere is going to solve the looming problems for me, and that I will be able to continue to use FinalBuilder with a USB dongle for the signing of EXE  and DLL files that has to be done on each release. I also have to be remote from where the USB would be plugged in, and no dongle will have to travel.

[Lars Fosdal 1792]

2 hours ago, Uwe Raabe said:

Do we all have to send our dongles to Amazon or Microsoft so they can plug them into their servers?

DaaS - Dongles as a Service 

[Vincent Parrett 750]

9 hours ago, timfrost said:

Excellent blog posting from Vincent - many thanks.  It seems that VirtualHere is going to solve the looming problems for me, and that I will be able to continue to use FinalBuilder with a USB dongle for the signing of EXE  and DLL files that has to be done on each release. I also have to be remote from where the USB would be plugged in, and no dongle will have to travel.

Thanks. One thing to bear in mind with usb is that it is very sensitve to latency - so not sure how it would work in high latency connections. I have fiber at home and get 6ms pings to the data center where our servers live - which is fine. I suspect anything over 30ms might be a problem for some devices.