Repository navigation
Transition from Digicert keylocker to Azure Trusted Signing #4036
Description
Activity
Here's how I think we'll solve this problem:
- Ensure that build machines have the required binaries, namely:
nuget install Microsoft.Windows.SDK.BuildTools -Version 10.0.26100.1742 -x nuget install Microsoft.Trusted.Signing.Client -Version 1.0.60 -x
- Ensure that build machines have a
trusted_code_signing.jsonin a well-defined path
{ "Endpoint": "https://wus2.codesigning.azure.net/", "CodeSigningAccountName": "TBD", "CertificateProfileName": "tbd" }-
Ensure that build machines have
AZURE_CLIENT_ID,AZURE_CLIENT_SECRET,AZURE_TENANT_IDset. We also need to know the path toAzure.CodeSigning.Dlib.dlland tobin\10.0.26100.0\x64\signtool.exe(it needs to be that version or newer). -
Then, we replace https://github.com/nodejs/node/blob/main/tools/sign.bat with:
@echo off @REM From March 2024, we use Azure Trusted Signing for code signing. @REM Release CI machines are configured to have it in the PATH so this can be used safely. path/to/signtool.exe sign /tr "http://timestamp.acs.microsoft.com" /td sha256 /fd sha256 /v /dlib %AZURE_CODE_SIGNING_DLIB% /dmdf %AZURE_METADATA_JSON% %1 if not ERRORLEVEL 1 ( echo Successfully signed %1 using signtool exit /b 0 ) echo Could not sign %1 using signtool exit /b 1
I have set up an App registration with Azure that is tied to the OpenJS Trusted signing account, so everything is prepared on the Azure side of things.
Since it appears as though the windows release machines are not controlled by ansible, I'm not sure what the next step is to get the secrets onto the windows release machines.
@StefanStojanovic would the NodeJS onepassword work as a mechanism to share these, or shall I stash these in the secrets repo?
(Side note: we have 432 signatures remaining as of April 16th)
Hey all, ClangCL is done. I'll probably have to help with the V8 update, but I can start working on this from next Monday (OOF before that). With that in mind, if there is something I should go through first (e.g. some docs, or anything like that), let me know here and I'll start with that.
@StefanStojanovic would the NodeJS onepassword work as a mechanism to share these, or shall I stash these in the secrets repo?
@ryanaslett Digicert provided a CLI to safely store secrets, so we used that. Does Azure provide something like that?
So If I understand correctly the secrets for digicert (the ones that allow auth into their vault) are stored locally on the machines using Window Credential Manager (similar to how the secrets for the OSX machines are stored locally in apple keychain).
I think it would be more ideal if we set those secrets in jenkins and exposed them as environment variables to the build process.
Essentially AZURE_CLIENT_SECRET can be put into the release jobs for windows builds and azure will use those (https://learn.microsoft.com/en-us/dotnet/api/azure.identity.environmentcredential?view=azure-dotnet)
The contents of trusted_code_signing.json should be as follows:
{ "Endpoint": "https://eus.codesigning.azure.net", "CodeSigningAccountName": "OpenJS-CodeSigning", "CertificateProfileName": "NodeJS", }I have added the following secrets to the build environments on ci-release builds:
AZURE_TENANT_ID
AZURE_CLIENT_ID
AZURE_CLIENT_SECRETwhich should allow the signtool to understand how to authenticate to azure.
@StefanStojanovic if you'd like to test outside of the release environment, I shared them in the secrets repo.
Im pretty sure that Felix's instructions (#4036 (comment)) are the way we do this (It roughly follows the docs: https://learn.microsoft.com/en-us/azure/trusted-signing/how-to-signing-integrations#set-up-signtool-to-use-trusted-signing) (though I didnt check the version numbers etc)
@ryanaslett I was able to sign the binaries manually with the data from the secrets repo. Based on that I configured CI machines and made changes in the Node repo needed for signing. However, when I tried testing it in the release CI, it failed. The build can be found here. Based on the log, and the env vars, I think there is probably some issue with passing the
AZURE_...variables. When I set them directly in the env vars on the machine, the signing passed. Could you please look into that and let me know about your findings?Whoops. I added the three credentials, but accidentally assigned the wrong one to the client_ID:
I've fixed that, and am re-running that job.
Splendid. It appears to have signed this time: https://ci-release.nodejs.org/job/iojs+release/nodes=vs2022_clang-x64/10977/console
00:53:57.801 Trusted Signing 00:53:57.801 00:53:57.801 Version: 1.0.68 00:53:57.801 00:53:57.801 "Metadata": { 00:53:57.801 "Endpoint": "https://eus.codesigning.azure.net/", 00:53:57.801 "CodeSigningAccountName": "OpenJS-CodeSigning", 00:53:57.801 "CertificateProfileName": "NodeJS", 00:53:58.967 "ExcludeCredentials": [] 00:53:58.968 } 00:54:00.040 00:54:00.041 Submitting digest for signing... 00:54:00.041 00:54:00.559 OperationId cf6866d7-7350-43a1-9ca7-ff9a9799cd49: InProgress 00:54:00.559 00:54:00.561 Signing completed with status 'Succeeded' in 2.2386901s 00:54:00.561 00:54:00.561 Successfully signed: node-v25.0.0-testf46e3ead2efedc23ab9f38ab36ada692ecd72437-x64.msi 00:54:00.577 00:54:35.359 00:54:35.359 Number of files successfully Signed: 1 00:54:35.362 00:54:35.362 Number of warnings: 0 00:54:35.364 00:54:35.364 Number of errors: 0 00:54:35.367 00:54:35.367 Successfully signed node-v25.0.0-testf46e3ead2efedc23ab9f38ab36ada692ecd72437-x64.msi using signtoolOkay, there appears to be one issue:
The cert appears to be valid for 3 days. I'll investigate on the azure side of things.
(Edit: I may be misunderstanding how this works, and the cert is able to autorenew)
I was misunderstanding this. The cert/signature combination works because the signature is timestamped within the window that the cert is valid. So we should be good to go. (Though other eyes confirming that would give me more confidence)
- added a commit that references this issue
on Jun 5, 2025 - added a commit that references this issue
on Jun 6, 2025 This looks like its working.
The console output for the most recent nightly builds have the following:
Trusted Signing Version: 1.0.68 "Metadata": { "Endpoint": "https://eus.codesigning.azure.net/", "CodeSigningAccountName": "OpenJS-CodeSigning", "CertificateProfileName": "NodeJS", "ExcludeCredentials": [] } Submitting digest for signing... OperationId 7dea2e97-196c-4320-9930-3b38e00ea768: InProgress Signing completed with status 'Succeeded' in 2.1539244s Successfully signed: node-v25.0.0-nightly2025060625f2f27a55-arm64.msi Number of files successfully Signed: 1 Number of warnings: 0 Number of errors: 0 Successfully signed node-v25.0.0-nightly2025060625f2f27a55-arm64.msi using signtoolDigital Signatures show:
Hooray!
Does the old signing still work, e.g. for nodejs/node#58588? Or will will need to immediately land the change for that? (We'd usually prefer the change to go out first in a current release (i.e. Node.js 24.x) and then later in the LTS release lines.
- added 2 commits that reference this issue
on Jun 10, 2025 - added a commit that references this issue
on Jun 10, 2025 - added 2 commits that reference this issue
on Jun 10, 2025 I do not know if changes were made to the machines that prevent the old signing from working, but it should, if we need to revert the v20 backport to get the release out.
- added a commit that references this issue
on Aug 14, 2025 - added 2 commits that reference this issue
on Aug 25, 2025
Our digicert keylocker certificates have a limited number of signatures available (598 as of this issue).
We'll need to ensure continuity in signing windows releases (nightly/canary/full releases).
Azure trusted signing is a more cost effective mechanism for signing code going forward.
The OpenJS foundation has established a Trusted Signing account as per: https://learn.microsoft.com/en-us/azure/trusted-signing/quickstart?tabs=registerrp-portal%2Caccount-portal%2Corgvalidation%2Ccertificateprofile-portal%2Cdeleteresources-portal to allow OpenJS projects to use our Identity to sign windows binaries.
The rough estimate is that we have a couple of months worth of signatures before we run out, (worst case 75 days).
So the next steps to get this addressed: