Renovate image appears to include expired CA for Let's Encrypt certificates

[epowell]

Context on CA expiration: https://techcrunch.com/2021/09/21/lets-encrypt-root-expiry/

The renovate/renovate image, as used by a renovate-runner instance in our local Gitlab installation, is currently experiencing errors like this when accessing any repository, starting at ~9AM CDT 9/30/2021:

"message": "fatal: unable to access 'https://git.example.com/path/to/repository.git/': server certificate verification failed. CAfile: none CRLfile: none",
             "stack": "Error: fatal: unable to access 'https://git.example.com/path/to/repository.git/': server certificate verification failed. CAfile: none CRLfile: none
    at Object.action (/usr/src/app/node_modules/simple-git/src/lib/plugins/error-detection.plugin.ts:38:28)
    at PluginStore.exec (/usr/src/app/node_modules/simple-git/src/lib/plugins/plugin-store.ts:24:29)
    at /usr/src/app/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:114:40
    at new Promise (<anonymous>)
    at GitExecutorChain.handleTaskData (/usr/src/app/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:111:14)
    at GitExecutorChain.<anonymous> (/usr/src/app/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:88:40)
    at Generator.next (<anonymous>)
    at fulfilled (/usr/src/app/node_modules/simple-git/src/lib/runners/git-executor-chain.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)"

Our Gitlab installation uses a Let's Encrypt issued certificate, rooted in the new "ISRG Root X1" CA that is valid until 2035. Our current server cert is valid until 29 Nov 2021. Inspecting this cert in an up-to-date browser (both Chrome and Firefox) shows a valid certificate and trust chain.

This leads me to conclude that the trusted CAs inside the renovate/renovate image need to update to include an updated list of trusted CAs, including the newer ISRG Root X1 root CA.

[viceice]

Which version of the image are you using?

[viceice]

@rarkins seeing this too now at my bot. 😱

Node is working fine, but git checkout fails.
So i assume something is wrong in Ubuntu image.

Will try to fix later today. 😕

[rarkins]

https://jay.gooby.org/2021/09/30/remove-the-dst-root-ca-x3-crt-from-ubuntu-14-04-lts

[rarkins]

https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

[rarkins]

https://community.letsencrypt.org/t/production-chain-changes/150739

But I still don't understand why we're affected. Neither Ubuntu or git are old

[epowell]

Current image on my install is: renovate/renovate:27.21.0@sha256:c83b8ebf3ca994b04c5a135d03bd5aa3cb813e005f1a0f22b633d8f1d648c2c1

No pending updates from upstream (per Renovate itself anyway :) )

[rarkins]

@viceice can we add this?

sed -i"" 's/mozilla\/DST_Root_CA_X3.crt/!mozilla\/DST_Root_CA_X3.crt/' /etc/ca-certificates.conf
dpkg-reconfigure -fnoninteractive ca-certificates
update-ca-certificates

[viceice]

The issue should be fixed with next base image update in next two or three hours.

Maybe you need to manually update to latest docker digest.

[viceice]

Latest images are working again

[epowell]

Confirmed working with the latest 0.27.25 image on my infrastructure. Thank you for your extremely fast response and diligent work to fix this!

[rarkins]

Special thanks to @viceice for fixing it late at night while on vacation!