Skip to content

Conversation

DustinChaloupka
Copy link
Contributor

Fixes #4792

Updating the main.go template for the webhook and metrics server certificate handling. Instead of creating a separate CertWatcher, this supplies the three fields to the options. When controller-runtime has those three values, it will set up its own CertWatcher.

This allows non-leader replicas to properly reload the cert and key when updated, which is necessary for the webhook and metrics server that end up serving the endpoints even when they are not leaders.

This was tested with our own currently running controller by deleting the cert-manager managed Certificate resource, letting it recreate a new one, which updated the Secret and then the file in the Pods. Reloading the certificate and key can be seen in the logs:

2025-08-12T15:10:02Z	INFO	setup	Initializing webhook certificate watcher using provided certificates	{"webhook-cert-path": "/tmp/k8s-webhook-server/serving-certs", "webhook-cert-name": "tls.crt", "webhook-cert-key": "tls.key"}
2025-08-12T15:10:02Z	INFO	setup	Initializing metrics certificate watcher using provided certificates	{"metrics-cert-path": "/tmp/k8s-metrics-server/metrics-certs", "metrics-cert-name": "tls.crt", "metrics-cert-key": "tls.key"}
2025-08-12T15:10:02Z	INFO	controller-runtime.builder	Registering a mutating webhook	{"GVK": "my-group.example.com/v1alpha1, kind=Thing", "path": "/mutate-my-group-example-com-v1alpha1-thing"}
2025-08-12T15:10:02Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-my-group-example-com-v1alpha1-thing"}
2025-08-12T15:10:02Z	INFO	controller-runtime.builder	Registering a validating webhook	{"GVK": "my-group.example.com/v1alpha1, kind=Thing", "path": "/validate-my-group-example-com-v1alpha1-thing"}
2025-08-12T15:10:02Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-my-group-example-com-v1alpha1-thing"}
2025-08-12T15:10:02Z	INFO	setup	starting manager
2025-08-12T15:10:02Z	INFO	controller-runtime.metrics	Starting metrics server
2025-08-12T15:10:02Z	INFO	setup	disabling http/2
2025-08-12T15:10:02Z	INFO	starting server	{"name": "health probe", "addr": "[::]:8081"}
2025-08-12T15:10:02Z	INFO	controller-runtime.webhook	Starting webhook server
2025-08-12T15:10:02Z	INFO	setup	disabling http/2
I0812 15:10:02.713385       1 leaderelection.go:257] attempting to acquire leader lease my-namespace/84d4f733.example.com...
2025-08-12T15:10:02Z	INFO	controller-runtime.certwatcher	Updated current TLS certificate
2025-08-12T15:10:02Z	INFO	controller-runtime.metrics	Serving metrics server	{"bindAddress": ":8443", "secure": true}
2025-08-12T15:10:02Z	INFO	controller-runtime.certwatcher	Updated current TLS certificate
2025-08-12T15:10:02Z	INFO	controller-runtime.certwatcher	Starting certificate poll+watcher	{"interval": "10s"}
2025-08-12T15:10:02Z	INFO	controller-runtime.webhook	Serving webhook server	{"host": "", "port": 9443}
2025-08-12T15:10:02Z	INFO	controller-runtime.certwatcher	Starting certificate poll+watcher	{"interval": "10s"}
2025-08-12T15:15:50Z	DEBUG	controller-runtime.certwatcher	certificate event	{"event": "CHMOD         \"/tmp/k8s-webhook-server/serving-certs/tls.crt\""}
2025-08-12T15:15:50Z	DEBUG	controller-runtime.certwatcher	certificate event	{"event": "REMOVE        \"/tmp/k8s-webhook-server/serving-certs/tls.crt\""}
2025-08-12T15:15:50Z	DEBUG	controller-runtime.certwatcher	certificate event	{"event": "CHMOD         \"/tmp/k8s-webhook-server/serving-certs/tls.key\""}

…tion provided by controller-runtime to do the same
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 12, 2025
@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 12, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @DustinChaloupka. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@DustinChaloupka DustinChaloupka changed the title (go/v4): Remove the certWatcher in favor of use internal implementation provided by controller-runtime to do the same 🐛 (go/v4): Remove the certWatcher in favor of use internal implementation provided by controller-runtime to do the same Aug 12, 2025
@camilamacedo86 camilamacedo86 changed the title 🐛 (go/v4): Remove the certWatcher in favor of use internal implementation provided by controller-runtime to do the same 🐛 (go/v4): Replace custom CertWatcher with controller-runtime’s built-in implementation for webhook and metrics server cert reloads. Ensures non-leader replicas update certificates correctly. Aug 12, 2025
@camilamacedo86
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 13, 2025
@camilamacedo86 camilamacedo86 changed the title 🐛 (go/v4): Replace custom CertWatcher with controller-runtime’s built-in implementation for webhook and metrics server cert reloads. Ensures non-leader replicas update certificates correctly. 🐛 (go/v4): Replace custom CertWatcher with controller-runtime’s built-in implementation for webhook and metrics server cert reloads. Ensures non-leader replicas update certificates correctly. Aug 13, 2025
Copy link
Member

@camilamacedo86 camilamacedo86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 13, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: camilamacedo86, DustinChaloupka

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 13, 2025
@k8s-ci-robot k8s-ci-robot merged commit 43f1f5e into kubernetes-sigs:master Aug 13, 2025
35 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Certwatcher only detects cert changes on leader

3 participants