Description
We have developed a CRD and an operator, and are working on creating a new major version of that CRD. After following the instructions in the kubebuilder book for setting up the webhooks but the book doesn't describe how to use the conversion webhooks in the integration tests. We have managed to hack together a way to get the conversion webhooks running in integration tests, but it's not that easy to do, and didn't have documentation.
Below are the different changes that we needed to make to run our conversion webhook in our integration tests:
1. Point to a different definition of our CRD
The kubebuilder book recommends configuring your envtest environment to the CRDs located in the /config/crd/bases folder of a project. However, the conversion webhook section of the CRD is added by kustomize, so doesn't live in that directory. This means that before running our integration tests, we needed to run kustomize, save the outputted CRDs, and point our envtest Environment to that directory. This is a manual step needed that would be nice to automate away.
2. Update the kustomized section of our CRD
By default, kustomize is run using the following command ./bin/kustomize build config/default to create our CRDs. That will then add a section like the following to our CRD:
conversion:
strategy: Webhook
webhook:
clientConfig:
service:
name: webhook-service
namespace: default
path: /convert
conversionReviewVersions:
- v1beta1
- v1beta2
However, this doesn't work for running the conversion webhook in our integration tests because our controller is running locally on the computer running the integration tests, so instead of pointing to a service living in the envtest, we need to point to our machine. Therefore, we needed to update that section to instead look like:
conversion:
strategy: Webhook
webhook:
clientConfig:
url: "https://localhost:9443/convert"
conversionReviewVersions:
- v1beta1
- v1beta2
This is another manual change that would be great to not need to figure out and then do.
3. Creating certs for the conversion webhook
The APIServer requires that the conversion webhook uses https, so we needed to manually create a cert and key signed for the SAN of DNS:localhost. We then needed to add an argument to the manager that we create in suite_test.go for the CertDir and point to the directory holding our cert and key. Finally, we needed to update the conversion webhook of our CRD to look like this:
conversion:
strategy: Webhook
webhook:
clientConfig:
caBundle: <Base64_Encoded_Cert>
url: "https://localhost:9443/convert"
conversionReviewVersions:
- v1beta1
- v1beta2
It would be great to better support the use case of testing conversion webhooks in integration tests by adding documentation describing what needs to be done to set up and run conversion webhooks in integration tests and automating or somehow alleviating the manual steps described above necessary to get conversion webhooks running in integration tests. Please let me know if there is any other information I can provide!