[Telemetry Operator] Keep webhook CA bundle stable over Pod restart and webhook modifications

[chrkl]

Description
Dependent on kyma-project/telemetry-manager#36

The telemetry-operator generates a new self-signed certificate for the webhook server in the init phase and patches the ValidatingWebhookConfiguration with the new CA bundle on every Pod start. This might cause problems during a rolling restart of the operator pod where the ValidatingWebhookConfiguration is already updated with a new CA bundle, but requests are still served by the old Pod.

The init logic should be turned into a proper reconcilation of the webhook using a long-living CA certificate stored in a dedicated secret.

Criteria

• The telemetry-operator reconciles a secret containing a long-living CA bundle
• The operator will refresh the CA bundle if it is about to expire
• The operator will reconcile the webhook by leveraging the ca-bundle from the secret, with that it will be generated initially but also if it gets deleted/edited for any reasons
• The operator will generate a server-side certificate
• The operator deletes the webhook on deletion of the module CR
• webhook is removed from kustomize resources

[chrkl]

PR kyma-project/telemetry-manager#173 introduced the following changes:

• A long-living CA bundle for the webhook is used (stored in a secret and rotated yearly)
• Short-living server certificates are signed with the CA and used for the webhook server
• The webhook configuration and CA bundle are owed by the telemetry CR and deleted when the module is deleted
• The webhook configuration was never installed by kustomize, no changes needed here

[chrkl]

PR kyma-project/telemetry-manager#192 changed the behaviour to rotate the webhook server certificate only when it is about to expire. Also re-creating the certificate in the case of a missing module CR is ensured.