Add migration k8s job

[fanny-jiang]

What this PR does / why we need it:

• Updates datadog chart with a k8s job template for migrating installed helm release to a DDA manifest
• Changes include new configmap templates for mounting the helm release values and helm-dda mapper file
• Also adds some no-op updates to some default chart configs to harmonize with the operator's defaults

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

• fixes #

Special notes for your reviewer:

This branch includes changes from: #2112

Testing (in kind cluster)

1. Install datadog chart and override operator image to use 1.21.0-rc.2:

datadog:
  apiKeyExistingSecret: datadog-secret
  appKeyExistingSecret: datadog-secret
  clusterName: kind-kind
  kubelet:
    tlsVerify: false
  clusterChecks:
    enabled: true

operator:
  image:
    tag: 1.21.0-rc.2

helm install datadog ./charts/datadog -f values.yaml

2. Datadog operator pod should be running

3. Enable migration preview:

datadog:
  apiKeyExistingSecret: datadog-secret
  appKeyExistingSecret: datadog-secret
  clusterName: kind-kind
  kubelet:
    tlsVerify: false
  clusterChecks:
    enabled: true
    
  operator: 
    migration:
      preview: true

operator:
  image:
    tag: 1.21.0-rc.2

helm upgrade datadog ./charts/datadog -f values.yaml

4. Check that the migration job pod is running and completes successfully (should be only 1 container running mapper binary)

5. Run the kubectl command shown in the helm upgrade notes output to view the mapping logs output. Check that the command works and review the logs output--there shouldn't be any errors (warnings are ok).

6. Enable migration:

datadog:
  apiKeyExistingSecret: datadog-secret
  appKeyExistingSecret: datadog-secret
  clusterName: kind-kind
  kubelet:
    tlsVerify: false
  clusterChecks:
    enabled: true
    
  operator: 
    migration:
      # preview: true
      enabled: true

operator:
  image:
    tag: 1.21.0-rc.2
  datadogCRDs:
    keepCrds: true

7. Check that the migration job pod is running and completes successfully (should be 2 containers; 1 running mapper binary, another running kubectl)

8. Run the kubectl command shown in the helm upgrade notes output to view the migration logs output. Check that the command works and review the logs output--there shouldn't be any errors (warnings are ok). It should also confirm that the DDA has been created and annotated:

datadogagent.datadoghq.com/datadog created
datadogagent.datadoghq.com/datadog annotated

9. Check that a DatadogAgent custom resource has been created successfully: kubectl get dd and the name is the same as the {{datadog.fullname}}.

10. Check that a new daemonset has been created and its annotations indicate that it's being managed by the operator. The old helm-managed daemonset has been deleted.

kubectl get daemonset datadog-agent -n <NAMESPACE> \
  -o jsonpath='{.metadata.labels.app\.kubernetes\.io/managed-by}{"\n"}'

Should return datadog-operator

11. The new daemonset should also have the datadoghq.com/migrated: true annotation:

kubectl get daemonset datadog-agent -n <NAMESPACE> \
  -o jsonpath='{.metadata.labels.agent\.datadoghq\.com/migrated}{"\n"}'

12. Check that a rollingUpdate has started and helm-created agent pods are being replaced with operator-managed agent pods.
13. Check that new deployments have been created for the cluster-agent and cluster-checks-runner. The helm-managed deployments have been deleted. Cluster-agent and cluster-checks-runner pods have also been replaced with operator-managed pods

kubectl get deployment <DCA_OR_CCR_NAME> -n <NAMESPACE> \
  -o jsonpath='{.metadata.labels.app\.kubernetes\.io/managed-by}{"\n"}'

Should return datadog-operator

To test complete migration to the operator chart:

1. Verify that the datadogagent CRD has the helm.sh/resource-policy: keep annotation:

kubectl get crd datadogagents.datadoghq.com -o jsonpath='{.metadata.annotations}'

2. Install datadog operator chart with the --take-ownership flag:

helm install operator datadog/datadog-operator --devel --take-ownership

The DDA CRD annotations should now reflect ownership by the new operator chart release:

kubectl get crd datadogagents.datadoghq.com -o jsonpath='{.metadata.annotations}'
> {"controller-gen.kubebuilder.io/version":"v0.17.3","helm.sh/resource-policy":"keep","meta.helm.sh/release-name":"operator","meta.helm.sh/release-namespace":"default"}%

3. Uninstall the datadog chart:

helm uninstall datadog

The DDA CRD should not get deleted and you should get this output:

These resources were kept due to the resource policy:
[CustomResourceDefinition] datadogagents.datadoghq.com

4. The agent pods should remain running. There may be some downtime in the DCA, but that should resolve once the new operator pod gains the new lease and becomes leader.

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• Chart Version semver bump label added (use <chartName>/minor-version, <chartName>/patch-version, or <chartName>/no-version-bump)
• For datadog or datadog-operator chart or value changes, update the test baselines (run: make update-test-baselines)

GitHub CI takes care of the below, but are still required:

• Documentation has been updated with helm-docs (run: .github/helm-docs.sh)
• CHANGELOG.md has been updated
• Variables are documented in the README.md