[stable/nginx-ingress] Add Validation webhook for nginx ingress contr…

…oller (helm#17230)

* Add Validation webhook for nginx ingress controller

Signed-off-by: LucasBoisserie <lucas.boisserie@gmail.com>

* add tests

Signed-off-by: LucasBoisserie <lucas.boisserie@gmail.com>
Signed-off-by: Ali Sattari <ali.sattari@gmail.com>

stable/nginx-ingress/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: nginx-ingress
-version: 1.20.0
+version: 1.21.0
 appVersion: 0.25.1
 home: https://github.com/kubernetes/ingress-nginx
 description: An nginx Ingress controller that uses ConfigMap to store the nginx configuration.

stable/nginx-ingress/README.md

@@ -143,6 +143,24 @@ Parameter | Description | Default
 `controller.metrics.prometheusRule.additionalLabels` | Additional labels that can be used so prometheusRules will be discovered by Prometheus | `{}`
 `controller.metrics.prometheusRule.namespace` | namespace where prometheusRules resource should be created | `the same namespace as nginx ingress`
 `controller.metrics.prometheusRule.rules` | [rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) to be prometheus in YAML format, check values for an example. | `[]`
+`controller.admissionWebhooks.enabled` | Create Ingress admission webhooks. Validating webhook will check the ingress syntax. | `false`
+`controller.admissionWebhooks.failurePolicy` | Failure policy for admission webhooks | `Fail`
+`controller.admissionWebhooks.port` | Admission webhook port | `8080`
+`controller.admissionWebhooks.service.annotations` | Annotations for admission webhook service | `{}`
+`controller.admissionWebhooks.service.omitClusterIP` | To omit the `clusterIP` from the admission webhook service | `false`
+`controller.admissionWebhooks.service.clusterIP` | cluster IP address to assign to admission webhook service | `""`
+`controller.admissionWebhooks.service.externalIPs` | Admission webhook service external IP addresses | `[]`
+`controller.admissionWebhooks.service.loadBalancerIP` | IP address to assign to load balancer (if supported) | `""`
+`controller.admissionWebhooks.service.loadBalancerSourceRanges` | List of IP CIDRs allowed access to load balancer (if supported) | `[]`
+`controller.admissionWebhooks.service.servicePort` | Admission webhook service port | `443`
+`controller.admissionWebhooks.service.type` | Type of admission webhook service to create | `ClusterIP`
+`controller.admissionWebhooks.patch.enabled` | If true, will use a pre and post install hooks to generate a CA and certificate to use for the prometheus operator tls proxy, and patch the created webhooks with the CA. | `true`
+`controller.admissionWebhooks.patch.image.repository` | Repository to use for the webhook integration jobs | `jettech/kube-webhook-certgen`
+`controller.admissionWebhooks.patch.image.tag` |  Tag to use for the webhook integration jobs | `v1.0.0`
+`controller.admissionWebhooks.patch.image.pullPolicy` | Image pull policy for the webhook integration jobs | `IfNotPresent`
+`controller.admissionWebhooks.patch.priorityClassName` | Priority class for the webhook integration jobs | `""`
+`controller.admissionWebhooks.patch.podAnnotations` | Annotations for the webhook job pods | `{}`
+`controller.admissionWebhooks.patch.nodeSelector` | Node selector for running admission hook patch jobs | `{}`
 `controller.customTemplate.configMapName` | configMap containing a custom nginx template | `""`
 `controller.customTemplate.configMapKey` | configMap key containing the nginx template | `""`
 `controller.addHeaders` | configMap key:value pairs containing [custom headers](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#add-headers) added before sending response to the client | `{}`
@@ -283,6 +301,12 @@ controller:
       domainName: "kubernetes-example.com"
 ```
 
+## Ingress Admission Webhooks
+
+With nginx-ingress-controller version 0.25+, the nginx ingress controller pod exposes an endpoint that will integrate with the `validatingwebhookconfiguration` Kubernetes feature to prevent bad ingress from being added to the cluster.
+
+With nginx-ingress-controller in 0.25.* work only with kubernetes 1.14+, 0.26 fix [this issue](https://github.com/kubernetes/ingress-nginx/pull/4521)
+
 ## Helm error when upgrading: spec.clusterIP: Invalid value: ""
 
 If you are upgrading this chart from a version between 0.31.0 and 1.2.2 then you may get an error like this:

stable/nginx-ingress/ci/deamonset-webhook-and-psp-values.yaml

@@ -0,0 +1,7 @@
+controller:
+  kind: DaemonSet
+  admissionWebhooks:
+    enabled: true
+
+podSecurityPolicy:
+  enabled: true

stable/nginx-ingress/ci/deamonset-webhook-values.yaml

@@ -0,0 +1,4 @@
+controller:
+  kind: DaemonSet
+  admissionWebhooks:
+    enabled: true

stable/nginx-ingress/ci/deployment-webhook-and-psp-values.yaml

@@ -0,0 +1,6 @@
+controller:
+  admissionWebhooks:
+    enabled: true
+
+podSecurityPolicy:
+  enabled: true

stable/nginx-ingress/ci/deployment-webhook-values.yaml

@@ -0,0 +1,3 @@
+controller:
+  admissionWebhooks:
+    enabled: true

stable/nginx-ingress/templates/admission-webhooks/job-patch/clusterrole.yaml

@@ -0,0 +1,30 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name:  {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+rules:
+  - apiGroups:
+      - admissionregistration.k8s.io
+    resources:
+      - validatingwebhookconfigurations
+    verbs:
+      - get
+      - update
+{{- if .Values.podSecurityPolicy.enabled }}
+  - apiGroups: ['extensions']
+    resources: ['podsecuritypolicies']
+    verbs:     ['use']
+    resourceNames:
+    - {{ template "nginx-ingress.fullname" . }}-admission
+{{- end }}
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/clusterrolebinding.yaml

@@ -0,0 +1,23 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name:  {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: {{ template "nginx-ingress.fullname" . }}-admission
+subjects:
+  - kind: ServiceAccount
+    name: {{ template "nginx-ingress.fullname" . }}-admission
+    namespace: {{ .Release.Namespace }}
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/job-createSecret.yaml

@@ -0,0 +1,52 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "nginx-ingress.fullname" . }}-admission-create
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+spec:
+  ttlSecondsAfterFinished: 0
+  template:
+    metadata:
+      name: {{ template "nginx-ingress.fullname" . }}-admission-create
+{{- with .Values.controller.admissionWebhooks.patch.podAnnotations }}
+      annotations:
+{{ toYaml .  | indent 8 }}
+{{- end }}
+      labels:
+        app: {{ template "nginx-ingress.name" . }}
+        chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+        component: "{{ .Values.controller.name }}"
+        heritage: {{ .Release.Service }}
+        release: {{ .Release.Name }}
+    spec:
+      {{- if .Values.controller.admissionWebhooks.patch.priorityClassName }}
+      priorityClassName: {{ .Values.controller.admissionWebhooks.patch.priorityClassName }}
+      {{- end }}
+      containers:
+        - name: create
+          image: {{ .Values.controller.admissionWebhooks.patch.image.repository }}:{{ .Values.controller.admissionWebhooks.patch.image.tag }}
+          imagePullPolicy: {{ .Values.controller.admissionWebhooks.patch.image.pullPolicy }}
+          args:
+            - create
+            - --host={{ template "nginx-ingress.controller.fullname" . }}-admission,{{ template "nginx-ingress.controller.fullname" . }}-admission.{{ .Release.Namespace }}.svc
+            - --namespace={{ .Release.Namespace }}
+            - --secret-name={{ template "nginx-ingress.fullname". }}-admission
+      restartPolicy: OnFailure
+      serviceAccountName: {{ template "nginx-ingress.fullname" . }}-admission
+      {{- with .Values.controller.admissionWebhooks.patch.nodeSelector }}
+      nodeSelector:
+{{ toYaml . | indent 8 }}
+      {{- end }}
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 2000
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/job-patchWebhook.yaml

@@ -0,0 +1,54 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ template "nginx-ingress.fullname" . }}-admission-patch
+  annotations:
+    "helm.sh/hook": post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+spec:
+  ttlSecondsAfterFinished: 0
+  template:
+    metadata:
+      name:   {{ template "nginx-ingress.fullname" . }}-admission-patch
+{{- with .Values.controller.admissionWebhooks.patch.podAnnotations }}
+      annotations:
+{{ toYaml .  | indent 8 }}
+{{- end }}
+      labels:
+        app: {{ template "nginx-ingress.name" . }}
+        chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+        component: "{{ .Values.controller.name }}"
+        heritage: {{ .Release.Service }}
+        release: {{ .Release.Name }}
+    spec:
+      {{- if .Values.controller.admissionWebhooks.patch.priorityClassName }}
+      priorityClassName: {{ .Values.controller.admissionWebhooks.patch.priorityClassName }}
+      {{- end }}
+      containers:
+        - name: patch
+          image: {{ .Values.controller.admissionWebhooks.patch.image.repository }}:{{ .Values.controller.admissionWebhooks.patch.image.tag }}
+          imagePullPolicy: {{ .Values.controller.admissionWebhooks.patch.pullPolicy }}
+          args:
+            - patch
+            - --webhook-name={{ template "nginx-ingress.fullname" . }}-admission
+            - --namespace={{ .Release.Namespace }}
+            - --patch-mutating=false
+            - --secret-name={{ template "nginx-ingress.fullname". }}-admission
+            - --patch-failure-policy={{ .Values.controller.admissionWebhooks.failurePolicy }}
+      restartPolicy: OnFailure
+      serviceAccountName: {{ template "nginx-ingress.fullname" . }}-admission
+      {{- with .Values.controller.admissionWebhooks.patch.nodeSelector }}
+      nodeSelector:
+{{ toYaml . | indent 8 }}
+      {{- end }}
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 2000
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/psp.yaml

@@ -0,0 +1,39 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled .Values.podSecurityPolicy.enabled }}
+apiVersion: policy/v1beta1
+kind: PodSecurityPolicy
+metadata:
+  name: {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+spec:
+  allowPrivilegeEscalation: false
+  fsGroup:
+    ranges:
+    - max: 65535
+      min: 1
+    rule: MustRunAs
+  requiredDropCapabilities:
+  - ALL
+  runAsUser:
+    rule: MustRunAsNonRoot
+  seLinux:
+    rule: RunAsAny
+  supplementalGroups:
+    ranges:
+    - max: 65535
+      min: 1
+    rule: MustRunAs
+  volumes:
+  - configMap
+  - emptyDir
+  - projected
+  - secret
+  - downwardAPI
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/role.yaml

@@ -0,0 +1,23 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name:  {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    verbs:
+      - get
+      - create
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/rolebinding.yaml

@@ -0,0 +1,23 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name:  {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: {{ template "nginx-ingress.fullname" . }}-admission
+subjects:
+  - kind: ServiceAccount
+    name: {{ template "nginx-ingress.fullname" . }}-admission
+    namespace: {{ .Release.Namespace }}
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/job-patch/serviceaccount.yaml

@@ -0,0 +1,15 @@
+{{- if and .Values.controller.admissionWebhooks.enabled .Values.controller.admissionWebhooks.patch.enabled }}
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name:  {{ template "nginx-ingress.fullname" . }}-admission
+  annotations:
+    "helm.sh/hook": pre-install,pre-upgrade,post-install,post-upgrade
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+  labels:
+    app: {{ template "nginx-ingress.name" . }}
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "{{ .Values.controller.name }}"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+{{- end }}

stable/nginx-ingress/templates/admission-webhooks/validating-webhook.yaml

@@ -0,0 +1,31 @@
+{{- if .Values.controller.admissionWebhooks.enabled }}
+apiVersion: admissionregistration.k8s.io/v1beta1
+kind: ValidatingWebhookConfiguration
+metadata:
+  labels:
+    app: {{ template "nginx-ingress.name" . }}-admission
+    chart: {{ .Chart.Name }}-{{ .Chart.Version }}
+    component: "admission-webhook"
+    heritage: {{ .Release.Service }}
+    release: {{ .Release.Name }}
+  name: {{ template "nginx-ingress.fullname" . }}-admission
+webhooks:
+  - name: validate.nginx.ingress.kubernetes.io
+    rules:
+      - apiGroups:
+          - extensions
+          - networking.k8s.io
+        apiVersions:
+          - v1beta1
+        operations:
+          - CREATE
+          - UPDATE
+        resources:
+          - ingresses
+    failurePolicy: Fail
+    clientConfig:
+      service:
+        namespace: {{ .Release.Namespace }}
+        name: {{ template "nginx-ingress.controller.fullname" . }}-admission
+        path: /extensions/v1beta1/ingresses
+{{- end }}