Skip to content

Commit

Permalink
feat: Validating admission controller for RuleSet resources (#984)
Browse files Browse the repository at this point in the history
  • Loading branch information
dadrus authored Oct 27, 2023
1 parent 0d9666d commit 3357e57
Show file tree
Hide file tree
Showing 36 changed files with 1,745 additions and 95 deletions.
25 changes: 25 additions & 0 deletions charts/heimdall/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,31 @@ a| `service.management.name`
The name of the port exposed by the k8s Service created for heimdall's management endpoint.
a| `management`

a| `admissionController.labels`

Allows setting additional labels for the `ValidatingWebhookConfiguration` resource used to let the API server communicate with heimdall to validate `RuleSet` resources, before these made available to heimdall for loading.
a| `{}` (empty map)

a| `admissionController.annotations`

Can be used to specify required annotations for the `ValidatingWebhookConfiguration` resource, like e.g. `cert-manager.io/inject-ca-from: <secret name>`, `service.beta.openshift.io/inject-cabundle=true` and alike.
a| `{}` (empty map)

a| `admissionController.namespaceSelector`

Allows specifying a namespaceSelector for the `ValidatingWebhookConfiguration` resource
a| `{}` (empty map)

a| `admissionController.caBundle`

Allows configuration of the `caBundle` in the `ValidatingWebhookConfiguration` resource. Either this one, or a corresponding annotation (see annotations examples above) must be specified if the usage of the validating admission controller is desired. Otherwise, the API server won't be able to communicate with heimdall.
a| `""`

a| `admissionController.timeoutSeconds`

How long the webhook implemented by the admission controller is allowed to run while validating `RuleSet` resources. After the timeout passes, the webhook call will be ignored by the API server resulting in discarding the affected `RuleSet` ressource.
a| `5`

a| `env`

Environment variables, which should be made available to the heimdall deployment. E.g.
Expand Down
8 changes: 7 additions & 1 deletion charts/heimdall/crds/ruleset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,10 @@ spec:
x-kubernetes-validations:
- rule: "has(self.id)"
message: "a rule must have an id defined"
- rule: "has(self.match) && has(self.match.url) && size(self.match.url) > 0"
- rule: "has(self.match) && has(self.match.url)"
message: "a rule must have a url set to match incoming requests"
- rule: "size(self.match.url) > 0"
message: "the match url must be an expression and not be empty"
- rule: "has(self.execute) && size(self.execute) > 0"
message: "execute pipeline is not allowed to be empty"
properties:
Expand All @@ -70,9 +72,11 @@ spec:
url:
description: The url to match
type: string
maxLength: 512
strategy:
description: Strategy to match the url. Can either be regex or glob.
type: string
maxLength: 5
default: glob
enum:
- regex
Expand All @@ -87,6 +91,7 @@ spec:
host:
description: Host and port of the upstream service to forward the request to
type: string
maxLength: 512
rewrite:
description: Configures middlewares to rewrite parts of the URL
type: object
Expand All @@ -99,6 +104,7 @@ spec:
scheme:
description: If you want to overwrite the used HTTP scheme, set it here
type: string
maxLength: 5
strip_path_prefix:
description: If you want to cut a prefix from the URL path, set it here
type: string
Expand Down
46 changes: 46 additions & 0 deletions charts/heimdall/templates/heimdall/admissioncontroller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{{- $rules := default dict .Values.rules }}
{{- $providers := default dict $rules.providers }}
{{- $kubernetes := default dict $providers.kubernetes }}
{{- if $kubernetes.tls }}
# Only active if .Values.rules.providers.kubernetes.tls is configured
{{- $data := dict "Release" .Release "Values" .Values "Chart" .Chart "Component" "admissionController" }}
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: {{ include "heimdall.fullname" $data }}-webhook
namespace: {{ include "heimdall.namespace" $data }}
labels:
{{- include "heimdall.labels" $data | nindent 4 }}
annotations:
{{- toYaml .Values.admissionController.annotations | nindent 4 }}
webhooks:
- name: "admission-controller.heimdall.dadrus.github.com"
admissionReviewVersions: [ "v1" ]
sideEffects: None
timeoutSeconds: {{ .Values.admissionController.timeoutSeconds }}
{{- with .Values.admissionController.namespaceSelector }}
{{- toYaml . | nindent 8 }}
{{- end }}
rules:
- apiGroups: ["heimdall.dadrus.github.com"]
apiVersions: ["v1alpha2"]
operations: ["CREATE", "UPDATE"]
resources: ["rulesets"]
scope: "Namespaced"
matchConditions:
{{- $rules := .Values.rules | default dict }}
{{- $providers := $rules.providers | default dict }}
{{- $kubernetes := $providers.kubernetes | default dict}}
# Match only those rule sets, which relate to the configured auth class
- name: 'auth-class-filter'
expression: 'object.spec.authClassName == {{ default (quote "default") (quote $kubernetes.auth_class) }}'
clientConfig:
{{- with .Values.admissionController.caBundle }}
caBundle: {{ . }}
{{- end }}
service:
namespace: {{ include "heimdall.namespace" $data }}
name: {{ include "heimdall.fullname" $data }}
path: "/validate-ruleset"
port: {{ .Values.service.admissionController.port }}
{{- end }}
8 changes: 8 additions & 0 deletions charts/heimdall/templates/heimdall/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,14 @@ spec:
protocol: TCP
containerPort: {{ .Values.profiling.port }}
{{- end }}
{{- $rules := .Values.rules | default dict }}
{{- $providers := $rules.providers | default dict }}
{{- $kubernetes := $providers.kubernetes | default dict}}
{{- if $kubernetes.tls }}
- name: https-webhook
protocol: TCP
containerPort: 4458
{{- end }}
volumeMounts:
- name: {{ include "heimdall.name" . }}-config-volume
mountPath: /etc/heimdall
Expand Down
9 changes: 9 additions & 0 deletions charts/heimdall/templates/heimdall/service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,14 @@ spec:
protocol: TCP
name: {{ .Values.service.proxy.name }}
{{- end }}
{{- $rules := default dict .Values.rules }}
{{- $providers := default dict $rules.providers }}
{{- $kubernetes := default dict $providers.kubernetes }}
{{- if $kubernetes.tls }}
- port: {{ .Values.service.admissionController.port }}
targetPort: https-webhook
protocol: TCP
name: {{ .Values.service.admissionController.name }}
{{- end }}
selector:
{{- include "heimdall.selectorLabels" $data | nindent 4 }}
24 changes: 24 additions & 0 deletions charts/heimdall/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,30 @@ service:
port: 4457
# Service port name
name: management
# Only used if rules.providers.kubernetes.tls is configured
admissionController:
# Service port
port: 4458
# Service port name
name: admission-controller

# Only used if rules.providers.kubernetes.tls is configured
admissionController:
# Remove the curly braces after 'labels:' if you do want to specify additional labels
labels: { }

# Remove the curly braces after 'annotations:' if you want to specify annotations
# e.g. cert-manager.io/inject-ca-from: <secret name>, or service.beta.openshift.io/inject-cabundle=true
annotations: { }

# Remove the curly braces after 'namespaceSelector:' if you want to specify a namespaceSelector
namespaceSelector: {}

# Set to the required value if you want to specify the caBundle by yourself (see annotations for alternative)
caBundle: ""

# Set to another value if required
timeoutSeconds: 5

# Configures arbitrary environment variables for the deployment
env: { }
Expand Down
8 changes: 7 additions & 1 deletion cmd/validate/test_data/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -222,4 +222,10 @@ rules:
- url: s3://my-bucket/my-rule-set

kubernetes:
auth_class: foo
auth_class: foo
tls:
key_id: foo
key_store:
path: /path/to/pem.file
password: VerySecret!
min_version: TLS1.3
6 changes: 6 additions & 0 deletions docs/content/docs/configuration/reference/reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -389,5 +389,11 @@ rules:
kubernetes:
auth_class: foo
tls:
key_id: foo
key_store:
path: /path/to/pem.file
password: VerySecret!
min_version: TLS1.3
----

35 changes: 32 additions & 3 deletions docs/content/docs/configuration/rules/providers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -224,11 +224,24 @@ The last one instructs heimdall to load rule set from a specific blob, namely a

== Kubernetes

This provider is only supported if heimdall is running within Kubernetes and allows usage of link:{{< relref "#_ruleset_resource" >}}[Rule Set] resources deployed to the same Kubernetes environment. The configuration of this provider goes into the `kubernetes` property and supports the following configuration options:
This provider is only supported if heimdall is running within Kubernetes and allows usage (validation and loading) of link:{{< relref "#_ruleset_resource" >}}[Rule Set] resources deployed to the same Kubernetes environment. The configuration of this provider goes into the `kubernetes` property and supports the following configuration options:

* *`auth_class`*: _string_ (optional)
+
By making use of this property, you can specify which RuleSets should be used by this particular heimdall instance. If specified, heimdall will consider the value of the `authClassName` attribute of each RuleSet deployed to the cluster and load only those rules, which `authClassName` values match the value of `auth_class`. If not set all RuleSets will be used.
By making use of this property, you can specify which RuleSets should be used by this particular heimdall instance. If specified, heimdall will consider the value of the `authClassName` attribute of each RuleSet deployed to the cluster and validate, respectively load only those rules, which `authClassName` values match the value of `auth_class`. If not set all RuleSets will be used.

* *`tls`*: _link:{{< relref "/docs/configuration/reference/types.adoc#_tls" >}}[TLS]_ (optional)
+
If configured, heimdall will start and expose a validating admission controller service on port `4458` listening on all interfaces. This service allows integration with the Kubernetes API server enabling validation of the applied RuleSet resources before these are made available to heimdall for loading. This way you will get a direct feedback about RuleSet issues without the need to look into heimdall logs if a RuleSet could not be loaded (See also link:{{< relref "/openapi/#tag/Validating-Admission-Controller" >}}[API] documentation for more details).
+
To let the Kubernetes API server use the admission controller, there is a need for a properly configured https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#deploy-the-admission-webhook-service[`ValidatingWebhookConfiguration`]. The https://github.com/dadrus/heimdall/tree/main/charts/heimdall[Helm Chart] shipped with heimdall does this automatically as soon as this property is configured. It does however need a `caBundle` to be set or injected. Otherwise, the Kubernetes API server won't trust the configured TLS certificate and won't use the endpoint.

[CAUTION]
====
Since multiple heimdall deployments with different configured `auth_class` names can coexist, RuleSets with mismatching `authClassName` will be ignored by a particular deployment. In addition, Kubernetes API server validation requests for mismatching RuleSets result in a successful response. This behavior is required as otherwise, as soon as the API server receives even a single failed validation response, the affected RuleSet resource will be discarded and not made available for loading to any of the available heimdall deployments.
That also means, if there is no heimdall deployment feeling responsible for the given RuleSet (due to `authClassName` - `auth_class` mismatch), the affected RuleSet will be silently ignored.
====

.Minimal possible configuration
====
Expand All @@ -253,7 +266,23 @@ kubernetes:
----
====

[CAUTION]
.Configuration with `auth_class` set and enabled validating admission controller
====
As with the previous example, the provider is configured to consider only those RuleSets, which `authClassName` is set to `foo`. The admission controller is enabled as well and will validate `RuleSet` resources before these are made available for loading.
[source, yaml]
----
kubernetes:
auth_class: foo
tls:
# below is the minimal required configuration
key_store:
path: /path/to/file.pem
----
====

[NOTE]
====
This provider requires a RuleSet CRD being deployed, otherwise heimdall will not be able to monitor corresponding resources and emit error messages to the log.
Expand Down
Loading

0 comments on commit 3357e57

Please sign in to comment.