forked from gardener/gardener
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add tolerations for Project and Shoot resources
- Loading branch information
Showing
67 changed files
with
5,249 additions
and
1,218 deletions.
There are no files selected for viewing
16 changes: 16 additions & 0 deletions
16
.../gardener/controlplane/charts/runtime/templates/apiserver/configmap-admission-config.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{{- if and .Values.global.apiserver.enabled .Values.global.apiserver.admissionConfig }} | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: gardener-apiserver-admission-config | ||
namespace: garden | ||
labels: | ||
app: gardener | ||
role: apiserver | ||
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" | ||
release: "{{ .Release.Name }}" | ||
heritage: "{{ .Release.Service }}" | ||
data: | ||
configuration.yaml: |- | ||
{{ .Values.global.apiserver.admissionConfig | indent 4 }} | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Taints and Tolerations for `Seed`s and `Shoot`s | ||
|
||
Similar to [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) for `Node`s and `Pod`s in Kubernetes, the `Seed` resource supports specifying taints (`.spec.taints`) while the `Shoot` resource supports specifying tolerations (`.spec.tolerations`). | ||
The feature is used to control scheduling to seeds as well as decisions whether a shoot can use a certain seed. | ||
|
||
Compared to Kubernetes, Gardener's taints and tolerations are very much down-stripped right now and have some behavioral differences. | ||
Please read the following explanations carefully if you plan to use it. | ||
|
||
## Scheduling | ||
|
||
When scheduling a new shoot then the gardener-scheduler will filter all seed candidates whose taints are not tolerated by the shoot. | ||
As Gardener's taints/tolerations don't support `effect`s yet you can compare this behaviour with using a `NoSchedule` effect taint in Kubernetes. | ||
|
||
Be reminded that taints/tolerations are no means to define any affinity or selection for seeds - please use `.spec.seedSelector` in the `Shoot` to state such desires. | ||
|
||
⚠️ Please note that - unlike how it's implemented in Kubernetes - a certain seed cluster **may** only be used when the shoot tolerates **all** the seed's taints. | ||
This means that specifying `.spec.seedName` for a seed whose taints are not tolerated will make the gardener-apiserver rejecting the request. | ||
|
||
Consequently, the taints/tolerations feature can be used as means to restrict usage of certain seeds. | ||
|
||
## Toleration Defaults and Whitelist | ||
|
||
The `Project` resource features a `.spec.tolerations` object that may carry `defaults` and a `whitelist`. | ||
The corresponding `ShootTolerationRestriction` admission plugin (cf. Kubernetes' `PodTolerationRestriction` admission plugin) is responsible for evaluating these settings during creation/update of `Shoot`s. | ||
|
||
### Whitelist | ||
|
||
If a shoot gets created or updated with tolerations then it is validated that only those tolerations may be used which were added to either a) the `Project`'s `.spec.tolerations.whitelist`, or b) to the global whitelist in the `ShootTolerationRestriction`'s admission config (see [this example](https://github.com/rfranzke/gardener/blob/feature/tolerations/example/20-admissionconfig.yaml#L7-L14)). | ||
|
||
### Defaults | ||
|
||
If a shoot gets created then the default tolerations specified in both the `Project`'s `.spec.tolerations.defaults` and global default list in the `ShootTolerationRestriction` admission plugin's configuration will be added to the `.spec.tolerations` of the `Shoot` (unless it already specifies a certain key). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
--- | ||
apiVersion: apiserver.k8s.io/v1alpha1 | ||
kind: AdmissionConfiguration | ||
plugins: | ||
- name: ShootTolerationRestriction | ||
configuration: | ||
apiVersion: shoottolerationrestriction.admission.gardener.cloud/v1alpha1 | ||
kind: Configuration | ||
defaults: | ||
- key: foo | ||
whitelist: | ||
- key: foo | ||
- key: bar | ||
value: baz |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.