[operator] Manage Dashboard Web Terminal Controller Manager (gardener…

…#9646)

* Documentation

* Add component boilerplate

* Integrate terminal component into `Garden` controller

* Virtual garden access secret

* Server certificate

It's signed by the runtime CA which is auto-rotated each `30d`, so this server cert is also auto-rotated regularly.

* Service

* ConfigMap

* Deployment

* RBAC

* CRD

* ServiceMonitor

* Webhook Configurations

* Do not use Golang type for ConfigMap

Without this, the resulting JSON-marshalled config looks like this:

```
{
  "kind": "ControllerManagerConfiguration",
  "apiVersion": "dashboard.gardener.cloud/v1alpha1",
  "server": {
    "healthProbes": {
      "bindAddress": "",
      "port": 8081
    },
    "metrics": {
      "bindAddress": "",
      "port": 8443
    }
  },
  "controllers": {
    "terminal": {
      "maxConcurrentReconciles": 0,
      "maxConcurrentReconcilesPerNamespace": 0,
      "tokenRequestExpirationSeconds": null
    },
    "terminalHeartbeat": {
      "maxConcurrentReconciles": 0,
      "timeToLive": {
        "Duration": 0
      }
    },
    "serviceAccount": {
      "maxConcurrentReconciles": 0,
      "allowedServiceAccountNames": null
    }
  },
  "webhooks": {
    "terminalValidation": {
      "maxObjectSize": 0
    }
  },
  "honourServiceAccountRefHostCluster": false,
  "leaderElection": {
    "leaderElect": true,
    "leaseDuration": "0s",
    "renewDeadline": "0s",
    "retryPeriod": "0s",
    "resourceLock": "",
    "resourceName": "",
    "resourceNamespace": "kube-system"
  }
}
```

This makes the controller fail to start:

```
{"level":"error","ts":"2024-04-23T13:30:40Z","logger":"setup","msg":"error reading config","error":"controllers.terminal.maxConcurrentReconciles: Invalid value: 0: must be 1 or greater","stacktrace":"main.main\n\t/workspace/main.go:80\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271"}
```

* Enable `terminalEnabled` in frontend config when configured

Otherwise, people have to provide a frontend configuration where this is set to true.

* PodDisruptionBudget

* VerticalPodAutoscaler

* Address PR review feedback

docs/concepts/operator.md

@@ -147,8 +147,8 @@ The Gardener control plane components are:
 - `gardener-controller-manager`
 - `gardener-scheduler`
 
-Besides those, the optional [Gardener Dashboard](https://github.com/gardener/dashboard) will also get deployed when `.spec.virtualCluster.gardener.gardenerDashboard` is set.
-You can read more about it and its configuration in [this section](#dashboard).
+Besides those, the optional [Gardener Dashboard](https://github.com/gardener/dashboard) (and the [controller for web terminals](https://github.com/gardener/terminal-controller-manager)) can also get deployed when `.spec.virtualCluster.gardener.gardenerDashboard` (or `.spec.virtualCluster.gardener.gardenerDashboard.terminal`, respectively) is set.
+You can read more about it and its configuration in [this section](#gardener-dashboard).
 
 The reconciler also manages a few observability-related components (more planned as part of [GEP-19](../proposals/19-migrating-observability-stack-to-operators.md)):
 
@@ -258,6 +258,7 @@ This section highlights the most prominent fields:
   If this field is not provided and there is no `webhookSecret` key in the referenced secret, it will be implicitly defaulted to `15m`.
   The dashboard will use this to regularly poll the GitHub API for updates on issues.
 - `terminal`: This enables the web terminal feature, read more about it [here](https://github.com/gardener/dashboard/blob/master/docs/operations/webterminals.md).
+  When set, the [`terminal-controller-manager`](https://github.com/gardener/terminal-controller-manager) will be deployed to the runtime cluster.
   The `allowedHosts` field is explained [here](https://github.com/gardener/dashboard/blob/master/docs/operations/webterminals.md#configuration).
   The `container` section allows you to specify a container image and a description that should be used for the web terminals.
 

docs/development/priority-classes.md

@@ -25,7 +25,7 @@ When using the `gardener-operator` for managing the garden runtime and virtual c
 | `gardener-garden-system-500`      | 999999500 | `virtual-garden-etcd-events`, `virtual-garden-etcd-main`, `virtual-garden-kube-apiserver`, `gardener-apiserver`                                                                                                      |
 | `gardener-garden-system-400`      | 999999400 | `virtual-garden-gardener-resource-manager`, `gardener-admission-controller`                                                                                                                                          |
 | `gardener-garden-system-300`      | 999999300 | `virtual-garden-kube-controller-manager`, `vpa-admission-controller`, `etcd-druid`, `nginx-ingress-controller`                                                                                                       |
-| `gardener-garden-system-200`      | 999999200 | `vpa-recommender`, `vpa-updater`, `hvpa-controller`, `gardener-scheduler`, `gardener-controller-manager`, `gardener-dashboard`                                                                                       |
+| `gardener-garden-system-200`      | 999999200 | `vpa-recommender`, `vpa-updater`, `hvpa-controller`, `gardener-scheduler`, `gardener-controller-manager`, `gardener-dashboard`, `terminal-controller-manager`                                                        |
 | `gardener-garden-system-100`      | 999999100 | `fluent-operator`, `fluent-bit`, `gardener-metrics-exporter`, `kube-state-metrics`, `plutono`, `vali`, `prometheus-operator`, `alertmanager-garden`, `prometheus-garden`, `blackbox-exporter`, `prometheus-longterm` |
 
 ## Seed Clusters

go.mod

@@ -14,6 +14,7 @@ require (
 	github.com/gardener/etcd-druid v0.22.0
 	github.com/gardener/hvpa-controller/api v0.15.0
 	github.com/gardener/machine-controller-manager v0.53.0
+	github.com/gardener/terminal-controller-manager v0.32.0
 	github.com/go-logr/logr v1.4.1
 	github.com/go-test/deep v1.1.0
 	github.com/gogo/protobuf v1.3.2
@@ -196,11 +197,10 @@ require (
 	go.uber.org/multierr v1.11.0 // indirect
 	golang.org/x/mod v0.17.0 // indirect
 	golang.org/x/net v0.24.0 // indirect
-	golang.org/x/oauth2 v0.18.0 // indirect
+	golang.org/x/oauth2 v0.19.0 // indirect
 	golang.org/x/sync v0.7.0 // indirect
 	golang.org/x/sys v0.19.0 // indirect
 	golang.org/x/term v0.19.0 // indirect
-	google.golang.org/appengine v1.6.8 // indirect
 	google.golang.org/genproto v0.0.0-20240116215550-a9fa1716bcac // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20240116215550-a9fa1716bcac // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20240116215550-a9fa1716bcac // indirect

go.sum

@@ -150,6 +150,8 @@ github.com/gardener/hvpa-controller/api v0.15.0 h1:igsalL5Z6kFMn1+Kv1Eq0cRjYW+4o
 github.com/gardener/hvpa-controller/api v0.15.0/go.mod h1:fqb4wNrQLESDKpm7ppXyCM2Gvx96wRlLL35aH0ge07U=
 github.com/gardener/machine-controller-manager v0.53.0 h1:g2O0F7nEYZ9LjyPY6Gew8+q0n+rU88deexNq5k8CKks=
 github.com/gardener/machine-controller-manager v0.53.0/go.mod h1:XWXHaTy32TU0qmLjWqOgtw8NncdB0HfFzXhUUrcpr7Y=
+github.com/gardener/terminal-controller-manager v0.32.0 h1:NZlnJJ+NRk2ozfWw7mso7ByknqujjhmoEvb1qHLbqrQ=
+github.com/gardener/terminal-controller-manager v0.32.0/go.mod h1:W14iaprpNTyItEjUjHvBh07ZUVWFaIwew7HhX9W3u7Q=
 github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU=
@@ -226,7 +228,6 @@ github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD
 github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
 github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
 github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
-github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
 github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
 github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
@@ -620,8 +621,8 @@ golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4Iltr
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.18.0 h1:09qnuIAgzdx1XplqJvW6CQqMCtGZykZWcXzPMPUusvI=
-golang.org/x/oauth2 v0.18.0/go.mod h1:Wf7knwG0MPoWIMMBgFlEaSUDaKskp0dCfrlJRJXbBi8=
+golang.org/x/oauth2 v0.19.0 h1:9+E/EZBCbTLNrbN35fHv/a/d/mOBatymz1zbtQrXpIg=
+golang.org/x/oauth2 v0.19.0/go.mod h1:vYi7skDa1x015PmRRYZ7+s1cWyPgrPiSYRe4rnsexc8=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -669,7 +670,6 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
 golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
 golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
@@ -725,8 +725,6 @@ google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7
 google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
 google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
-google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
 google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
 google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
 google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
@@ -764,7 +762,6 @@ google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpAD
 google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
-google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
 google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
 google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
 google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=

hack/generate-crds.sh

@@ -75,6 +75,9 @@ get_group_package () {
   "cert.gardener.cloud")
     echo "github.com/gardener/cert-management/pkg/apis/cert/v1alpha1"
     ;;
+  "dashboard.gardener.cloud")
+    echo "github.com/gardener/terminal-controller-manager/api/v1alpha1"
+    ;;
   *)
     >&2 echo "unknown group $1"
     return 1
@@ -93,6 +96,7 @@ generate_all_groups () {
   generate_group monitoring.coreos.com_v1beta1
   generate_group monitoring.coreos.com_v1alpha1
   generate_group machine.sapcloud.io
+  generate_group dashboard.gardener.cloud
 }
 
 generate_group () {

imagevector/images.yaml

@@ -38,10 +38,16 @@ images:
   repository: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent
   resourceId:
     name: node-agent
+
+# Gardener Dashboard components
 - name: gardener-dashboard
   sourceRepository: github.com/gardener/dashboard
   repository: europe-docker.pkg.dev/gardener-project/releases/gardener/dashboard
   tag: "1.74.1"
+- name: terminal-controller-manager
+  sourceRepository: github.com/gardener/terminal-controller-manager
+  repository: europe-docker.pkg.dev/gardener-project/releases/gardener/terminal-controller-manager
+  tag: "v0.32.0"
 
 # Seed bootstrap
 - name: pause-container

pkg/component/gardener/dashboard/configmaps.go → pkg/component/gardener/dashboard/configmap.go

@@ -62,6 +62,20 @@ func (g *gardenerDashboard) configMap(ctx context.Context) (*corev1.ConfigMap, e
 		loginCfg = &config.LoginConfig{}
 	)
 
+	if frontendConfig != nil {
+		cfg.Frontend = frontendConfig
+
+		if v, ok := frontendConfig["landingPageUrl"]; ok {
+			loginCfg.LandingPageURL = v.(string)
+		}
+		if v, ok := frontendConfig["branding"]; ok {
+			loginCfg.Branding = v.(map[string]interface{})
+		}
+		if v, ok := frontendConfig["themes"]; ok {
+			loginCfg.Themes = v.(map[string]interface{})
+		}
+	}
+
 	if g.values.EnableTokenLogin {
 		loginCfg.LoginTypes = append(loginCfg.LoginTypes, "token")
 	}
@@ -87,6 +101,14 @@ func (g *gardenerDashboard) configMap(ctx context.Context) (*corev1.ConfigMap, e
 				Namespace: metav1.NamespaceSystem,
 			}}},
 		}
+
+		if cfg.Frontend == nil {
+			cfg.Frontend = make(map[string]interface{})
+		}
+		if cfg.Frontend["features"] == nil {
+			cfg.Frontend["features"] = make(map[string]bool)
+		}
+		cfg.Frontend["features"].(map[string]bool)["terminalEnabled"] = true
 	}
 
 	if g.values.OIDC != nil {
@@ -133,20 +155,6 @@ func (g *gardenerDashboard) configMap(ctx context.Context) (*corev1.ConfigMap, e
 		}
 	}
 
-	if frontendConfig != nil {
-		cfg.Frontend = frontendConfig
-
-		if v, ok := frontendConfig["landingPageUrl"]; ok {
-			loginCfg.LandingPageURL = v.(string)
-		}
-		if v, ok := frontendConfig["branding"]; ok {
-			loginCfg.Branding = v.(map[string]interface{})
-		}
-		if v, ok := frontendConfig["themes"]; ok {
-			loginCfg.Themes = v.(map[string]interface{})
-		}
-	}
-
 	rawConfig := &bytes.Buffer{}
 	yamlEncoder := yaml.NewEncoder(rawConfig)
 	yamlEncoder.SetIndent(2)

pkg/component/gardener/dashboard/dashboard_test.go

@@ -235,6 +235,9 @@ terminal:
       serviceAccountRef:
         name: dashboard-terminal-admin
         namespace: kube-system
+frontend:
+  features:
+    terminalEnabled: true
 `
 			}