☂️-Issue for "Support for Kubernetes v1.23”

[rfranzke]

/area open-source
/kind enhancement
/priority 3
/topology garden seed shoot

Kubernetes v1.23 is about to be released (CW49), find the changelog here: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md

Work can be started and pushed to the kubernetes-v1.23 branch. There should be similar branches in the provider extension repositories in case the work has been started already.

---

gardener/hyperkube

• Build new images for v1.23

gardener/gardener

• Support for Kubernetes v1.23 #5255
  • Release-independent tasks
    • Allow instantiating v1.23 Kubernetes clients and update README.md and docs/usage/supported_k8s_versions.md
    • Update feature gates in https://github.com/gardener/gardener/blob/master/pkg/utils/kubernetes/featuregates.go
    • Adapt cipher list: add chacha20 ciphers to meet DISA STIG compliance #4823
    • Bump used Kubernetes version for local shoot and local e2e test
  • Release-specific tasks
    • Change the cgroup driver from cgroupfs to systemd for clusters on k8s >= 1.23
    • Adapt to deprecated seccomp annotations, ref Mention seccomp annotation removal in v1.25 kubernetes/kubernetes#104389
    • autoscaling/v2beta2 was promoted to autoscaling/v2
    • Prevent usage of deprecated flags for KCM and KSCHED, ref refactor: disable insecure serving in kube-scheduler kubernetes/kubernetes#96345
    • Use v1beta3 config version for KSCHED, ref Scheduling v1beta3 kubernetes/kubernetes#104251
• Revert "Use systemd as cgroup driver for shoots >= 1.23" #5324

gardener/gardener-extension-provider-alicloud

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-alicloud#422
• Support for Kubernetes v1.23 gardener-extension-provider-alicloud#426
  • Update README.md with support for 1.23
  • Replace deprecated kubelet flags with recommended configuration #4739
  • cloud provider configuration will be deprecated for kubelet prameters gardener-extension-provider-alicloud#202

gardener/gardener-extension-provider-aws

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-aws#478
  • Revendor and release gardener/cloud-provider-aws with v1.23.0 sources
• Support for Kubernetes v1.23 gardener-extension-provider-aws#480
  • Update README.md with support for 1.23
  • Replace deprecated kubelet flags with recommended configuration #4739
  • Update cloud-provider-aws images

gardener/gardener-extension-provider-azure

• Prerequisites
  • Revendor gardener/gardener@v1.39.4 gardener-extension-provider-azure#428
  • Revendor gardener/cloud-provider-azure with v1.23.0 sources, see ☂️-Issue for "Support for Kubernetes v1.23” #5102 (comment)
• Support for Kubernetes v1.23 gardener-extension-provider-azure#431
  • Update README.md with support for 1.23
  • Update cloud-provider-azure images (Use upstream cloud-provider-azure release, see also [UPDATE] use offical image from ms gardener-extension-provider-azure#376)
  • Replace deprecated kubelet flags with recommended configuration #4739

gardener/gardener-extension-provider-gcp

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-gcp#381
  • Revendor and release gardener/cloud-provider-gcp with v1.23.0 sources
• Support for Kubernetes v1.23 gardener-extension-provider-gcp#383
  • Update README.md with support for 1.23
  • Update cloud-provider-gcp images
  • Replace deprecated kubelet flags with recommended configuration #4739

gardener/gardener-extension-provider-openstack

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-openstack#383
• Support for Kubernetes v1.23 gardener-extension-provider-openstack#386
  • Update README.md with support for 1.23
  • Use latest version of OpenStack cloud-controller-manager
  • Replace deprecated kubelet flags with recommended configuration #4739

gardener/gardener-extension-provider-equinix-metal

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-equinix-metal#188
• Support for Kubernetes v1.23 gardener-extension-provider-equinix-metal#190
  • Update README.md with support for 1.23
  • Replace deprecated kubelet flags with recommended configuration #4739

gardener/gardener-extension-provider-vsphere

• Prerequisites
  • Revendor gardener/gardener@v1.39 gardener-extension-provider-vsphere#200
• Support for Kubernetes v1.23 gardener-extension-provider-vsphere#201
  • Update README.md with support for 1.23
  • Replace deprecated kubelet flags with recommended configuration #4739

[ialidzhikov]

One minor comment wrt:

• ❌ Revendor gardener/cloud-provider-azure with v1.23.0 sources

There was the agreement to start using the out-of-tree CCM for Azure starting with K8s v1.23 - ref gardener/gardener-extension-provider-azure#376 (comment). In such case, I guess we can skip the update of gardener/cloud-provider-azure with v1.23.0 sources.

[vpnachev]

What do you think about changing the cgroup driver from cgroupfs to systemd for clusters on k8s >= 1.23?
I am asking because:

• systemd is the recommended cgroup driver, ref

  We have seen cases in the field where nodes that are configured to use cgroupfs for kubelet and docker and systemd for the rest can become unstable under resource pressure. Changing the settings such that docker and kubelet use systems as a cgroup-driver stabilized the systems.

• cgroupfs does not support cgroup v2 which the OS vendors start to consider to be the default

[rfranzke]

I think your proposal sounds reasonable @vpnachev

[rfranzke]

/assign @BeckerMax @rfranzke