v2.23.0
Deprecation / Removal
- Ubuntu 16 and 18 are no longer tested (#10107, @MrFreezeex)
- Drop support for ansible-core 2.11 and update tests dependencies (#10034, @MrFreezeex)
- Drop Kubernetes 1.24 support (#10234, @MrFreezeex)
Feature / Major Changes
- Make kubernetes v1.27.5 default (#10392, @mzaian)
- Add kubernetes v1.27.4 (#10359, @mzaian)
- Add Kubernetes 1.27.2 (#9976, @mzaian)
- Add hashes for 1.27.3 1.26.6, 1.25.11 (#10220, @mzaian)
- Add hashes for 1.27.4 1.26.7, 1.25.12 (#10300, @mzaian)
- Add CPU Management Policies on the Node (#10309, @yankay)
- Add Debian 12(bookworm) support (#10221, @tu1h)
- Add
download.timeout
to update download timeout value (#10149, @yjqg6666) - Add corresponding coredns versions to all the supported kubernetes releases. (#10233, @mzaian)
- Add growpart azure enabled (#10241, @pedro-peter)
- Add ingressClass resource for ingress_nginx by default (#10091, @peschmae)
- Add kubelet topology manager policy on the node (
kubelet_topology_manager_scope
andkubelet_topoloy_manager_policy
) (#10370, @tu1h) - Add labels to kube-vip static pods (#10139, @liupeng0518)
- Add node_taints to aws_inventory script (#10170, @mstoetzer)
- Add option to set
SSL_CERT_FILE
for offline installation using custom CA for https proxy (#10215, @HappyFX) - Add terraform support for NIFCLOUD (#10227, @ystkfujii)
- Add the huawei cloud controller as external cloud controller (#10198, @dabeck)
- Show detected ansible version when it isn't compatible with kubespray (#10109, @jcpunk)
- Allow to override etcd listen-metrics-urls configuration (using
etcd_listen_metrics_urls
variable) (#10332, @forselli-stratio) - Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
- Permit custom names for API server lb/proxy containers (#10166, @jcpunk)
- Permit skipping helm update (#10169, @jcpunk)
- Split defaults main file into 2 files (checksums and version) (#10121, @electrocucaracha)
- System upgrade for Debian-family nodes is available with system_upgrade=true (#10184, @sathieu)
- Update download_hash.sh script (#10120, @electrocucaracha)
- Use a uniform way to get the local path of the binaries (#10211, @ErikJiang)
- Disable fapolicyd service (#10081, @epif4nio)
- Upgrade the load balancer ( nginx and haproxy ) image version to Nginx 1.25, Haproxy 2.8. (#10409, @yankay)
- [etcd] Default version to 3.5.7 for kubernetes 1.27 (#10410, @mzaian)
Applications
- [argocd] update argocd to v2.7.4 (#10226, @mzaian)
- [argocd] update argocd to v2.8.0 (#10364, @mzaian)
- [argocd] Add argocd_install_url option to allow changing argocd url (#10176, @liupeng0518)
- [helm] upgrade to 3.12.1 (#10225, @mzaian)
- [helm] upgrade to 3.12.3 (#10365, @mzaian)
- [helm] add python dependency check for helm-apps (#10192, @palmeXx)
- [krew] add krew_no_upgrade_check (#10175, @liupeng0518)
- [coredns] Bump coredns version to 1.10.1 (#10199, @eminaktas)
- [coredns] Bump nodelocaldns version to 1.22.20 (#10200, @eminaktas)
- [cert-manager] This introduces a new variable for the cert-manager implementation that will allow one to pass in extra arguments to the cert-manager controller.(#10049, @phunyguy)
- Update Helm (v3.12.2) / Skopeo (v1.13.0) and yq (v4.34.2) (#10295, @tu1h)
- Upgrade many tool versions (Helm, crun, kata, youki, gvisor, skopeo, Calico, Cilium etc...) (#9798, @electrocucaracha)
- [local_path_provisioner] Fix invalid podhelper yaml (#10237, @MrFreezeex)
- Update metrics server to v0.6.4 (#10400, @mzaian)
Container-Managers
- [containerd] Make containerd 1.7.5 default (#10397, @mzaian)
- [containerd] Support containerd v1.7.2 (#10219, @Dentrax)
- [containerd] Support containerd 1.7.3 (#10368, @mzaian)
- [containerd] containerd config_path enable mirrors config using new variable
containerd_registries_mirrors
(deprecate and removecontainerd_insecure_registries
for containrd andnerdctl_extra_flags
andinsecure_registry
setting for nerdctl (#10196, @yckaolalala) - [crio] Add crio_insecure_registries option for specifying insecure_registries of crio (#10142, @qlijin)
- [crio]
runroot
now needs to be setup in storage.conf instead of crio.conf (#10372, @floryut) - [crio] Fix etcdctl copy operation (#10242, @ErikJiang)
- [Kata] Set/keep owner/group root/root when unarchiving kata-containers (#10338, @rybnico)
- [youki] Fix youki binary download url (not requiring 'v' in version) (#10337, @ErikJiang)
Network
- [calico] Use configmap to configure calico cni config (#10177, @cyclinder)
- [calico] Update calico v3.25.2 (#10414, @mzaian)
- [calico] Add calico version to v3.26.0 (#10224, @mzaian)
- [calico] Add calico version to v3.26.1 (#10235, @mzaian)
- [calico] Clean up calicoctl_alternate_download_url and calicoctl.mirrors (#10271, @yckaolalala)
- [cilium] Add custom rules to clusterrole for cilium operator (#10267, @jeremythuon)
- [cilium] Upgrade to version 1.13.4 (#10269, @yulng)
- [Cilium] Do not mount tls when 'cilium_hubble_tls_generate' is false (#10357, @charlychiu)
- [Cilium] Update cilium to 1.13.3 (#10158, @jcpunk)
- [flannel] Only create /var/lib/calico when needed (#10156, @jcpunk)
- [flannel] Bump flannel version to v0.22.0 and flannel-cni-plugin version to v1.1.2. Also, changes flannel repository from flannelcni to flannel (#10205, @eminaktas)
- [flannel] Remove unused flannel_cni_download_url (#10188, @oomichi)
- [kube-ovn]: update version v1.11.5 (#10125, @yankay)
- [multus] Fix loop_control template error when item is None (#10347, @nicolas-goudry)
API Change
- Unless the pod security standard versions are changed on intentionally, as default it will be the same major version with Kubernetes version. (#10210, @ugur99)
- Upgrade ansible to 7.0 and ansible-core to 2.14.x (#10190, @MrFreezeex)
⚠️ (See Notes 2)
Documentation
- Add github container registry (
github_image_repo
) to docs/offline-environment.md (#10265, @blackliner) - Update doc for ansible-core 2.14 support and clarify issues running older python versions (#10261, @MrFreezeex)
- Update links for aws_alb_ingress_controller (#10264, @kundan2707)
- Update links in ingress-controller and kuberentes-apps (#10239, @vaibhav2107)
- Update Calico to lowercase and fix broken calico link in README (#10232, @Xieql)
- Document containerd command to restart nginx-proxy container when adding control plane node (#10406, @nicolas-goudry)
Failing Test
- Increase metallb wait timeout from 30sec to 2min (#10260, @MrFreezeex)
- Update CentOS 7 image and test fedora 37 and 38 instead of fedora 35 and 36 (#10108, @MrFreezeex)
Bug or Regression
- Fix Dockerfile for newest directory layout (#10128, @dabeck)
- Fix Flatcar bootstrap issues (yaml module missing and ntp issue) (#10363, @tenni-paws)
- Fix argocd install not working using the kubespray docker image (#10371, @cortex3)
- Fix correctly mount ssl ca directories (#9794, @maxime1907)
- Fix etcdctl copy operation (#10230, @ErikJiang)
- Fix gce-pd-csi driver (#10208, @ashishsinghdev)
- Fix grep command without -w option causing prefix matched while adding one etcd member (#10291, @yangsenzk)
- Fix hcloud-cloud-controller-manager not working in certain setups (#10297, @cortex3)
- Fix helm (kubelet-csr-approver) installation on redhat distro (#10204, @MrFreezeex)
- Fix kubelet-csr-approver usage with upgrade-cluster.yml and missing package with helm role (#10165, @j4m3s-s)
- Fix nginxingress-class template (missing newline) (#10174, @richard-fairthorne)
- Fix problem migration problem with k8s 1.27 (#10136, @batazor)
- Fix reset_confirmation not working when inputing correct value (#10288, @somewho)
- Fix wrong path in manage-offline-files script (#9886, @Medosopher)
- Fix an issue where using Rocky Linux 8 as OS for Vagrant for testing purposes causing etcd to fail on start. (#10252, @nltimv)
- Fix ansible-lint galaxy rule (#10277, @MrFreezeex)
- Fix ansible-lint key-order error (#10314, @MrFreezeex)
- Fix outdated tag and experimental ansible-lint rules (#10254, @MrFreezeex)
- Fix dockerfile build error (#10127, @yankay)
- Fix metrics-server deployment to run with kubernetes 1.26+ (#10183, @mzaian)
- Fix undefined
reset_confirmation_prompt
variable in reset play (#10303, @Mishavint) - Fix CIS Kubernetes V1.23 Benchmark item number 4.1.9 to enhance security (Change kubelet-config.yaml and kubelet.env file permissions from 640 to 600) (#10304, @satandyh)
- Fix parsing of RHSM proxy configuration (#10228, @tmurakam)
- Fix var-spacing ansible rule (#10266, @MrFreezeex)
- Fix specify owner to kube_owner in task of copy cni plugins (#10407, @NierYYDS)
- Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8)
- Fix recover_control_plane playbook (also add debian 12 with cilium as a new nightly test) (#10411, @floryut)
- Fix nameserver inline comments in /etc/resolv.conf (#10415, @yankay)
- Added
systemd_resolved_disable_stub_listener
variable to disable systemd-resolved's stub listener, defaults totrue
on Flatcar. (#9875, @cosandr) - Remove
auto_attach
andsyspurpose
in RHEL subscription Organization ID/Activation Key registration. (#10258, @yckaolalala) - Replace "crio_packages" with "crio_bin_files" (#10182, @yckaolalala)
- Update MetalLB deployment, wait for resource. (#9995, @Jeroen0494)
- Upgrade ansible to 7.0 and ansible-core to 2.14.x in Dockerfile (#10259, @yckaolalala)
- Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8)
⚠️ (See Notes 1) - Change maximal_ansible_version to 2.15(exclusive) (#10395, @yankay)
- Install etcdutl file by default (#10385, @liupeng0518)
Other (Cleanup or Flake)
- [CI] Add CI VM for debian12 (#10222, @yankay)
- [CI] Removes Ansible reinstall from build pipeline (#10032, @luksi1)
- [CI] cleanup stale packet namespace automatically (#10245, @MrFreezeex)
- [CI] fix tf-elastx_cleanup fail (#10133, @yankay)
- [CI] Sanitize branch name in testing before using it in kubernetes label for packet-ci (#10315, @MrFreezeex)
- Add an exception for youki in download_hash script (#10346, @ErikJiang)
- Drop support for Kubernetes 1.24.x (move min version to 1.25.x) (#10126, @yankay)
- Ensure host entries from /etc/host are absent when
populate_inventory_to_hosts_file
is false (#10144, @rptaylor) - Exclude terraform.tfstate backups in .gitignore (#10216, @rptaylor)
- Ping is no longer reported as a changed task (#10160, @jcpunk)
- Reading mounted volumes no longer considered a changed task (#10161, @jcpunk)
- Resolve ansible-lint name errors (#10253, @MrFreezeex)
- Update KUBESPRAY_VERSION for v2.22.1 (#10201, @yankay)
Supported Components
- Core
- kubernetes v1.27.5
- etcd v3.5.7
- docker v20.10 (see note)
- containerd v1.7.5
- cri-o v1.27 (experimental: see CRI-O Note. Only on fedora, ubuntu and centos based OS)
- Network Plugin
- cni-plugins v1.2.0
- calico v3.25.2
- cilium v1.13.4
- flannel v0.22.0
- kube-ovn v1.11.5
- kube-router v1.5.1
- multus v3.8
- weave v2.8.1
- kube-vip v0.5.12
- Application
- cert-manager v1.11.1
- coredns v1.10.1
- ingress-nginx v1.8.1
- krew v0.4.4
- argocd v2.8.0
- helm v3.12.3
- metallb v0.13.9
- registry v2.8.1
- Storage Plugin
- cephfs-provisioner v2.1.0-k8s1.11
- rbd-provisioner v2.1.1-k8s1.11
- aws-ebs-csi-plugin v0.5.0
- azure-csi-plugin v1.10.0
- cinder-csi-plugin v1.22.0
- gcp-pd-csi-plugin v1.9.2
- local-path-provisioner v0.0.24
- local-volume-provisioner v2.5.0
Known issues
N/A
Notes
- Variable kubelet_topoloy_manager_policy change to kubelet_topology_manager_policy, please update your inventory
- Upgrade ansible to 7.0 and ansible-core to 2.14.x
⚠️ breaking change : containerd config_path enable mirrors config using new variablecontainerd_registries_mirrors
(#10196, @yckaolalala)