Open
Description
Describe the bug
When upgrading to 1.31.1 preview, kubelet fails to start with the removed flag --keep-terminated-pod-volumes to kubelet. Thus failing to register nodes to AKS within specified timeout (15m according to logs)
To Reproduce
Steps to reproduce the behavior:
- Start cluster at 1.30.x
- Upgrade cluster to "1.31.1 Preview". (Which I believe is set to GA soon also)
Expected behavior
Cluster upgrades, and starts new nodes in VMSS, that are enrolled into AKS Cluster.
The new nodes boot, and join the cluster.
Environment (please complete the following information):
- CLI Version [e.g. 3.22] N/A (Portal initiated operation)
- Kubernetes version [1.31.1] (Upgrading from 1.30.5
- Node image: "osSku": "AzureLinux", AKSAzureLinux-V2gen2-202410.09.0
Additional context
Full output of /var/log/messages from a running node (That is not joined into AKS Cluster:
2024-10-25T07:00:03.137300+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137384+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --container-log-max-size has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137464+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --enforce-node-allocatable has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137545+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --event-qps has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137628+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --eviction-hard has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137711+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --feature-gates has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137796+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --image-gc-high-threshold has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.137876+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: Flag --image-gc-low-threshold has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
2024-10-25T07:00:03.138020+00:00 aks-nodepool2-85190728-vmss000055 kubelet[1147205]: E1025 07:00:03.137990 1147205 run.go:72] \"command failed\" err=\"failed to parse kubelet flag: unknown flag: --keep-terminated-pod-volumes\"
2024-10-25T07:00:03.144864+00:00 aks-nodepool2-85190728-vmss000055 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
2024-10-25T07:00:03.145096+00:00 aks-nodepool2-85190728-vmss000055 systemd[1]: kubelet.service: Failed with result 'exit-code'.