Open
Description
Is your feature request related to a problem? Please describe.
Currently trying to bring a single node with CPUManager static policy with the following k0s.yaml:
apiVersion: k0s.k0sproject.io/v1beta1
kind: ClusterConfig
metadata:
creationTimestamp: null
name: k0s
spec:
api:
address: 192.168.100.63
k0sApiPort: 9443
port: 6443
sans:
- 192.168.100.63
- fe80::be24:11ff:fe0c:7c87
controllerManager: {}
extensions:
helm:
charts: null
concurrencyLevel: 5
repositories: null
storage:
create_default_storage_class: false
type: external_storage
installConfig:
users:
etcdUser: etcd
kineUser: kube-apiserver
konnectivityUser: konnectivity-server
kubeAPIserverUser: kube-apiserver
kubeSchedulerUser: kube-scheduler
konnectivity:
adminPort: 8133
agentPort: 8132
network:
calico: null
clusterDomain: cluster.local
dualStack: {}
kubeProxy:
iptables:
minSyncPeriod: 0s
syncPeriod: 0s
ipvs:
minSyncPeriod: 0s
syncPeriod: 0s
tcpFinTimeout: 0s
tcpTimeout: 0s
udpTimeout: 0s
metricsBindAddress: 0.0.0.0:10249
mode: iptables
kuberouter:
autoMTU: true
hairpin: Enabled
ipMasq: false
metricsPort: 8080
mtu: 0
peerRouterASNs: ""
peerRouterIPs: ""
nodeLocalLoadBalancing:
envoyProxy:
apiServerBindPort: 7443
konnectivityServerBindPort: 7132
type: EnvoyProxy
podCIDR: 10.244.0.0/16
provider: kuberouter
serviceCIDR: 10.96.0.0/12
scheduler: {}
storage:
etcd:
externalCluster: null
peerAddress: 192.168.100.63
type: etcd
telemetry:
enabled: true
workerProfiles:
- name: custom-cpu
values:
cpuManagerPolicy: static
reservedSystemCPUs: "0-5"
I used this command for the installation: k0s install controller --profile custom-cpu --single -c /etc/k0s/k0s.yaml
But k0s is adding a conflicting parameter that won't allow the CPUManager policy to be applied, below is the eventual kubelet-config.yaml:
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous: {}
webhook:
cacheTTL: 0s
x509:
clientCAFile: /var/lib/k0s/pki/ca.crt
authorization:
webhook:
cacheAuthorizedTTL: 0s
cacheUnauthorizedTTL: 0s
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
containerRuntimeEndpoint: unix:///run/k0s/containerd.sock
cpuManagerPolicy: static
cpuManagerReconcilePeriod: 0s
eventRecordQPS: 0
evictionPressureTransitionPeriod: 0s
failSwapOn: false
fileCheckFrequency: 0s
httpCheckFrequency: 0s
imageMaximumGCAge: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
kubeReservedCgroup: system.slice
kubeletCgroups: /system.slice/containerd.service
logging:
flushFrequency: 0
options:
json:
infoBufferSize: "0"
verbosity: 0
memorySwap: {}
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
registerWithTaints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
reservedSystemCPUs: 0-5
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
serverTLSBootstrap: true
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
tlsCipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
tlsMinVersion: VersionTLS12
volumePluginDir: /usr/libexec/k0s/kubelet-plugins/volume/exec
It is adding kubeReservedCgroup and kubeletCgroups and this seems to be hard-coded in the code:
grep -r -i kubeReservedCgroup
pkg/component/worker/kubelet.go: KubeReservedCgroup string
pkg/component/worker/kubelet.go: KubeReservedCgroup: "system.slice",
pkg/component/worker/kubelet.go: preparedConfig.KubeReservedCgroup = kubeletConfigData.KubeReservedCgroup
grep -r -i kubeletCgroups
pkg/component/worker/kubelet.go: KubeletCgroups string
pkg/component/worker/kubelet.go: KubeletCgroups: "/system.slice/containerd.service",
pkg/component/worker/kubelet.go: preparedConfig.KubeletCgroups = kubeletConfigData.KubeletCgroups
With this the kubelet daemon can not start up, gives the following error:
run.go:74] "command failed" err="failed to validate kubelet configuration, error: invalid configuration: can't use reservedSystemCPUs (--reserved-cpus) with systemReservedCgroup
Describe the solution you would like
Support cpuManagerPolicy and reservedSystemCPUs in the kubelet configuration
Describe alternatives you've considered
No response
Additional context
No response
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment