Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rancher desktop] v0.13.0 not starting in Mac (Intel) #2765

Closed
mariomac opened this issue May 13, 2022 · 19 comments
Closed

[rancher desktop] v0.13.0 not starting in Mac (Intel) #2765

mariomac opened this issue May 13, 2022 · 19 comments
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug.
Milestone

Comments

@mariomac
Copy link

With v0.13.0 trying to create a cluster in Mac OS X fails due a timeout when trying to check the health of the control plane. I reverted back to v0.12.0 and I could create it without any problem.

See logs at the end of the message.

  • kind version: (use kind version): v0.13.0
  • Kubernetes version: (use kubectl version): Client Version:
version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:58:47Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/amd64"}
  • Docker version: (use docker info): Using Rancher Desktop for Mac (Intel)
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.7.1)
  compose: Docker Compose (Docker Inc., v2.2.3)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: 52b36a2dd837e8462de8e01458bf02cf9eea47dd
 init version:
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.32-0-virt
 Operating System: Alpine Linux v3.15
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 5.809GiB
 Name: lima-rancher-desktop
 ID: 644A:SKGR:IOJD:7HAF:AGST:3UJS:IJ6G:64K7:C2FP:DPRD:WM4T:PM3U
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
  • OS (e.g. from /etc/os-release): Mac OS X

  • Logs:

Logs:

$ kind create cluster
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.24.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✗ Starting control-plane 🕹️
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0513 09:35:20.921308     241 initconfiguration.go:255] loading configuration from "/kind/kubeadm.conf"
W0513 09:35:20.928108     241 initconfiguration.go:332] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.24.0
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0513 09:35:20.934666     241 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0513 09:35:21.213713     241 certs.go:522] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.18.0.2 127.0.0.1]
I0513 09:35:21.446998     241 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
I0513 09:35:21.542530     241 certs.go:522] validating certificate period for front-proxy-ca certificate
I0513 09:35:21.680049     241 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
I0513 09:35:21.746977     241 certs.go:522] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
I0513 09:35:22.429382     241 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
I0513 09:35:22.577931     241 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
I0513 09:35:22.687841     241 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0513 09:35:22.843880     241 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0513 09:35:22.980671     241 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0513 09:35:23.126765     241 kubelet.go:65] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0513 09:35:23.327391     241 manifests.go:99] [control-plane] getting StaticPodSpecs
I0513 09:35:23.327588     241 certs.go:522] validating certificate period for CA certificate
I0513 09:35:23.327670     241 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0513 09:35:23.327693     241 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0513 09:35:23.327715     241 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0513 09:35:23.327740     241 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0513 09:35:23.327768     241 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I0513 09:35:23.332124     241 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0513 09:35:23.333566     241 manifests.go:99] [control-plane] getting StaticPodSpecs
I0513 09:35:23.334742     241 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0513 09:35:23.335135     241 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0513 09:35:23.335834     241 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0513 09:35:23.336530     241 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0513 09:35:23.337138     241 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0513 09:35:23.337547     241 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0513 09:35:23.338903     241 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0513 09:35:23.340378     241 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0513 09:35:23.341959     241 manifests.go:99] [control-plane] getting StaticPodSpecs
I0513 09:35:23.342809     241 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0513 09:35:23.343776     241 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0513 09:35:23.346871     241 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0513 09:35:23.347582     241 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0513 09:35:23.348637     241 loader.go:372] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0513 09:35:23.352570     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:23.858292     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:24.361201     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:24.858739     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:25.359225     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:25.863217     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:26.359925     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:26.864379     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:27.360891     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:27.862193     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:28.361031     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:28.858272     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:29.359187     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:29.863027     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:30.358526     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:30.863877     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:31.358243     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:31.862197     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:32.357425     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:32.858005     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:33.363581     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:33.858061     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:34.357965     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:34.857503     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:35.362738     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:35.863745     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:36.356444     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:36.858030     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:37.357683     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:37.865711     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:38.363595     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:38.858682     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:39.362878     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:39.860513     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:40.362217     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:40.858744     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:41.358274     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:41.860758     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 4 milliseconds
I0513 09:35:42.361353     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:42.858291     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:43.361856     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:43.859752     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:44.356966     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:44.859000     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:45.361909     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:45.861033     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:46.356292     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:46.859288     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:47.358313     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:47.863914     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:48.365166     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:48.858053     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:49.360915     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:49.857822     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:50.358155     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:50.856719     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:51.358747     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:51.859617     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:52.358217     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:52.866201     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:53.364179     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:53.859169     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:54.362252     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:54.865647     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:55.360397     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:55.860273     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:56.357105     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:56.859899     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:57.359387     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:57.859187     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:35:58.364292     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:58.859274     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:35:59.361176     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:35:59.859226     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:00.359045     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:00.859295     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:01.357751     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:01.858706     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:02.359347     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:02.859140     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] Initial timeout of 40s passed.
I0513 09:36:03.356855     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0513 09:36:03.859437     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:04.361443     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:04.861733     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:05.359698     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:05.856739     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:06.356801     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:06.858846     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:07.360681     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 4 milliseconds
I0513 09:36:07.857734     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:08.356752     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0513 09:36:08.860052     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:09.357469     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:09.858634     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:10.358544     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:10.864122     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:11.358739     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:11.863250     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:12.361029     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:12.858645     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:13.359663     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:13.859161     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:14.359562     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:14.859947     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:15.361046     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:15.859810     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:16.357603     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:16.858373     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:17.360059     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:17.858828     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:18.358483     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0513 09:36:18.858479     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:19.359557     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:19.858456     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:20.360451     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:20.858862     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:21.356829     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:21.857109     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:22.358414     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:22.858447     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:23.359191     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:23.859046     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:24.363974     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:24.859260     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:25.360683     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:25.860340     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:26.358440     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:26.865297     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:27.358980     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:27.859663     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:28.360339     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:28.858583     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:29.360410     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:29.858953     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:30.359311     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:30.858204     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:31.358286     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:31.859867     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:32.358770     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:32.859437     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:33.359121     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:33.857910     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:34.360417     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:34.863117     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:35.359169     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:35.859923     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:36.357249     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:36.860717     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:37.358244     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:37.863672     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:38.363941     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0513 09:36:38.858507     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:39.360278     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:39.859266     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:40.363372     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:40.859641     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:41.358561     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:41.865282     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:42.359501     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:42.861765     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:43.357384     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:43.857763     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:44.360214     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:44.865193     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:45.359633     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:45.857842     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:46.357654     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:46.860848     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:47.359358     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:47.860690     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:48.359244     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:48.857055     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:49.361062     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:49.857810     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:50.359016     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:50.860984     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:51.357657     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:51.860703     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:52.358258     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:52.859732     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:53.357852     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:53.858757     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:54.360196     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:54.860449     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:55.357506     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:55.858133     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:56.358123     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:56.865050     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 5 milliseconds
I0513 09:36:57.360619     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:57.860828     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:36:58.362508     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:36:58.858155     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:59.358323     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:36:59.861975     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:00.363252     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:00.858121     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:01.355824     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:01.856767     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:02.361536     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:02.859393     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:03.359571     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:03.857458     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:04.360894     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:04.861509     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:05.356974     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:05.860543     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:06.355789     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:06.856339     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:07.356184     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:07.861446     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:08.358721     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:08.857282     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:09.359270     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:09.861176     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0513 09:37:10.359665     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:10.858085     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:11.357757     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:11.858365     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:12.358727     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:12.860134     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:13.358107     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:13.856784     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:14.359685     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:14.863441     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:15.360519     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0513 09:37:15.859665     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:16.357360     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:16.860035     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:17.357633     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:17.863977     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0513 09:37:18.361050     241 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:234
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:153
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:856
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:974
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:902
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1571
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:235
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:153
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:856
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:974
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:902
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1571
@mariomac mariomac added the kind/bug Categorizes issue or PR as related to a bug. label May 13, 2022
@aojea
Copy link
Contributor

aojea commented May 13, 2022

can you check the node image is the one mentioned in the release notes?

https://github.com/kubernetes-sigs/kind/releases/tag/v0.13.0

New Node images have been built for kind v0.13.0, please use these exact images (IE like kindest/node:v1.24.0@sha256:406fd86d48eaf4c04c7280cd1d2ca1d61e7d0d61ddef0125cb097bc7b82ed6a1 including the digest) or build your own as we may need to change the image format again in the future sweat_smile

if it is , try to create the cluster with the --retain flag, and attach the logs after exporting them with kind export logs

@stmcginnis
Copy link
Contributor

Just a data point, this is on macOS 12.3.1:

❯ kind version
kind v0.13.0 go1.18 darwin/amd64
❯ kind create cluster
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.24.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a nice day! 👋
❯ uname -a
Darwin smcginnis-a01.vmware.com 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:45:05 PDT 2022; root:xnu-8020.101.4~15/RELEASE_X86_64 x86_64

@mariomac
Copy link
Author

Thanks for the quick response.

I cleaned up whatever I could clean in my local docker, and kind create cluster still fails with the same error. The downloaded image seems to coincide with the image from @aojea :

$ uname -a
Darwin mmacias-mac 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:45:05 PDT 2022; root:xnu-8020.101.4~15/RELEASE_X86_64 x86_64
$ docker image inspect f607f70d3a1a
[
    {
        "Id": "sha256:f607f70d3a1abc777191399d72689ed65ff4bf5f95630d19e8b72d5b9624af32",
        "RepoTags": [],
        "RepoDigests": [
            "kindest/node@sha256:406fd86d48eaf4c04c7280cd1d2ca1d61e7d0d61ddef0125cb097bc7b82ed6a1"

Please find attached the logs:

kind-logs.zip

@aojea
Copy link
Contributor

aojea commented May 13, 2022

👁️

 Kernel Version: 5.15.32-0-virt
 Operating System: Alpine Linux v3.15
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 5.809GiB
 Name: lima-rancher-desktop

alpine + lima rancher?

@aojea
Copy link
Contributor

aojea commented May 13, 2022

kubelet fails to start because is missing the kubelet cgroup path cc: @BenTheElder

May 13 12:26:13 kind-control-plane kubelet[1974]: W0513 12:26:13.016306    1974 sysinfo.go:203] Nodes topology is not available, providing CPU topology
May 13 12:26:13 kind-control-plane kubelet[1974]: Error: failed to run Kubelet: invalid configuration: cgroup ["kubelet"] has some missing paths: /sys/fs/cgroup/systemd/kubelet.slice
May 13 12:26:13 kind-control-plane kubelet[1974]: Usage:
May 13 12:26:13 kind-control-plane kubelet[1974]:   kubelet [flags]

there is also this error in the serial.log

Welcome to �[1mUbuntu 21.10�[0m!

�[0;1;38;5;185mCouldn't move remaining userspace processes, ignoring: Input/output error�[0m
Queued start job for default target Graphical Interface.
�[0;1;38;5;185m-.slice: Failed to migrate controller cgroups from /docker, ignoring: Input/output error�[0m

@mariomac
Copy link
Author

@aojea

alpine + lima rancher?

Yes, I'm using https://rancherdesktop.io/ with dockerd container runtime.

@aojea
Copy link
Contributor

aojea commented May 13, 2022

@aojea

alpine + lima rancher?

Yes, I'm using https://rancherdesktop.io/ with dockerd container runtime.

first time I heard about it, didn't know it worked, nice

@aojea
Copy link
Contributor

aojea commented May 13, 2022

ahh, I just checked, this is limavm under the hood , @AkihiroSuda I think we broke something with latest changes moving to systemd cgroup driver in lima

#2737

@BenTheElder BenTheElder changed the title v0.13.0 not starting in Mac (Intel) [rancher desktop] v0.13.0 not starting in Mac (Intel) May 13, 2022
@BenTheElder
Copy link
Member

IIRC docker desktop (not rancher desktop) is on cgroupsv2 for some time now, so while we checked with mac user(s) for docker desktop it did not cover this.

We have coverage of (cgroupsv1, cgroupfs, dockerd) in https://github.com/kubernetes-sigs/kind/actions/runs/2299283697 but I suppose that's in the GitHub Actions env which is a bit different.

In the meantime I think you'll find that v0.13.0 works if you use a --image earlier than v1.24.0 which makes the switch to systemd cgroup driver. https://github.com/kubernetes-sigs/kind/releases/tag/v0.13.0#new-features has a list of possible images.

@BenTheElder
Copy link
Member

BenTheElder commented May 13, 2022

We should confirm that kind create cluster --image=kindest/node:v1.23.6@sha256:1af0f1bee4c3c0fe9b07de5e5d3fafeb2eec7b4e1b268ae89fcab96ec67e8355 works on v0.13.0 with rancher desktop.

@BenTheElder
Copy link
Member

In #2766 the issue looks is isolated between v1.24 and v1.23.6.

It should also be possible to use the following to force cgroupfs instead of systemd:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = false

  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.test-handler.options]
    SystemdCgroup = false
kubeadmConfigPatches:
- |
  apiVersion: kubelet.config.k8s.io/v1beta1
  kind: KubeletConfiguration
  cgroupDriver: cgroupfs

@BenTheElder
Copy link
Member

BenTheElder commented May 13, 2022

Interesting, in github actions docker / cgroupv1 we see in node serial:

INFO: unmounting and removing /sys/fs/cgroup/rdma
INFO: unmounting and removing /sys/fs/cgroup/misc

Which is expected from #2709

but then in the logs here I see:

INFO: unmounting and removing /sys/fs/cgroup/openrc

So this seems to be broken when systemd not running on the host on cgroups v1. (seems to imply rancher desktop uses openrc instead of systemd).

Kubelet log as mentioned above:

["kubelet"] has some missing paths: /sys/fs/cgroup/systemd/kubelet.slice

@BenTheElder
Copy link
Member

BenTheElder commented May 13, 2022

So we're failing here because the /sys/fs/cgroup/systemd directory doesn't exist from the host and so it's not in the list when we produce the hacked up kubelet.slice cgroup root mounts.

So unlike our bad release note assertion, currently systemd is once again required on the host when running with kind + Kubernetes v1.24.0.

We should roll forward with a fix, and update the release notes to reference this issue + workaround #2766 (comment) (the config above #2765 (comment)).

@BenTheElder
Copy link
Member

bentheelder/kind-node:v1.24.0 has #2767, but I lack the handy access to testing this environment, if you could try kind v0.13.0 with kind create cluster --image=bentheelder/kind-node:v1.24.0 --retain and report success or share the logs that would be great!

@jankoprowski
Copy link

kind create cluster --image=kindest/node:v1.23.6@sha256:1af0f1bee4c3c0fe9b07de5e5d3fafeb2eec7b4e1b268ae89fcab96ec67e8355

I have similar issues on WSL 2 (docker installed as linux service) and below command works like a charm

@BenTheElder
Copy link
Member

Hi folks, sorry for the delay.

I've tested an updated fix in #2767 on colima xref #2778 which should be the same root issue (kind v0.13.0 / v1.24 imagse brokene on non-systemd hosts)

Please let me know if this updated image works with kind v0.13.0
kind create cluster --image=bentheelder/kind-node:v1.24.0@sha256:cf92db251fb3f420634ec46d9c222812fee4d75f1e0105a325e57679b0f57370

Hopefully that should be it, and we'll cut another release soon with this fix.

@BenTheElder BenTheElder added this to the v0.14.0 milestone May 19, 2022
@mariomac
Copy link
Author

Hi @BenTheElder ! I just verified that this image works with a fresh installation of Rancher Destkop. Thank you for addressing this issue!

@cndoit18
Copy link

Hi folks, sorry for the delay.

I've tested an updated fix in #2767 on colima xref #2778 which should be the same root issue (kind v0.13.0 / v1.24 imagse brokene on non-systemd hosts)

Please let me know if this updated image works with kind v0.13.0 kind create cluster --image=bentheelder/kind-node:v1.24.0@sha256:cf92db251fb3f420634ec46d9c222812fee4d75f1e0105a325e57679b0f57370

Hopefully that should be it, and we'll cut another release soon with this fix.

@BenTheElder Thanks you, works for me as well

@BenTheElder
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

6 participants