`minikube start` on M1 Mac failing with `'kubelet --version': exit status 255`

[rams3sh]

Steps to reproduce the issue:

1. Run the following command :-
   minikube start --driver=docker

Full output of failed command:

Log file created at: 2021/08/16 14:07:42 Running on machine: FDLDL064 Binary: Built with gc go1.16.4 for linux/amd64 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg I0816 14:07:42.554271 202041 out.go:286] Setting OutFile to fd 1 ... I0816 14:07:42.554436 202041 out.go:338] isatty.IsTerminal(1) = true I0816 14:07:42.554443 202041 out.go:299] Setting ErrFile to fd 2... I0816 14:07:42.554451 202041 out.go:338] isatty.IsTerminal(2) = true I0816 14:07:42.554716 202041 root.go:312] Updating PATH: /home/rseshadri/.minikube/bin I0816 14:07:42.555439 202041 out.go:293] Setting JSON to false I0816 14:07:42.576973 202041 start.go:111] hostinfo: {"hostname":"FDLDL064","uptime":331985,"bootTime":1628771077,"procs":320,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.11.0-25-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"6499bef5-befd-4de8-b950-457d91d90574"} I0816 14:07:42.577154 202041 start.go:121] virtualization: kvm host I0816 14:07:42.582284 202041 out.go:165] 😄 minikube v1.22.0 on Ubuntu 20.04 I0816 14:07:42.583324 202041 notify.go:169] Checking for updates... I0816 14:07:42.584078 202041 driver.go:335] Setting default libvirt URI to qemu:///system I0816 14:07:42.645531 202041 docker.go:132] docker version: linux-20.10.7 I0816 14:07:42.645589 202041 cli_runner.go:115] Run: docker system info --format "{{json .}}" I0816 14:07:42.692299 202041 info.go:263] docker info: {ID:63YF:7UB3:XY35:AJ3T:V4DH:3LU5:UWNZ:C3KS:Y2MC:L4BM:UYRV:WLCK Containers:15 ContainersRunning:0 ContainersPaused:0 ContainersStopped:15 Images:26 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:34 SystemTime:2021-08-16 14:07:42.677547407 +0530 IST LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.11.0-25-generic OperatingSystem:Ubuntu 20.04.2 LTS OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:4 MemTotal:16665108480 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:FDLDL064 Labels:[] ExperimentalBuild:false ServerVersion:20.10.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID: Expected:} RuncCommit:{ID: Expected:} InitCommit:{ID: Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings: ServerErrors:[] ClientInfo:{Debug:false Plugins:[] Warnings:}} I0816 14:07:42.692356 202041 docker.go:244] overlay module found I0816 14:07:42.696004 202041 out.go:165] ✨ Using the docker driver based on user configuration I0816 14:07:42.696053 202041 start.go:278] selected driver: docker I0816 14:07:42.696056 202041 start.go:751] validating driver "docker" against I0816 14:07:42.696089 202041 start.go:762] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Reason: Fix: Doc:} I0816 14:07:42.696350 202041 cli_runner.go:115] Run: docker system info --format "{{json .}}" I0816 14:07:42.738341 202041 info.go:263] docker info: {ID:63YF:7UB3:XY35:AJ3T:V4DH:3LU5:UWNZ:C3KS:Y2MC:L4BM:UYRV:WLCK Containers:15 ContainersRunning:0 ContainersPaused:0 ContainersStopped:15 Images:26 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus: Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization: Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:34 SystemTime:2021-08-16 14:07:42.72940272 +0530 IST LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.11.0-25-generic OperatingSystem:Ubuntu 20.04.2 LTS OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:4 MemTotal:16665108480 GenericResources: DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:FDLDL064 Labels:[] ExperimentalBuild:false ServerVersion:20.10.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID: Expected:} RuncCommit:{ID: Expected:} InitCommit:{ID: Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings: ServerErrors:[] ClientInfo:{Debug:false Plugins:[] Warnings:}} I0816 14:07:42.738407 202041 start_flags.go:261] no existing cluster config was found, will generate one from the flags I0816 14:07:42.759051 202041 start_flags.go:342] Using suggested 3900MB memory alloc based on sys=15893MB, container=15893MB I0816 14:07:42.759121 202041 start_flags.go:669] Wait components to verify : map[apiserver:true system_pods:true] I0816 14:07:42.759134 202041 cni.go:93] Creating CNI manager for "" I0816 14:07:42.759139 202041 cni.go:167] CNI unnecessary in this configuration, recommending no CNI I0816 14:07:42.759142 202041 start_flags.go:275] config: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false} I0816 14:07:42.762790 202041 out.go:165] 👍 Starting control plane node minikube in cluster minikube I0816 14:07:42.762869 202041 cache.go:117] Beginning downloading kic base image for docker with docker I0816 14:07:42.769290 202041 out.go:165] 🚜 Pulling base image ... I0816 14:07:42.769333 202041 preload.go:134] Checking if preload exists for k8s version v1.21.2 and runtime docker I0816 14:07:42.769376 202041 preload.go:150] Found local preload: /home/rseshadri/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.2-docker-overlay2-amd64.tar.lz4 I0816 14:07:42.769380 202041 cache.go:56] Caching tarball of preloaded images I0816 14:07:42.769428 202041 image.go:75] Checking for gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 in local docker daemon I0816 14:07:42.769560 202041 preload.go:174] Found /home/rseshadri/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download I0816 14:07:42.769574 202041 cache.go:59] Finished verifying existence of preloaded tar for v1.21.2 on docker I0816 14:07:42.769872 202041 profile.go:148] Saving config to /home/rseshadri/.minikube/profiles/minikube/config.json ... I0816 14:07:42.769885 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/config.json: {Name:mkf6684a51bf53a67befbbcfcbdbaaa517795e7e Clock:{} Delay:500ms Timeout:1m0s Cancel:} I0816 14:07:42.814546 202041 image.go:79] Found gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 in local docker daemon, skipping pull I0816 14:07:42.814557 202041 cache.go:139] gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 exists in daemon, skipping load I0816 14:07:42.814567 202041 cache.go:205] Successfully downloaded all kic artifacts I0816 14:07:42.814585 202041 start.go:313] acquiring machines lock for minikube: {Name:mk9eb9753f6d172b5bde2444a2583cf5bcb0bdf8 Clock:{} Delay:500ms Timeout:10m0s Cancel:} I0816 14:07:42.814670 202041 start.go:317] acquired machines lock for "minikube" in 75.148µs I0816 14:07:42.814693 202041 start.go:89] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.21.2 ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false} &{Name: IP: Port:8443 KubernetesVersion:v1.21.2 ControlPlane:true Worker:true} I0816 14:07:42.814741 202041 start.go:126] createHost starting for "" (driver="docker") I0816 14:07:42.818350 202041 out.go:192] 🔥 Creating docker container (CPUs=2, Memory=3900MB) ... I0816 14:07:42.818537 202041 start.go:160] libmachine.API.Create for "minikube" (driver="docker") I0816 14:07:42.818552 202041 client.go:168] LocalClient.Create starting I0816 14:07:42.818596 202041 main.go:130] libmachine: Reading certificate data from /home/rseshadri/.minikube/certs/ca.pem I0816 14:07:42.818621 202041 main.go:130] libmachine: Decoding PEM data... I0816 14:07:42.818636 202041 main.go:130] libmachine: Parsing certificate... I0816 14:07:42.818723 202041 main.go:130] libmachine: Reading certificate data from /home/rseshadri/.minikube/certs/cert.pem I0816 14:07:42.818739 202041 main.go:130] libmachine: Decoding PEM data... I0816 14:07:42.818747 202041 main.go:130] libmachine: Parsing certificate... I0816 14:07:42.819009 202041 cli_runner.go:115] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" W0816 14:07:42.854560 202041 cli_runner.go:162] docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1 I0816 14:07:42.854605 202041 network_create.go:255] running [docker network inspect minikube] to gather additional debugging logs... I0816 14:07:42.854616 202041 cli_runner.go:115] Run: docker network inspect minikube W0816 14:07:42.888780 202041 cli_runner.go:162] docker network inspect minikube returned with exit code 1 I0816 14:07:42.888794 202041 network_create.go:258] error running [docker network inspect minikube]: docker network inspect minikube: exit status 1 stdout: []

stderr:
Error: No such network: minikube
I0816 14:07:42.888801 202041 network_create.go:260] output of [docker network inspect minikube]: -- stdout --
[]

-- /stdout --
** stderr **
Error: No such network: minikube

** /stderr **
I0816 14:07:42.888853 202041 cli_runner.go:115] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0816 14:07:42.922837 202041 network.go:288] reserving subnet 192.168.49.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.49.0:0xc0000c7028] misses:0}
I0816 14:07:42.922863 202041 network.go:235] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0816 14:07:42.922878 202041 network_create.go:106] attempt to create docker network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 1500 ...
I0816 14:07:42.922912 202041 cli_runner.go:115] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube
I0816 14:07:43.039016 202041 network_create.go:90] docker network minikube 192.168.49.0/24 created
I0816 14:07:43.039040 202041 kic.go:106] calculated static IP "192.168.49.2" for the "minikube" container
I0816 14:07:43.039085 202041 cli_runner.go:115] Run: docker ps -a --format {{.Names}}
I0816 14:07:43.083367 202041 cli_runner.go:115] Run: docker volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true
I0816 14:07:43.124482 202041 oci.go:102] Successfully created a docker volume minikube
I0816 14:07:43.124534 202041 cli_runner.go:115] Run: docker run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -d /var/lib
I0816 14:07:44.139895 202041 cli_runner.go:168] Completed: docker run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -d /var/lib: (1.01533568s)
I0816 14:07:44.139907 202041 oci.go:106] Successfully prepared a docker volume minikube
W0816 14:07:44.139928 202041 oci.go:135] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
W0816 14:07:44.139934 202041 oci.go:119] Your kernel does not support memory limit capabilities or the cgroup is not mounted.
I0816 14:07:44.139978 202041 cli_runner.go:115] Run: docker info --format "'{{json .SecurityOptions}}'"
I0816 14:07:44.140000 202041 preload.go:134] Checking if preload exists for k8s version v1.21.2 and runtime docker
I0816 14:07:44.140012 202041 kic.go:179] Starting extracting preloaded images to volume ...
I0816 14:07:44.140049 202041 cli_runner.go:115] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/rseshadri/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -I lz4 -xf /preloaded.tar -C /extractDir
I0816 14:07:44.202540 202041 cli_runner.go:115] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79
I0816 14:07:44.920110 202041 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Running}}
I0816 14:07:44.979887 202041 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}}
I0816 14:07:45.054012 202041 cli_runner.go:115] Run: docker exec minikube stat /var/lib/dpkg/alternatives/iptables
I0816 14:07:45.258524 202041 oci.go:278] the created container "minikube" has a running status.
I0816 14:07:45.258546 202041 kic.go:210] Creating ssh key for kic: /home/rseshadri/.minikube/machines/minikube/id_rsa...
I0816 14:07:45.316482 202041 kic_runner.go:188] docker (temp): /home/rseshadri/.minikube/machines/minikube/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0816 14:07:45.541289 202041 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}}
I0816 14:07:45.626588 202041 kic_runner.go:94] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0816 14:07:45.626598 202041 kic_runner.go:115] Args: [docker exec --privileged minikube chown docker:docker /home/docker/.ssh/authorized_keys]
I0816 14:07:54.673465 202041 cli_runner.go:168] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/rseshadri/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -I lz4 -xf /preloaded.tar -C /extractDir: (10.533332284s)
I0816 14:07:54.673497 202041 kic.go:188] duration metric: took 10.533478 seconds to extract preloaded images to volume
I0816 14:07:54.673648 202041 cli_runner.go:115] Run: docker container inspect minikube --format={{.State.Status}}
I0816 14:07:54.728192 202041 machine.go:88] provisioning docker machine ...
I0816 14:07:54.728223 202041 ubuntu.go:169] provisioning hostname "minikube"
I0816 14:07:54.728269 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:54.771078 202041 main.go:130] libmachine: Using SSH client type: native
I0816 14:07:54.771223 202041 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x8022e0] 0x8022a0 [] 0s} 127.0.0.1 49177 }
I0816 14:07:54.771231 202041 main.go:130] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I0816 14:07:55.025683 202041 main.go:130] libmachine: SSH cmd err, output: : minikube

I0816 14:07:55.025731 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:55.073434 202041 main.go:130] libmachine: Using SSH client type: native
I0816 14:07:55.073550 202041 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x8022e0] 0x8022a0 [] 0s} 127.0.0.1 49177 }
I0816 14:07:55.073561 202041 main.go:130] libmachine: About to run SSH command:

	if ! grep -xq '.*\sminikube' /etc/hosts; then
		if grep -xq '127.0.1.1\s.*' /etc/hosts; then
			sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
		else 
			echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; 
		fi
	fi

I0816 14:07:55.242160 202041 main.go:130] libmachine: SSH cmd err, output: :
I0816 14:07:55.242175 202041 ubuntu.go:175] set auth options {CertDir:/home/rseshadri/.minikube CaCertPath:/home/rseshadri/.minikube/certs/ca.pem CaPrivateKeyPath:/home/rseshadri/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/rseshadri/.minikube/machines/server.pem ServerKeyPath:/home/rseshadri/.minikube/machines/server-key.pem ClientKeyPath:/home/rseshadri/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/rseshadri/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/rseshadri/.minikube}
I0816 14:07:55.242186 202041 ubuntu.go:177] setting up certificates
I0816 14:07:55.242192 202041 provision.go:83] configureAuth start
I0816 14:07:55.242236 202041 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I0816 14:07:55.296162 202041 provision.go:137] copyHostCerts
I0816 14:07:55.296205 202041 exec_runner.go:145] found /home/rseshadri/.minikube/key.pem, removing ...
I0816 14:07:55.296209 202041 exec_runner.go:190] rm: /home/rseshadri/.minikube/key.pem
I0816 14:07:55.296261 202041 exec_runner.go:152] cp: /home/rseshadri/.minikube/certs/key.pem --> /home/rseshadri/.minikube/key.pem (1675 bytes)
I0816 14:07:55.296327 202041 exec_runner.go:145] found /home/rseshadri/.minikube/ca.pem, removing ...
I0816 14:07:55.296331 202041 exec_runner.go:190] rm: /home/rseshadri/.minikube/ca.pem
I0816 14:07:55.296354 202041 exec_runner.go:152] cp: /home/rseshadri/.minikube/certs/ca.pem --> /home/rseshadri/.minikube/ca.pem (1086 bytes)
I0816 14:07:55.296404 202041 exec_runner.go:145] found /home/rseshadri/.minikube/cert.pem, removing ...
I0816 14:07:55.296407 202041 exec_runner.go:190] rm: /home/rseshadri/.minikube/cert.pem
I0816 14:07:55.296428 202041 exec_runner.go:152] cp: /home/rseshadri/.minikube/certs/cert.pem --> /home/rseshadri/.minikube/cert.pem (1131 bytes)
I0816 14:07:55.296468 202041 provision.go:111] generating server cert: /home/rseshadri/.minikube/machines/server.pem ca-key=/home/rseshadri/.minikube/certs/ca.pem private-key=/home/rseshadri/.minikube/certs/ca-key.pem org=rseshadri.minikube san=[192.168.49.2 127.0.0.1 localhost 127.0.0.1 minikube minikube]
I0816 14:07:55.463290 202041 provision.go:171] copyRemoteCerts
I0816 14:07:55.463363 202041 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0816 14:07:55.463407 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:55.510555 202041 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49177 SSHKeyPath:/home/rseshadri/.minikube/machines/minikube/id_rsa Username:docker}
I0816 14:07:55.647288 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1086 bytes)
I0816 14:07:55.776337 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0816 14:07:55.874150 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0816 14:07:55.971601 202041 provision.go:86] duration metric: configureAuth took 729.401524ms
I0816 14:07:55.971613 202041 ubuntu.go:193] setting minikube options for container-runtime
I0816 14:07:55.971750 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:56.006771 202041 main.go:130] libmachine: Using SSH client type: native
I0816 14:07:56.006887 202041 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x8022e0] 0x8022a0 [] 0s} 127.0.0.1 49177 }
I0816 14:07:56.006894 202041 main.go:130] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0816 14:07:56.260434 202041 main.go:130] libmachine: SSH cmd err, output: : overlay

I0816 14:07:56.260456 202041 ubuntu.go:71] root file system type: overlay
I0816 14:07:56.260885 202041 provision.go:308] Updating docker unit: /lib/systemd/system/docker.service ...
I0816 14:07:56.260997 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:56.320806 202041 main.go:130] libmachine: Using SSH client type: native
I0816 14:07:56.320953 202041 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x8022e0] 0x8022a0 [] 0s} 127.0.0.1 49177 }
I0816 14:07:56.321029 202041 main.go:130] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity
TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0816 14:07:56.596635 202041 main.go:130] libmachine: SSH cmd err, output: : [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity
TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install]
WantedBy=multi-user.target

I0816 14:07:56.596704 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:56.644217 202041 main.go:130] libmachine: Using SSH client type: native
I0816 14:07:56.644370 202041 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x8022e0] 0x8022a0 [] 0s} 127.0.0.1 49177 }
I0816 14:07:56.644386 202041 main.go:130] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0816 14:07:59.417023 202041 main.go:130] libmachine: SSH cmd err, output: : --- /lib/systemd/system/docker.service 2021-06-02 11:54:50.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2021-08-16 08:37:56.594056366 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
+BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60

[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always

-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure

-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity

-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process
-OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker

I0816 14:07:59.417063 202041 machine.go:91] provisioned docker machine in 4.688859388s
I0816 14:07:59.417070 202041 client.go:171] LocalClient.Create took 16.598515625s
I0816 14:07:59.417078 202041 start.go:168] duration metric: libmachine.API.Create for "minikube" took 16.598540305s
I0816 14:07:59.417082 202041 start.go:267] post-start starting for "minikube" (driver="docker")
I0816 14:07:59.417085 202041 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0816 14:07:59.417136 202041 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0816 14:07:59.417170 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:59.459843 202041 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49177 SSHKeyPath:/home/rseshadri/.minikube/machines/minikube/id_rsa Username:docker}
I0816 14:07:59.629652 202041 ssh_runner.go:149] Run: cat /etc/os-release
I0816 14:07:59.654861 202041 main.go:130] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0816 14:07:59.654886 202041 main.go:130] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0816 14:07:59.654892 202041 main.go:130] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0816 14:07:59.654896 202041 info.go:137] Remote host: Ubuntu 20.04.2 LTS
I0816 14:07:59.654902 202041 filesync.go:126] Scanning /home/rseshadri/.minikube/addons for local assets ...
I0816 14:07:59.654940 202041 filesync.go:126] Scanning /home/rseshadri/.minikube/files for local assets ...
I0816 14:07:59.654966 202041 start.go:270] post-start completed in 237.879908ms
I0816 14:07:59.655225 202041 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I0816 14:07:59.689534 202041 profile.go:148] Saving config to /home/rseshadri/.minikube/profiles/minikube/config.json ...
I0816 14:07:59.689720 202041 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0816 14:07:59.689745 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:59.724134 202041 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49177 SSHKeyPath:/home/rseshadri/.minikube/machines/minikube/id_rsa Username:docker}
I0816 14:07:59.867093 202041 start.go:129] duration metric: createHost completed in 17.052340655s
I0816 14:07:59.867103 202041 start.go:80] releasing machines lock for "minikube", held for 17.052415539s
I0816 14:07:59.867174 202041 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I0816 14:07:59.900775 202041 ssh_runner.go:149] Run: systemctl --version
I0816 14:07:59.900807 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:59.900853 202041 ssh_runner.go:149] Run: curl -sS -m 2 https://k8s.gcr.io/
I0816 14:07:59.900892 202041 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0816 14:07:59.987285 202041 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49177 SSHKeyPath:/home/rseshadri/.minikube/machines/minikube/id_rsa Username:docker}
I0816 14:07:59.987307 202041 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49177 SSHKeyPath:/home/rseshadri/.minikube/machines/minikube/id_rsa Username:docker}
I0816 14:08:00.110564 202041 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service containerd
I0816 14:08:00.519782 202041 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0816 14:08:00.567551 202041 cruntime.go:249] skipping containerd shutdown because we are bound to it
I0816 14:08:00.567616 202041 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service crio
I0816 14:08:00.607289 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0816 14:08:00.686613 202041 ssh_runner.go:149] Run: sudo systemctl unmask docker.service
I0816 14:08:00.929231 202041 ssh_runner.go:149] Run: sudo systemctl enable docker.socket
I0816 14:08:01.199685 202041 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0816 14:08:01.246120 202041 ssh_runner.go:149] Run: sudo systemctl daemon-reload
I0816 14:08:01.525694 202041 ssh_runner.go:149] Run: sudo systemctl start docker
I0816 14:08:01.578797 202041 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0816 14:08:01.666758 202041 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0816 14:08:01.759831 202041 out.go:192] 🐳 Preparing Kubernetes v1.21.2 on Docker 20.10.7 ...
I0816 14:08:01.760032 202041 cli_runner.go:115] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0816 14:08:01.818969 202041 ssh_runner.go:149] Run: grep 192.168.49.1 host.minikube.internal$ /etc/hosts
I0816 14:08:01.842549 202041 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.49.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0816 14:08:01.906115 202041 preload.go:134] Checking if preload exists for k8s version v1.21.2 and runtime docker
I0816 14:08:01.906156 202041 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0816 14:08:01.962834 202041 docker.go:535] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.21.2
k8s.gcr.io/kube-proxy:v1.21.2
k8s.gcr.io/kube-scheduler:v1.21.2
k8s.gcr.io/kube-controller-manager:v1.21.2
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/pause:3.4.1
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/coredns/coredns:v1.8.0
k8s.gcr.io/etcd:3.4.13-0
kubernetesui/metrics-scraper:v1.0.4

-- /stdout --
I0816 14:08:01.962844 202041 docker.go:466] Images already preloaded, skipping extraction
I0816 14:08:01.962881 202041 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0816 14:08:02.020272 202041 docker.go:535] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.21.2
k8s.gcr.io/kube-controller-manager:v1.21.2
k8s.gcr.io/kube-scheduler:v1.21.2
k8s.gcr.io/kube-proxy:v1.21.2
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/pause:3.4.1
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/coredns/coredns:v1.8.0
k8s.gcr.io/etcd:3.4.13-0
kubernetesui/metrics-scraper:v1.0.4

-- /stdout --
I0816 14:08:02.020282 202041 cache_images.go:74] Images are preloaded, skipping loading
I0816 14:08:02.020320 202041 ssh_runner.go:149] Run: docker info --format {{.CgroupDriver}}
I0816 14:08:02.155333 202041 cni.go:93] Creating CNI manager for ""
I0816 14:08:02.155342 202041 cni.go:167] CNI unnecessary in this configuration, recommending no CNI
I0816 14:08:02.155347 202041 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0816 14:08:02.155356 202041 kubeadm.go:153] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.49.2 APIServerPort:8443 KubernetesVersion:v1.21.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.49.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.49.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0816 14:08:02.155440 202041 kubeadm.go:157] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.49.2
bindPort: 8443
bootstrapTokens:

• groups:
  • system:bootstrappers:kubeadm:default-node-token
    ttl: 24h0m0s
    usages:
  • signing
  • authentication
    nodeRegistration:
    criSocket: /var/run/dockershim.sock
    name: "minikube"
    kubeletExtraArgs:
    node-ip: 192.168.49.2
    taints: []

---

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.49.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.21.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"

disable disk resource management by default

imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0

I0816 14:08:02.155498 202041 kubeadm.go:909] kubelet [Unit]
Wants=docker.socket

[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.21.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2

[Install]
config:
{KubernetesVersion:v1.21.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0816 14:08:02.155535 202041 ssh_runner.go:149] Run: sudo ls /var/lib/minikube/binaries/v1.21.2
I0816 14:08:02.196445 202041 binaries.go:44] Found k8s binaries, skipping transfer
I0816 14:08:02.196483 202041 ssh_runner.go:149] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0816 14:08:02.233094 202041 ssh_runner.go:316] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (334 bytes)
I0816 14:08:02.301884 202041 ssh_runner.go:316] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0816 14:08:02.371809 202041 ssh_runner.go:316] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (1867 bytes)
I0816 14:08:02.435891 202041 ssh_runner.go:149] Run: grep 192.168.49.2 control-plane.minikube.internal$ /etc/hosts
I0816 14:08:02.459067 202041 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.49.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0816 14:08:02.522363 202041 certs.go:52] Setting up /home/rseshadri/.minikube/profiles/minikube for IP: 192.168.49.2
I0816 14:08:02.522397 202041 certs.go:179] skipping minikubeCA CA generation: /home/rseshadri/.minikube/ca.key
I0816 14:08:02.522412 202041 certs.go:179] skipping proxyClientCA CA generation: /home/rseshadri/.minikube/proxy-client-ca.key
I0816 14:08:02.522446 202041 certs.go:294] generating minikube-user signed cert: /home/rseshadri/.minikube/profiles/minikube/client.key
I0816 14:08:02.522450 202041 crypto.go:69] Generating cert /home/rseshadri/.minikube/profiles/minikube/client.crt with IP's: []
I0816 14:08:02.742439 202041 crypto.go:157] Writing cert to /home/rseshadri/.minikube/profiles/minikube/client.crt ...
I0816 14:08:02.742477 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/client.crt: {Name:mka25b8dfe6da1dd652fd59311787990fd05efd3 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.742670 202041 crypto.go:165] Writing key to /home/rseshadri/.minikube/profiles/minikube/client.key ...
I0816 14:08:02.742681 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/client.key: {Name:mk7e9ceedb068f2b0d0843bed28e8546418acedb Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.742780 202041 certs.go:294] generating minikube signed cert: /home/rseshadri/.minikube/profiles/minikube/apiserver.key.dd3b5fb2
I0816 14:08:02.742784 202041 crypto.go:69] Generating cert /home/rseshadri/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 with IP's: [192.168.49.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0816 14:08:02.863474 202041 crypto.go:157] Writing cert to /home/rseshadri/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 ...
I0816 14:08:02.863484 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2: {Name:mk60d063bec22fa616627410348563d568dcc50f Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.863601 202041 crypto.go:165] Writing key to /home/rseshadri/.minikube/profiles/minikube/apiserver.key.dd3b5fb2 ...
I0816 14:08:02.863607 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/apiserver.key.dd3b5fb2: {Name:mkb5799fb1f22e98e82cb75c6852708ea2b88868 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.863663 202041 certs.go:305] copying /home/rseshadri/.minikube/profiles/minikube/apiserver.crt.dd3b5fb2 -> /home/rseshadri/.minikube/profiles/minikube/apiserver.crt
I0816 14:08:02.863710 202041 certs.go:309] copying /home/rseshadri/.minikube/profiles/minikube/apiserver.key.dd3b5fb2 -> /home/rseshadri/.minikube/profiles/minikube/apiserver.key
I0816 14:08:02.863750 202041 certs.go:294] generating aggregator signed cert: /home/rseshadri/.minikube/profiles/minikube/proxy-client.key
I0816 14:08:02.863753 202041 crypto.go:69] Generating cert /home/rseshadri/.minikube/profiles/minikube/proxy-client.crt with IP's: []
I0816 14:08:02.957187 202041 crypto.go:157] Writing cert to /home/rseshadri/.minikube/profiles/minikube/proxy-client.crt ...
I0816 14:08:02.957197 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/proxy-client.crt: {Name:mke4e49b3d8ccd8d8b01db8f87825bbff4624c24 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.957306 202041 crypto.go:165] Writing key to /home/rseshadri/.minikube/profiles/minikube/proxy-client.key ...
I0816 14:08:02.957311 202041 lock.go:36] WriteFile acquiring /home/rseshadri/.minikube/profiles/minikube/proxy-client.key: {Name:mk8edbe22e8858e7690765fc132af2735d30dccb Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0816 14:08:02.957439 202041 certs.go:369] found cert: /home/rseshadri/.minikube/certs/home/rseshadri/.minikube/certs/ca-key.pem (1675 bytes)
I0816 14:08:02.957464 202041 certs.go:369] found cert: /home/rseshadri/.minikube/certs/home/rseshadri/.minikube/certs/ca.pem (1086 bytes)
I0816 14:08:02.957482 202041 certs.go:369] found cert: /home/rseshadri/.minikube/certs/home/rseshadri/.minikube/certs/cert.pem (1131 bytes)
I0816 14:08:02.957500 202041 certs.go:369] found cert: /home/rseshadri/.minikube/certs/home/rseshadri/.minikube/certs/key.pem (1675 bytes)
I0816 14:08:02.958184 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/profiles/minikube/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0816 14:08:03.045735 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/profiles/minikube/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0816 14:08:03.135713 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/profiles/minikube/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0816 14:08:03.226138 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/profiles/minikube/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0816 14:08:03.314632 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0816 14:08:03.401961 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0816 14:08:03.494163 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0816 14:08:03.591570 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0816 14:08:03.680700 202041 ssh_runner.go:316] scp /home/rseshadri/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0816 14:08:03.774548 202041 ssh_runner.go:316] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0816 14:08:03.843380 202041 ssh_runner.go:149] Run: openssl version
I0816 14:08:03.867972 202041 ssh_runner.go:149] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0816 14:08:03.917171 202041 ssh_runner.go:149] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0816 14:08:03.940902 202041 certs.go:410] hashing: -rw-r--r-- 1 root root 1111 Jul 8 04:17 /usr/share/ca-certificates/minikubeCA.pem
I0816 14:08:03.940944 202041 ssh_runner.go:149] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0816 14:08:03.969153 202041 ssh_runner.go:149] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0816 14:08:04.018313 202041 kubeadm.go:390] StartCluster: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.21.2 ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0816 14:08:04.018385 202041 ssh_runner.go:149] Run: docker ps --filter status=paused --filter=name=k8s_.*(kube-system) --format={{.ID}}
I0816 14:08:04.070214 202041 ssh_runner.go:149] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0816 14:08:04.110014 202041 ssh_runner.go:149] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0816 14:08:04.147255 202041 kubeadm.go:220] ignoring SystemVerification for kubeadm because of docker driver
I0816 14:08:04.147306 202041 ssh_runner.go:149] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0816 14:08:04.183312 202041 kubeadm.go:151] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:

stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0816 14:08:04.183332 202041 ssh_runner.go:240] Start: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
W0816 14:09:06.217236 202041 out.go:230] 💢 initialization failed, will try again: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.21.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 5.11.0-25-generic
DOCKER_VERSION: 20.10.7
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled

stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.11.0-25-generic\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': signal: killed
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

I0816 14:09:06.217350 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0816 14:09:06.398485 202041 ssh_runner.go:149] Run: sudo systemctl stop -f kubelet
I0816 14:09:06.437162 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_.*(kube-system) --format={{.ID}}
I0816 14:09:06.489333 202041 kubeadm.go:220] ignoring SystemVerification for kubeadm because of docker driver
I0816 14:09:06.489367 202041 ssh_runner.go:149] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0816 14:09:06.526594 202041 kubeadm.go:151] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:

stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0816 14:09:06.526625 202041 ssh_runner.go:240] Start: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0816 14:10:07.650951 202041 kubeadm.go:392] StartCluster complete in 2m3.632639842s
I0816 14:10:07.651002 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0816 14:10:07.716785 202041 logs.go:270] 0 containers: []
W0816 14:10:07.716797 202041 logs.go:272] No container was found matching "kube-apiserver"
I0816 14:10:07.716836 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0816 14:10:07.769475 202041 logs.go:270] 0 containers: []
W0816 14:10:07.769483 202041 logs.go:272] No container was found matching "etcd"
I0816 14:10:07.769517 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0816 14:10:07.822992 202041 logs.go:270] 0 containers: []
W0816 14:10:07.823027 202041 logs.go:272] No container was found matching "coredns"
I0816 14:10:07.823073 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0816 14:10:07.891713 202041 logs.go:270] 0 containers: []
W0816 14:10:07.891722 202041 logs.go:272] No container was found matching "kube-scheduler"
I0816 14:10:07.891794 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0816 14:10:07.947744 202041 logs.go:270] 0 containers: []
W0816 14:10:07.947753 202041 logs.go:272] No container was found matching "kube-proxy"
I0816 14:10:07.947830 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0816 14:10:07.999969 202041 logs.go:270] 0 containers: []
W0816 14:10:07.999978 202041 logs.go:272] No container was found matching "kubernetes-dashboard"
I0816 14:10:08.000014 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0816 14:10:08.058055 202041 logs.go:270] 0 containers: []
W0816 14:10:08.058064 202041 logs.go:272] No container was found matching "storage-provisioner"
I0816 14:10:08.058098 202041 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0816 14:10:08.112316 202041 logs.go:270] 0 containers: []
W0816 14:10:08.112324 202041 logs.go:272] No container was found matching "kube-controller-manager"
I0816 14:10:08.112330 202041 logs.go:123] Gathering logs for kubelet ...
I0816 14:10:08.112336 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0816 14:10:08.162297 202041 logs.go:123] Gathering logs for dmesg ...
I0816 14:10:08.162308 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0816 14:10:08.232431 202041 logs.go:123] Gathering logs for describe nodes ...
I0816 14:10:08.232442 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.21.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0816 14:10:08.409936 202041 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.21.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.21.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:

stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?

** /stderr **
I0816 14:10:08.409945 202041 logs.go:123] Gathering logs for Docker ...
I0816 14:10:08.409952 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0816 14:10:08.464919 202041 logs.go:123] Gathering logs for container status ...
I0816 14:10:08.464930 202041 ssh_runner.go:149] Run: /bin/bash -c "sudo which crictl || echo crictl ps -a || sudo docker ps -a"
I0816 14:10:10.616627 202041 ssh_runner.go:189] Completed: /bin/bash -c "sudo which crictl || echo crictl ps -a || sudo docker ps -a": (2.15168587s)
W0816 14:10:10.616756 202041 out.go:359] Error starting cluster: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.21.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 5.11.0-25-generic
DOCKER_VERSION: 20.10.7
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled

stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.11.0-25-generic\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': signal: killed
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher
W0816 14:10:10.616766 202041 out.go:230]
W0816 14:10:10.616870 202041 out.go:230] 💣 Error starting cluster: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.21.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 5.11.0-25-generic
DOCKER_VERSION: 20.10.7
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled

stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.11.0-25-generic\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': signal: killed
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

W0816 14:10:10.616903 202041 out.go:230]
W0816 14:10:10.617560 202041 out.go:230] ╭────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please attach the following file to the GitHub issue: │
│ - /home/rseshadri/.minikube/logs/lastStart.txt │
│ │
╰────────────────────────────────────────────────────────────────────╯
I0816 14:10:10.624257 202041 out.go:165]
W0816 14:10:10.624388 202041 out.go:230] ❌ Exiting due to GUEST_START: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.21.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 5.11.0-25-generic
DOCKER_VERSION: 20.10.7
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled

stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.11.0-25-generic\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': signal: killed
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

W0816 14:10:10.624415 202041 out.go:230]
W0816 14:10:10.625059 202041 out.go:230] ╭────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please attach the following file to the GitHub issue: │
│ - /home/rseshadri/.minikube/logs/lastStart.txt │
│ │
╰────────────────────────────────────────────────────────────────────╯

Let me know if any other details are required from my end.

[Rits333]

Hey,
I use the minikube on the virtual box
minikube start --kubernetes-version=v1.20.0 --driver=virtualbox
I use this command if it is helpful to you can go with it and let me know that on which platform you using minikube on Docker or Linux or virtual box.
and if it helps to you please let me know.

[spowelljr]

Hi @rams3sh, thanks for reporting your issue with minikube!

First recommendation, I'd suggest updating to the latest version of minikube (v1.23.2).

Could you please try running minikube delete --all and then minikube start --driver=docker --force-systemd=true and let me know if that solves your issue, thanks!

[spowelljr]

Hi @rams3sh, we haven't heard back from you, do you still have this issue?
There isn't enough information in this issue to make it actionable, and a long enough duration has passed, so this issue is likely difficult to replicate.

I will close this issue for now but feel free to reopen when you feel ready to provide more information.

[DPS0340]

Hi! I'm struggling with the same errors on macOS m1.
minikube works by ignoring this error message, but custom start options like --extra-config=apiserver.service-node-port-range= won't work.

I think that kubelet couldn't be installed on macos. is it related?

• Command to reproduce

minikube start

• Main error messages

❌  Exiting due to GUEST_START: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.22.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.22.3
[preflight] Running pre-flight checks

stderr:
        [WARNING Swap]: running with swap on is not supported. Please disable swap
        [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': exit status 255
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

[DPS0340]

I was wrong. minikube doesn't start. I was confused with docker desktop k8s.

[chris-lsn]

I have this issue on m1 mac.

[vimalkansal]

I am also having the same issue. My host system is :

Model Name: MacBook Pro
Chip: Apple M1 Max

And this is the complete output
minikube start --driver=docker --force-systemd=true
😄 minikube v1.24.0 on Darwin 12.0.1 (arm64)
✨ Using the docker driver based on user configuration
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=1988MB) ...
🐳 Preparing Kubernetes v1.22.3 on Docker 20.10.8 ...
💢 initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.22.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.22.3
[preflight] Running pre-flight checks

stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': exit status 255
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

💣 Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.22.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.22.3
[preflight] Running pre-flight checks

stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': exit status 255
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

❌ Exiting due to GUEST_START: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.22.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.22.3
[preflight] Running pre-flight checks

stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': exit status 255
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯