unable to start minikube with docker without cri-dockerd

[sudlo]

What Happened?

I have installed minikube on aws ec2 instance followed all the steps.
Please find the logs attached for your reference

Attach the log file

root@ip-172-31-47-92:~# cat logs.txt
*

• ==> Audit <==

• |---------|-------------------------|----------|------|---------|---------------------|---------------------|
  | Command | Args | Profile | User | Version | Start Time | End Time |
  |---------|-------------------------|----------|------|---------|---------------------|---------------------|
  | start | --vm-driver=none | minikube | root | v1.26.0 | 24 Jun 22 03:57 UTC | |
  | delete | | minikube | root | v1.26.0 | 24 Jun 22 04:00 UTC | 24 Jun 22 04:00 UTC |
  | start | --driver=docker | minikube | root | v1.26.0 | 24 Jun 22 04:00 UTC | |
  | delete | | minikube | root | v1.26.0 | 24 Jun 22 04:00 UTC | 24 Jun 22 04:00 UTC |
  | start | | minikube | root | v1.26.0 | 24 Jun 22 04:00 UTC | |
  | start | --force --driver=docker | minikube | root | v1.26.0 | 24 Jun 22 04:01 UTC | |
  | start | --driver=none | minikube | root | v1.26.0 | 24 Jun 22 04:01 UTC | |
  | start | | minikube | root | v1.26.0 | 24 Jun 22 04:02 UTC | |
  |---------|-------------------------|----------|------|---------|---------------------|---------------------|

• ==> Last Start <==

• Log file created at: 2022/06/24 04:02:49
  Running on machine: ip-172-31-47-92
  Binary: Built with gc go1.18.3 for linux/amd64
  Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
  I0624 04:02:49.109566 6897 out.go:296] Setting OutFile to fd 1 ...
  I0624 04:02:49.109700 6897 out.go:348] isatty.IsTerminal(1) = true
  I0624 04:02:49.109705 6897 out.go:309] Setting ErrFile to fd 2...
  I0624 04:02:49.109712 6897 out.go:348] isatty.IsTerminal(2) = true
  I0624 04:02:49.109921 6897 root.go:329] Updating PATH: /root/.minikube/bin
  W0624 04:02:49.110036 6897 root.go:307] Error reading config file at /root/.minikube/config/config.json: open /root/.minikube/config/config.json: no such file or directory
  I0624 04:02:49.110148 6897 out.go:303] Setting JSON to false
  I0624 04:02:49.110933 6897 start.go:115] hostinfo: {"hostname":"ip-172-31-47-92","uptime":493,"bootTime":1656042876,"procs":122,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"18.04","kernelVersion":"5.4.0-1078-aws","kernelArch":"x86_64","virtualizationSystem":"xen","virtualizationRole":"guest","hostId":"ec29327a-c75c-82bd-d5d2-09198152d1c4"}
  I0624 04:02:49.110994 6897 start.go:125] virtualization: xen guest
  I0624 04:02:49.113713 6897 out.go:177] 😄 minikube v1.26.0 on Ubuntu 18.04 (xen/amd64)
  W0624 04:02:49.116051 6897 preload.go:295] Failed to list preload files: open /root/.minikube/cache/preloaded-tarball: no such file or directory
  I0624 04:02:49.116256 6897 notify.go:193] Checking for updates...
  I0624 04:02:49.116293 6897 config.go:178] Loaded profile config "minikube": Driver=none, ContainerRuntime=docker, KubernetesVersion=v1.24.1
  I0624 04:02:49.116332 6897 driver.go:360] Setting default libvirt URI to qemu:///system
  I0624 04:02:49.116647 6897 exec_runner.go:51] Run: systemctl --version
  I0624 04:02:49.122833 6897 out.go:177] ✨ Using the none driver based on existing profile
  I0624 04:02:49.125045 6897 start.go:284] selected driver: none
  I0624 04:02:49.125059 6897 start.go:805] validating driver "none" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:none HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:resolv-conf Value:/run/systemd/resolve/resolv.conf}] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:m01 IP:172.31.47.92 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/root:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
  I0624 04:02:49.125144 6897 start.go:816] status for none: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Reason: Fix: Doc: Version:}
  I0624 04:02:49.125166 6897 start.go:1541] auto setting extra-config to "kubelet.resolv-conf=/run/systemd/resolve/resolv.conf".
  I0624 04:02:49.125609 6897 cni.go:95] Creating CNI manager for ""
  I0624 04:02:49.125617 6897 cni.go:149] Driver none used, CNI unnecessary in this configuration, recommending no CNI
  I0624 04:02:49.125632 6897 start_flags.go:310] config:
  {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:none HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:resolv-conf Value:/run/systemd/resolve/resolv.conf}] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:m01 IP:172.31.47.92 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/root:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
  I0624 04:02:49.131832 6897 out.go:177] 👍 Starting control plane node minikube in cluster minikube
  I0624 04:02:49.134339 6897 profile.go:148] Saving config to /root/.minikube/profiles/minikube/config.json ...
  I0624 04:02:49.134543 6897 cache.go:208] Successfully downloaded all kic artifacts
  I0624 04:02:49.134579 6897 start.go:352] acquiring machines lock for minikube: {Name:mkc8ab01ad3ea83211c505c81a7ee49a8e3ecb89 Clock:{} Delay:500ms Timeout:13m0s Cancel:}
  I0624 04:02:49.134751 6897 start.go:356] acquired machines lock for "minikube" in 146.017µs
  I0624 04:02:49.134771 6897 start.go:94] Skipping create...Using existing machine configuration
  I0624 04:02:49.134778 6897 fix.go:55] fixHost starting: m01
  W0624 04:02:49.134971 6897 none.go:130] unable to get port: "minikube" does not appear in /root/.kube/config
  I0624 04:02:49.134981 6897 api_server.go:165] Checking apiserver status ...
  I0624 04:02:49.135007 6897 exec_runner.go:51] Run: sudo pgrep -xnf kube-apiserver.minikube.
  W0624 04:02:49.147114 6897 api_server.go:169] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.minikube.: exit status 1
  stdout:

stderr:
I0624 04:02:49.147151 6897 exec_runner.go:51] Run: sudo systemctl is-active --quiet service kubelet
I0624 04:02:49.156870 6897 fix.go:103] recreateIfNeeded on minikube: state=Stopped err=
W0624 04:02:49.156885 6897 fix.go:129] unexpected machine state, will restart:
I0624 04:02:49.159426 6897 out.go:177] 🔄 Restarting existing none bare metal machine for "minikube" ...
I0624 04:02:49.162619 6897 profile.go:148] Saving config to /root/.minikube/profiles/minikube/config.json ...
I0624 04:02:49.162753 6897 start.go:306] post-start starting for "minikube" (driver="none")
I0624 04:02:49.162762 6897 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0624 04:02:49.162803 6897 exec_runner.go:51] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0624 04:02:49.169850 6897 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0624 04:02:49.169872 6897 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0624 04:02:49.169883 6897 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0624 04:02:49.175670 6897 out.go:177] ℹ️ OS release is Ubuntu 18.04.6 LTS
I0624 04:02:49.180296 6897 filesync.go:126] Scanning /root/.minikube/addons for local assets ...
I0624 04:02:49.180353 6897 filesync.go:126] Scanning /root/.minikube/files for local assets ...
I0624 04:02:49.180373 6897 start.go:309] post-start completed in 17.610894ms
I0624 04:02:49.180382 6897 fix.go:57] fixHost completed within 45.605194ms
I0624 04:02:49.180388 6897 start.go:81] releasing machines lock for "minikube", held for 45.62773ms
I0624 04:02:49.180936 6897 exec_runner.go:51] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock
image-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0624 04:02:49.181000 6897 exec_runner.go:51] Run: curl -sS -m 2 https://k8s.gcr.io/
I0624 04:02:49.202779 6897 exec_runner.go:51] Run: sudo systemctl unmask docker.service
I0624 04:02:49.367152 6897 exec_runner.go:51] Run: sudo systemctl enable docker.socket
I0624 04:02:49.510093 6897 exec_runner.go:51] Run: sudo systemctl daemon-reload
I0624 04:02:49.686716 6897 exec_runner.go:51] Run: sudo systemctl restart docker
I0624 04:02:49.964699 6897 exec_runner.go:51] Run: sudo systemctl enable cri-docker.socket
I0624 04:02:49.977302 6897 out.go:177]
W0624 04:02:49.980229 6897 out.go:239] ❌ Exiting due to RUNTIME_ENABLE: sudo systemctl enable cri-docker.socket: exit status 1
stdout:

stderr:
Failed to enable unit: Unit file cri-docker.socket does not exist.

W0624 04:02:49.980260 6897 out.go:239]
W0624 04:02:49.984043 6897 out.go:239] ╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
I0624 04:02:49.986533 6897 out.go:177]

Operating System

Other

Driver

No response

[spowelljr]

Hi @sudlo, thanks for reporting your issue with minikube.

Kubernetes v1.24 dropped support for Dockershim, so if you want to use the combination of the none driver, Kubernetes v1.24+, and the Docker container runtime you'll need to install cri-dockerd on your system, as listed in our requirements page.

Other options include using a different driver, using an older version of Kubernetes --kubernetes-version, or using a different container runtime --container-runtime.

Please try one of those solution and let me know if it resolves your issue, thanks

[sudlo]

thanks a lot let me try it again

[afbjorklund]

Some more details in #14403, the docker driver should check for both cri-dockerd and crictl.

Currently it just looks for the "docker" client (?), but not for "dockerd" and "cri-dockerd" servers

// Available returns an error if it is not possible to use this runtime on a host
func (r *Docker) Available() error {
        _, err := exec.LookPath("docker")
        return err
}

[spowelljr]

Some more details in #14403, the docker driver should check for both cri-dockerd and crictl.

Currently it just looks for the "docker" client (?), but not for "dockerd" and "cri-dockerd" servers

// Available returns an error if it is not possible to use this runtime on a host
func (r *Docker) Available() error {
        _, err := exec.LookPath("docker")
        return err
}

That's a good point, I created an issue for this request #14460

[spowelljr]

So there are two error cases:

none driver

Users are unaware that minikube v1.26.0 requires a new dependency of cri-docker.

Background:

Kubernetes v1.24 dropped support for Dockershim, so if you want to use the combination of the none driver, Kubernetes v1.24+, and the Docker container runtime you'll need to install cri-dockerd on your system, as listed in our requirements page.

Other options include using a different driver, using an older version of Kubernetes --kubernetes-version, or using a different container runtime --container-runtime.

Implementing #14460 will detect that they're missing a system dependency. Will this prevent us from needing a solution message?

Solution:
Installing cri-docker, installation instructions: https://github.com/Mirantis/cri-dockerd#build-and-install

Other drivers (docker, etc.)

Users that try doing minikube start --kubernetes-version=stable (or v1.24.0+) on a cluster previously created before minikube v1.26.0 can result in failing to find cri-docker. I've tried reproducing it but oddly wasn't able to do so, but we have logs of users experiencing it (#14411).

Solution:
Run minikube delete then try starting minikube again and all should be resolved