Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minikube start failure #14456

Open
kashifest opened this issue Jun 28, 2022 · 14 comments
Open

Minikube start failure #14456

kashifest opened this issue Jun 28, 2022 · 14 comments
Labels
co/kvm2-driver KVM2 driver related issues kind/support Categorizes issue or PR as a support question. long-term-support Long-term support issues that can't be fixed in code

Comments

@kashifest
Copy link

What Happened?

minikube restart failing . We first create a minikube with --driver=kvm2. First start is successful. We then need to stop minikube and do some more configuration and later we restart the minikube again. The second start is failing. Attaching the first and the second minikube start here.

Attach the log file

First

sudo su -l -c 'minikube start --driver=kvm2 --insecure-registry 192.168.111.1:5000' ****
* minikube v1.26.0 on Centos 9 (kvm/amd64)
* Using the kvm2 driver based on user configuration
* Downloading driver docker-machine-driver-kvm2:
* Downloading VM boot image ...
* Starting control plane node minikube in cluster minikube
* Downloading Kubernetes v1.24.1 preload ...
* Creating kvm2 VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
* Preparing Kubernetes v1.24.1 on Docker 20.10.16 ...
 - Generating certificates and keys ...
 - Booting up control plane ...
 - Configuring RBAC rules ...
* Verifying Kubernetes components...
 - Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Enabled addons: storage-provisioner, default-storageclass
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
+ [[ 0 -eq 0 ]]
+ break
+ sudo su -l -c 'minikube stop' ****
* Stopping node "minikube"  ...
* 1 node stopped.

Second

sudo su -l -c 'minikube start --driver=kvm2' ****
* minikube v1.26.0 on Centos 9 (kvm/amd64)
* Using the kvm2 driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing kvm2 VM for "minikube" ...

X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-28T13:41:39Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"

Operating System

No response

Driver

No response

@kashifest
Copy link
Author

+ sudo su -l -c 'minikube start --alsologtostderr --driver=kvm2 --insecure-registry 192.168.111.1:5000' ****
I0628 09:53:24.796613    5655 out.go:296] Setting OutFile to fd 1 ...
I0628 09:53:24.797126    5655 out.go:343] TERM=unknown,COLORTERM=, which probably does not support color
I0628 09:53:24.797191    5655 out.go:309] Setting ErrFile to fd 2...
I0628 09:53:24.797388    5655 out.go:343] TERM=unknown,COLORTERM=, which probably does not support color
I0628 09:53:24.797699    5655 root.go:329] Updating PATH: /home/****/.minikube/bin
I0628 09:53:24.798114    5655 out.go:303] Setting JSON to false
I0628 09:53:24.800779    5655 start.go:115] hostinfo: {"hostname":"ci-test-vm-20220628134925-nxgi.openstacklocal","uptime":210,"bootTime":1656424195,"procs":193,"os":"linux","platform":"centos","platformFamily":"rhel","platformVersion":"9","kernelVersion":"5.14.0-115.el9.x86_64","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"8533cce4-4e87-42fe-9942-fd5b2aba60d3"}
I0628 09:53:24.800892    5655 start.go:125] virtualization: kvm guest
I0628 09:53:24.824538    5655 out.go:177] * minikube v1.26.0 on Centos 9 (kvm/amd64)
* minikube v1.26.0 on Centos 9 (kvm/amd64)
I0628 09:53:24.828285    5655 notify.go:193] Checking for updates...
I0628 09:53:24.829945    5655 preload.go:306] deleting older generation preload /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v17-v1.23.3-docker-overlay2-amd64.tar.lz4
I0628 09:53:24.830237    5655 preload.go:306] deleting older generation preload /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v17-v1.23.3-docker-overlay2-amd64.tar.lz4.checksum
I0628 09:53:24.830395    5655 driver.go:360] Setting default libvirt URI to qemu:///system
I0628 09:53:26.194282    5655 out.go:177] * Using the kvm2 driver based on user configuration
* Using the kvm2 driver based on user configuration
I0628 09:53:26.202822    5655 start.go:284] selected driver: kvm2
I0628 09:53:26.202860    5655 start.go:805] validating driver "kvm2" against <nil>
I0628 09:53:26.202893    5655 start.go:816] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0628 09:53:26.203211    5655 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0628 09:53:26.203740    5655 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/****/.minikube/bin:/home/****/.krew/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/local/go/bin
I0628 09:53:26.301595    5655 install.go:137] /usr/local/bin/docker-machine-driver-kvm2 version is 1.25.2
W0628 09:53:26.301655    5655 install.go:62] docker-machine-driver-kvm2: docker-machine-driver-kvm2 is version 1.25.2, want 1.26.0
I0628 09:53:26.305203    5655 out.go:177] * Downloading driver docker-machine-driver-kvm2:
* Downloading driver docker-machine-driver-kvm2:
I0628 09:53:26.307800    5655 download.go:101] Downloading: https://github.com/kubernetes/minikube/releases/download/v1.26.0/docker-machine-driver-kvm2-amd64?checksum=file:https://github.com/kubernetes/minikube/releases/download/v1.26.0/docker-machine-driver-kvm2-amd64.sha256 -> /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:53:28.211855    5655 start_flags.go:296] no existing cluster config was found, will generate one from the flags 
I0628 09:53:28.212483    5655 start_flags.go:835] Wait components to verify : map[apiserver:true system_pods:true]
I0628 09:53:28.212529    5655 cni.go:95] Creating CNI manager for ""
I0628 09:53:28.212542    5655 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0628 09:53:28.212551    5655 start_flags.go:310] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[192.168.111.1:5000] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/****:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0628 09:53:28.212739    5655 iso.go:128] acquiring lock: {Name:mkfd1ab3d05ae9ca1111d9e050f9880c81c13514 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0628 09:53:28.222887    5655 out.go:177] * Downloading VM boot image ...
* Downloading VM boot image ...
I0628 09:53:28.225802    5655 download.go:101] Downloading: https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso?checksum=file:https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso.sha256 -> /home/****/.minikube/cache/iso/amd64/minikube-v1.26.0-amd64.iso
I0628 09:53:31.057780    5655 out.go:177] * Starting control plane node minikube in cluster minikube
* Starting control plane node minikube in cluster minikube
I0628 09:53:31.061568    5655 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:53:31.226637    5655 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.24.1/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4
I0628 09:53:31.228351    5655 cache.go:57] Caching tarball of preloaded images
I0628 09:53:31.228886    5655 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:53:31.461080    5655 out.go:177] * Downloading Kubernetes v1.24.1 preload ...
* Downloading Kubernetes v1.24.1 preload ...
I0628 09:53:31.464219    5655 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4 ...
I0628 09:53:31.726962    5655 download.go:101] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.24.1/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4?checksum=md5:cbe3dd94d1bd66cb19d71bba2673d3e7 -> /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4
I0628 09:53:35.387433    5655 preload.go:249] saving checksum for preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4 ...
I0628 09:53:35.387672    5655 preload.go:256] verifying checksumm of /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4 ...
I0628 09:53:36.271398    5655 cache.go:60] Finished verifying existence of preloaded tar for  v1.24.1 on docker
I0628 09:53:36.272168    5655 profile.go:148] Saving config to /home/****/.minikube/profiles/minikube/config.json ...
I0628 09:53:36.272269    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/config.json: {Name:mk68e07efb31aabe7a6221e3ec0454544e3674a1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:53:36.272638    5655 cache.go:208] Successfully downloaded all kic artifacts
I0628 09:53:36.272736    5655 start.go:352] acquiring machines lock for minikube: {Name:mk405a3db8e483af8dee0a375e4dabc91c94683c Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0628 09:53:36.272830    5655 start.go:356] acquired machines lock for "minikube" in 70.591µs
I0628 09:53:36.272884    5655 start.go:91] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[192.168.111.1:5000] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/****:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:} &{Name: IP: Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0628 09:53:36.273028    5655 start.go:131] createHost starting for "" (driver="kvm2")
I0628 09:53:36.279033    5655 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
* Creating kvm2 VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
I0628 09:53:36.279425    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:53:36.279605    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:53:36.304581    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:38379
I0628 09:53:36.305773    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:53:36.309381    5655 main.go:134] libmachine: Using API Version  1
I0628 09:53:36.309449    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:53:36.309906    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:53:36.310099    5655 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:53:36.310318    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:53:36.310496    5655 start.go:165] libmachine.API.Create for "minikube" (driver="kvm2")
I0628 09:53:36.310563    5655 client.go:168] LocalClient.Create starting
I0628 09:53:36.310676    5655 main.go:134] libmachine: Reading certificate data from /home/****/.minikube/certs/ca.pem
I0628 09:53:36.311531    5655 main.go:134] libmachine: Decoding PEM data...
I0628 09:53:36.311593    5655 main.go:134] libmachine: Parsing certificate...
I0628 09:53:36.312552    5655 main.go:134] libmachine: Reading certificate data from /home/****/.minikube/certs/cert.pem
I0628 09:53:36.313102    5655 main.go:134] libmachine: Decoding PEM data...
I0628 09:53:36.313163    5655 main.go:134] libmachine: Parsing certificate...
I0628 09:53:36.313221    5655 main.go:134] libmachine: Running pre-create checks...
I0628 09:53:36.313252    5655 main.go:134] libmachine: (minikube) Calling .PreCreateCheck
I0628 09:53:36.313715    5655 main.go:134] libmachine: (minikube) Calling .GetConfigRaw
I0628 09:53:36.314253    5655 main.go:134] libmachine: Creating machine...
I0628 09:53:36.314302    5655 main.go:134] libmachine: (minikube) Calling .Create
I0628 09:53:36.314493    5655 main.go:134] libmachine: (minikube) Creating KVM machine...
I0628 09:53:36.632538    5655 main.go:134] libmachine: (minikube) DBG | found existing default KVM network
I0628 09:53:36.638401    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:36.638014    5736 network.go:288] reserving subnet 192.168.39.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.39.0:0xc000010860] misses:0}
I0628 09:53:36.638469    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:36.638285    5736 network.go:235] using free private subnet 192.168.39.0/24: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0628 09:53:36.651626    5655 main.go:134] libmachine: (minikube) DBG | trying to create private KVM network mk-minikube 192.168.39.0/24...
I0628 09:53:36.766472    5655 main.go:134] libmachine: (minikube) DBG | private KVM network mk-minikube 192.168.39.0/24 created
I0628 09:53:36.766511    5655 main.go:134] libmachine: (minikube) Setting up store path in /home/****/.minikube/machines/minikube ...
I0628 09:53:36.766549    5655 main.go:134] libmachine: (minikube) Building disk image from file:///home/****/.minikube/cache/iso/amd64/minikube-v1.26.0-amd64.iso
I0628 09:53:36.766783    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:36.766704    5736 common.go:107] Making disk image using store path: /home/****/.minikube
I0628 09:53:36.766938    5655 main.go:134] libmachine: (minikube) Downloading /home/****/.minikube/cache/boot2docker.iso from file:///home/****/.minikube/cache/iso/amd64/minikube-v1.26.0-amd64.iso...
I0628 09:53:37.484047    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:37.483846    5736 common.go:114] Creating ssh key: /home/****/.minikube/machines/minikube/id_rsa...
I0628 09:53:38.191416    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:38.191238    5736 common.go:120] Creating raw disk image: /home/****/.minikube/machines/minikube/minikube.rawdisk...
I0628 09:53:38.191466    5655 main.go:134] libmachine: (minikube) DBG | Writing magic tar header
I0628 09:53:38.191485    5655 main.go:134] libmachine: (minikube) DBG | Writing SSH key tar header
I0628 09:53:38.192548    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:38.192421    5736 common.go:134] Fixing permissions on /home/****/.minikube/machines/minikube ...
I0628 09:53:38.192593    5655 main.go:134] libmachine: (minikube) DBG | Checking permissions on dir: /home/****/.minikube/machines/minikube
I0628 09:53:38.192636    5655 main.go:134] libmachine: (minikube) Setting executable bit set on /home/****/.minikube/machines/minikube (perms=drwx------)
I0628 09:53:38.192710    5655 main.go:134] libmachine: (minikube) Setting executable bit set on /home/****/.minikube/machines (perms=drwxrwxr-x)
I0628 09:53:38.192724    5655 main.go:134] libmachine: (minikube) DBG | Checking permissions on dir: /home/****/.minikube/machines
I0628 09:53:38.192738    5655 main.go:134] libmachine: (minikube) Setting executable bit set on /home/****/.minikube (perms=drwxrwxr-x)
I0628 09:53:38.192749    5655 main.go:134] libmachine: (minikube) DBG | Checking permissions on dir: /home/****/.minikube
I0628 09:53:38.192767    5655 main.go:134] libmachine: (minikube) DBG | Checking permissions on dir: /home/****
I0628 09:53:38.192784    5655 main.go:134] libmachine: (minikube) DBG | Checking permissions on dir: /home
I0628 09:53:38.192813    5655 main.go:134] libmachine: (minikube) Setting executable bit set on /home/**** (perms=drwx--x--x)
I0628 09:53:38.192824    5655 main.go:134] libmachine: (minikube) DBG | Skipping /home - not owner
I0628 09:53:38.192831    5655 main.go:134] libmachine: (minikube) Creating domain...
I0628 09:53:38.217730    5655 main.go:134] libmachine: (minikube) define libvirt domain using xml: 
I0628 09:53:38.217766    5655 main.go:134] libmachine: (minikube) <domain type='kvm'>
I0628 09:53:38.217778    5655 main.go:134] libmachine: (minikube)   <name>minikube</name>
I0628 09:53:38.217787    5655 main.go:134] libmachine: (minikube)   <memory unit='MiB'>4096</memory>
I0628 09:53:38.217796    5655 main.go:134] libmachine: (minikube)   <vcpu>2</vcpu>
I0628 09:53:38.217808    5655 main.go:134] libmachine: (minikube)   <features>
I0628 09:53:38.217825    5655 main.go:134] libmachine: (minikube)     <acpi/>
I0628 09:53:38.217833    5655 main.go:134] libmachine: (minikube)     <apic/>
I0628 09:53:38.217842    5655 main.go:134] libmachine: (minikube)     <pae/>
I0628 09:53:38.217849    5655 main.go:134] libmachine: (minikube)     
I0628 09:53:38.217858    5655 main.go:134] libmachine: (minikube)   </features>
I0628 09:53:38.217872    5655 main.go:134] libmachine: (minikube)   <cpu mode='host-passthrough'>
I0628 09:53:38.217888    5655 main.go:134] libmachine: (minikube)   
I0628 09:53:38.217895    5655 main.go:134] libmachine: (minikube)   </cpu>
I0628 09:53:38.217904    5655 main.go:134] libmachine: (minikube)   <os>
I0628 09:53:38.217912    5655 main.go:134] libmachine: (minikube)     <type>hvm</type>
I0628 09:53:38.217921    5655 main.go:134] libmachine: (minikube)     <boot dev='cdrom'/>
I0628 09:53:38.217930    5655 main.go:134] libmachine: (minikube)     <boot dev='hd'/>
I0628 09:53:38.217943    5655 main.go:134] libmachine: (minikube)     <bootmenu enable='no'/>
I0628 09:53:38.217950    5655 main.go:134] libmachine: (minikube)   </os>
I0628 09:53:38.217958    5655 main.go:134] libmachine: (minikube)   <devices>
I0628 09:53:38.217977    5655 main.go:134] libmachine: (minikube)     <disk type='file' device='cdrom'>
I0628 09:53:38.217989    5655 main.go:134] libmachine: (minikube)       <source file='/home/****/.minikube/machines/minikube/boot2docker.iso'/>
I0628 09:53:38.218003    5655 main.go:134] libmachine: (minikube)       <target dev='hdc' bus='scsi'/>
I0628 09:53:38.218012    5655 main.go:134] libmachine: (minikube)       <readonly/>
I0628 09:53:38.218020    5655 main.go:134] libmachine: (minikube)     </disk>
I0628 09:53:38.218029    5655 main.go:134] libmachine: (minikube)     <disk type='file' device='disk'>
I0628 09:53:38.218039    5655 main.go:134] libmachine: (minikube)       <driver name='qemu' type='raw' cache='default' io='threads' />
I0628 09:53:38.218087    5655 main.go:134] libmachine: (minikube)       <source file='/home/****/.minikube/machines/minikube/minikube.rawdisk'/>
I0628 09:53:38.218215    5655 main.go:134] libmachine: (minikube)       <target dev='hda' bus='virtio'/>
I0628 09:53:38.218235    5655 main.go:134] libmachine: (minikube)     </disk>
I0628 09:53:38.218257    5655 main.go:134] libmachine: (minikube)     <interface type='network'>
I0628 09:53:38.218279    5655 main.go:134] libmachine: (minikube)       <source network='mk-minikube'/>
I0628 09:53:38.218292    5655 main.go:134] libmachine: (minikube)       <model type='virtio'/>
I0628 09:53:38.218306    5655 main.go:134] libmachine: (minikube)     </interface>
I0628 09:53:38.218349    5655 main.go:134] libmachine: (minikube)     <interface type='network'>
I0628 09:53:38.218364    5655 main.go:134] libmachine: (minikube)       <source network='default'/>
I0628 09:53:38.218410    5655 main.go:134] libmachine: (minikube)       <model type='virtio'/>
I0628 09:53:38.218434    5655 main.go:134] libmachine: (minikube)     </interface>
I0628 09:53:38.218459    5655 main.go:134] libmachine: (minikube)     <serial type='pty'>
I0628 09:53:38.218471    5655 main.go:134] libmachine: (minikube)       <target port='0'/>
I0628 09:53:38.218490    5655 main.go:134] libmachine: (minikube)     </serial>
I0628 09:53:38.218528    5655 main.go:134] libmachine: (minikube)     <console type='pty'>
I0628 09:53:38.218547    5655 main.go:134] libmachine: (minikube)       <target type='serial' port='0'/>
I0628 09:53:38.218566    5655 main.go:134] libmachine: (minikube)     </console>
I0628 09:53:38.218582    5655 main.go:134] libmachine: (minikube)     <rng model='virtio'>
I0628 09:53:38.218604    5655 main.go:134] libmachine: (minikube)       <backend model='random'>/dev/random</backend>
I0628 09:53:38.218616    5655 main.go:134] libmachine: (minikube)     </rng>
I0628 09:53:38.218633    5655 main.go:134] libmachine: (minikube)     
I0628 09:53:38.218656    5655 main.go:134] libmachine: (minikube)     
I0628 09:53:38.218679    5655 main.go:134] libmachine: (minikube)   </devices>
I0628 09:53:38.218690    5655 main.go:134] libmachine: (minikube) </domain>
I0628 09:53:38.218728    5655 main.go:134] libmachine: (minikube) 
I0628 09:53:38.448467    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:70:49:c7 in network default
I0628 09:53:38.449429    5655 main.go:134] libmachine: (minikube) Ensuring networks are active...
I0628 09:53:38.449515    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:38.465756    5655 main.go:134] libmachine: (minikube) Ensuring network default is active
I0628 09:53:38.469241    5655 main.go:134] libmachine: (minikube) Ensuring network mk-minikube is active
I0628 09:53:38.471651    5655 main.go:134] libmachine: (minikube) Getting domain xml...
I0628 09:53:38.489348    5655 main.go:134] libmachine: (minikube) Creating domain...
I0628 09:53:39.305088    5655 main.go:134] libmachine: (minikube) Waiting to get IP...
I0628 09:53:39.307331    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:39.308479    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:39.308624    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:39.308538    5736 retry.go:31] will retry after 263.082536ms: waiting for machine to come up
I0628 09:53:39.573361    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:39.573821    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:39.573901    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:39.573791    5736 retry.go:31] will retry after 381.329545ms: waiting for machine to come up
I0628 09:53:39.957640    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:40.048365    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:40.048401    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:39.958789    5736 retry.go:31] will retry after 422.765636ms: waiting for machine to come up
I0628 09:53:40.384053    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:40.385013    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:40.385041    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:40.384916    5736 retry.go:31] will retry after 473.074753ms: waiting for machine to come up
I0628 09:53:40.860251    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:40.861109    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:40.861183    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:40.861002    5736 retry.go:31] will retry after 587.352751ms: waiting for machine to come up
I0628 09:53:41.450614    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:41.454433    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:41.454478    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:41.454355    5736 retry.go:31] will retry after 834.206799ms: waiting for machine to come up
I0628 09:53:42.291334    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:42.292205    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:42.292234    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:42.292124    5736 retry.go:31] will retry after 746.553905ms: waiting for machine to come up
I0628 09:53:43.040569    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:43.041218    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:43.041277    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:43.041184    5736 retry.go:31] will retry after 987.362415ms: waiting for machine to come up
I0628 09:53:44.030811    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:44.031689    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:44.031809    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:44.031731    5736 retry.go:31] will retry after 1.189835008s: waiting for machine to come up
I0628 09:53:45.227404    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:45.228283    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:45.228313    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:45.228187    5736 retry.go:31] will retry after 1.677229867s: waiting for machine to come up
I0628 09:53:46.908879    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:46.909275    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:46.909416    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:46.909296    5736 retry.go:31] will retry after 2.346016261s: waiting for machine to come up
I0628 09:53:49.259562    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:49.264197    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:49.264295    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:49.264240    5736 retry.go:31] will retry after 3.36678925s: waiting for machine to come up
I0628 09:53:52.637110    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:52.637773    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:52.637815    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:52.637726    5736 retry.go:31] will retry after 3.11822781s: waiting for machine to come up
I0628 09:53:55.760985    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:53:55.761983    5655 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:53:55.762087    5655 main.go:134] libmachine: (minikube) DBG | I0628 09:53:55.761838    5736 retry.go:31] will retry after 4.276119362s: waiting for machine to come up
I0628 09:54:00.043910    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.045353    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has current primary IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.045395    5655 main.go:134] libmachine: (minikube) Found IP for machine: 192.168.39.235
I0628 09:54:00.045428    5655 main.go:134] libmachine: (minikube) Reserving static IP address...
I0628 09:54:00.051051    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "minikube", mac: "52:54:00:45:cf:30", ip: "192.168.39.235"} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.051216    5655 main.go:134] libmachine: (minikube) DBG | skip adding static IP to network mk-minikube - found existing host DHCP lease matching {name: "minikube", mac: "52:54:00:45:cf:30", ip: "192.168.39.235"}
I0628 09:54:00.051292    5655 main.go:134] libmachine: (minikube) Reserved static IP address: 192.168.39.235
I0628 09:54:00.051315    5655 main.go:134] libmachine: (minikube) DBG | Getting to WaitForSSH function...
I0628 09:54:00.051377    5655 main.go:134] libmachine: (minikube) Waiting for SSH to be available...
I0628 09:54:00.099013    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.102808    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.103020    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.103510    5655 main.go:134] libmachine: (minikube) DBG | Using SSH client type: external
I0628 09:54:00.103726    5655 main.go:134] libmachine: (minikube) DBG | Using SSH private key: /home/****/.minikube/machines/minikube/id_rsa (-rw-------)
I0628 09:54:00.103776    5655 main.go:134] libmachine: (minikube) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.235 -o IdentitiesOnly=yes -i /home/****/.minikube/machines/minikube/id_rsa -p 22] /usr/bin/ssh <nil>}
I0628 09:54:00.103822    5655 main.go:134] libmachine: (minikube) DBG | About to run SSH command:
I0628 09:54:00.103850    5655 main.go:134] libmachine: (minikube) DBG | exit 0
I0628 09:54:00.276840    5655 main.go:134] libmachine: (minikube) DBG | SSH cmd err, output: <nil>: 
I0628 09:54:00.277544    5655 main.go:134] libmachine: (minikube) KVM machine creation complete!
I0628 09:54:00.278836    5655 main.go:134] libmachine: (minikube) Calling .GetConfigRaw
I0628 09:54:00.279956    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:00.280200    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:00.280492    5655 main.go:134] libmachine: Waiting for machine to be running, this may take a few minutes...
I0628 09:54:00.280542    5655 main.go:134] libmachine: (minikube) Calling .GetState
I0628 09:54:00.300840    5655 main.go:134] libmachine: Detecting operating system of created instance...
I0628 09:54:00.300877    5655 main.go:134] libmachine: Waiting for SSH to be available...
I0628 09:54:00.300898    5655 main.go:134] libmachine: Getting to WaitForSSH function...
I0628 09:54:00.300910    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:00.340785    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.343471    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.343503    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.343798    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:00.344103    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.344307    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.344517    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:00.344711    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:00.345263    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:00.345284    5655 main.go:134] libmachine: About to run SSH command:
exit 0
I0628 09:54:00.486255    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: 
I0628 09:54:00.486456    5655 main.go:134] libmachine: Detecting the provisioner...
I0628 09:54:00.486575    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:00.524012    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.526826    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.526999    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.527529    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:00.527816    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.528110    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.528341    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:00.528671    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:00.528927    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:00.528950    5655 main.go:134] libmachine: About to run SSH command:
cat /etc/os-release
I0628 09:54:00.668643    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2021.02.12-1-g14f2929-dirty
ID=buildroot
VERSION_ID=2021.02.12
PRETTY_NAME="Buildroot 2021.02.12"

I0628 09:54:00.668845    5655 main.go:134] libmachine: found compatible host: buildroot
I0628 09:54:00.668877    5655 main.go:134] libmachine: Provisioning with buildroot...
I0628 09:54:00.668893    5655 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:54:00.669238    5655 buildroot.go:166] provisioning hostname "minikube"
I0628 09:54:00.669277    5655 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:54:00.669520    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:00.704464    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.706492    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.706517    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.706838    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:00.707055    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.707268    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.707485    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:00.707682    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:00.707915    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:00.707939    5655 main.go:134] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I0628 09:54:00.865342    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: minikube

I0628 09:54:00.865547    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:00.911222    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.913932    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:00.914125    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:00.914384    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:00.914612    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.914899    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:00.915171    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:00.915416    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:00.915699    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:00.915811    5655 main.go:134] libmachine: About to run SSH command:

		if ! grep -xq '.*\sminikube' /etc/hosts; then
			if grep -xq '127.0.1.1\s.*' /etc/hosts; then
				sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
			else 
				echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; 
			fi
		fi
I0628 09:54:01.054998    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: 
I0628 09:54:01.055046    5655 buildroot.go:172] set auth options {CertDir:/home/****/.minikube CaCertPath:/home/****/.minikube/certs/ca.pem CaPrivateKeyPath:/home/****/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/****/.minikube/machines/server.pem ServerKeyPath:/home/****/.minikube/machines/server-key.pem ClientKeyPath:/home/****/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/****/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/****/.minikube}
I0628 09:54:01.055101    5655 buildroot.go:174] setting up certificates
I0628 09:54:01.055171    5655 provision.go:83] configureAuth start
I0628 09:54:01.055192    5655 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:54:01.055699    5655 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:54:01.093416    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.097775    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:01.097857    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.099134    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:01.135607    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.137846    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:01.137914    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.138293    5655 provision.go:138] copyHostCerts
I0628 09:54:01.138557    5655 exec_runner.go:144] found /home/****/.minikube/ca.pem, removing ...
I0628 09:54:01.138627    5655 exec_runner.go:207] rm: /home/****/.minikube/ca.pem
I0628 09:54:01.138788    5655 exec_runner.go:151] cp: /home/****/.minikube/certs/ca.pem --> /home/****/.minikube/ca.pem (1082 bytes)
I0628 09:54:01.139246    5655 exec_runner.go:144] found /home/****/.minikube/cert.pem, removing ...
I0628 09:54:01.139264    5655 exec_runner.go:207] rm: /home/****/.minikube/cert.pem
I0628 09:54:01.139396    5655 exec_runner.go:151] cp: /home/****/.minikube/certs/cert.pem --> /home/****/.minikube/cert.pem (1127 bytes)
I0628 09:54:01.139591    5655 exec_runner.go:144] found /home/****/.minikube/key.pem, removing ...
I0628 09:54:01.139612    5655 exec_runner.go:207] rm: /home/****/.minikube/key.pem
I0628 09:54:01.139718    5655 exec_runner.go:151] cp: /home/****/.minikube/certs/key.pem --> /home/****/.minikube/key.pem (1679 bytes)
I0628 09:54:01.140665    5655 provision.go:112] generating server cert: /home/****/.minikube/machines/server.pem ca-key=/home/****/.minikube/certs/ca.pem private-key=/home/****/.minikube/certs/ca-key.pem org=****.minikube san=[192.168.39.235 192.168.39.235 localhost 127.0.0.1 minikube minikube]
I0628 09:54:01.401422    5655 provision.go:172] copyRemoteCerts
I0628 09:54:01.401688    5655 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0628 09:54:01.401730    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:01.442626    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.445923    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:01.446005    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.446461    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:01.446831    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:01.447028    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:01.447227    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:01.547815    5655 ssh_runner.go:362] scp /home/****/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0628 09:54:01.575107    5655 ssh_runner.go:362] scp /home/****/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0628 09:54:01.601237    5655 ssh_runner.go:362] scp /home/****/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0628 09:54:01.627108    5655 provision.go:86] duration metric: configureAuth took 571.90399ms
I0628 09:54:01.627226    5655 buildroot.go:189] setting minikube options for container-runtime
I0628 09:54:01.627545    5655 config.go:178] Loaded profile config "minikube": Driver=kvm2, ContainerRuntime=docker, KubernetesVersion=v1.24.1
I0628 09:54:01.627589    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:01.628066    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:01.672517    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.675863    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:01.675908    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.676533    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:01.677010    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:01.677327    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:01.677649    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:01.678084    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:01.678372    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:01.678468    5655 main.go:134] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0628 09:54:01.821949    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: tmpfs

I0628 09:54:01.821995    5655 buildroot.go:70] root file system type: tmpfs
I0628 09:54:01.822399    5655 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0628 09:54:01.822449    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:01.858882    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.862351    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:01.862457    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:01.862905    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:01.863115    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:01.863373    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:01.863521    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:01.863675    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:01.863842    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:01.863917    5655 main.go:134] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=kvm2 --insecure-registry 10.96.0.0/12 --insecure-registry 192.168.111.1:5000 
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0628 09:54:02.019555    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=kvm2 --insecure-registry 10.96.0.0/12 --insecure-registry 192.168.111.1:5000 
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

I0628 09:54:02.019593    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:02.058294    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:02.061582    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:02.061636    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:02.062045    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:02.062360    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:02.062600    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:02.062825    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:02.063051    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:02.063279    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:02.063344    5655 main.go:134] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0628 09:54:03.146849    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

I0628 09:54:03.146873    5655 main.go:134] libmachine: Checking connection to Docker...
I0628 09:54:03.146887    5655 main.go:134] libmachine: (minikube) Calling .GetURL
I0628 09:54:03.164362    5655 main.go:134] libmachine: (minikube) DBG | Using libvirt version 8004000
I0628 09:54:03.203257    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.205717    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.205827    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.206187    5655 main.go:134] libmachine: Docker is up and running!
I0628 09:54:03.206205    5655 main.go:134] libmachine: Reticulating splines...
I0628 09:54:03.206219    5655 client.go:171] LocalClient.Create took 26.895626306s
I0628 09:54:03.206249    5655 start.go:173] duration metric: libmachine.API.Create for "minikube" took 26.895752181s
I0628 09:54:03.206264    5655 start.go:306] post-start starting for "minikube" (driver="kvm2")
I0628 09:54:03.206271    5655 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0628 09:54:03.206314    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.206675    5655 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0628 09:54:03.206751    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:03.246434    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.249956    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.250006    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.250430    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:03.250779    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:03.250997    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:03.251273    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:03.356335    5655 ssh_runner.go:195] Run: cat /etc/os-release
I0628 09:54:03.362641    5655 info.go:137] Remote host: Buildroot 2021.02.12
I0628 09:54:03.362676    5655 filesync.go:126] Scanning /home/****/.minikube/addons for local assets ...
I0628 09:54:03.362770    5655 filesync.go:126] Scanning /home/****/.minikube/files for local assets ...
I0628 09:54:03.362792    5655 start.go:309] post-start completed in 156.521151ms
I0628 09:54:03.362826    5655 main.go:134] libmachine: (minikube) Calling .GetConfigRaw
I0628 09:54:03.363699    5655 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:54:03.400354    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.403215    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.403319    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.403922    5655 profile.go:148] Saving config to /home/****/.minikube/profiles/minikube/config.json ...
I0628 09:54:03.404196    5655 start.go:134] duration metric: createHost completed in 27.131145063s
I0628 09:54:03.404231    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:03.445254    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.448865    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.448890    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.449403    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:03.449650    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:03.449896    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:03.450178    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:03.450418    5655 main.go:134] libmachine: Using SSH client type: native
I0628 09:54:03.450650    5655 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:54:03.450664    5655 main.go:134] libmachine: About to run SSH command:
date +%s.%N
I0628 09:54:03.595502    5655 main.go:134] libmachine: SSH cmd err, output: <nil>: 1656424443.579727052

I0628 09:54:03.595600    5655 fix.go:207] guest clock: 1656424443.579727052
I0628 09:54:03.595633    5655 fix.go:220] Guest: 2022-06-28 09:54:03.579727052 -0400 EDT Remote: 2022-06-28 09:54:03.404217723 -0400 EDT m=+38.676282081 (delta=175.509329ms)
I0628 09:54:03.595722    5655 fix.go:191] guest clock delta is within tolerance: 175.509329ms
I0628 09:54:03.595762    5655 start.go:81] releasing machines lock for "minikube", held for 27.322899909s
I0628 09:54:03.595865    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.596368    5655 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:54:03.636649    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.639977    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.640019    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.640537    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.640966    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.641190    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.642191    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.642412    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:03.642613    5655 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0628 09:54:03.642711    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:03.642921    5655 ssh_runner.go:195] Run: systemctl --version
I0628 09:54:03.642948    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:03.691427    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.694084    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.694112    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.694681    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:03.695018    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:03.695321    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:03.695594    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:03.708923    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.714136    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:03.714184    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:03.714622    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:03.714966    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:03.715259    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:03.715541    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:03.879472    5655 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:54:03.879980    5655 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0628 09:54:03.925715    5655 docker.go:602] Got preloaded images: 
I0628 09:54:03.925746    5655 docker.go:608] k8s.gcr.io/kube-apiserver:v1.24.1 wasn't preloaded
I0628 09:54:03.925806    5655 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0628 09:54:03.936897    5655 ssh_runner.go:195] Run: which lz4
I0628 09:54:03.940645    5655 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0628 09:54:03.944655    5655 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:

stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0628 09:54:03.944706    5655 ssh_runner.go:362] scp /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (425543115 bytes)
I0628 09:54:05.178996    5655 docker.go:567] Took 1.238404 seconds to copy over tarball
I0628 09:54:05.179101    5655 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0628 09:54:08.092979    5655 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.913790403s)
I0628 09:54:08.093023    5655 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0628 09:54:08.133681    5655 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0628 09:54:08.153011    5655 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2523 bytes)
I0628 09:54:08.176792    5655 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0628 09:54:08.297346    5655 ssh_runner.go:195] Run: sudo systemctl restart docker
I0628 09:54:11.221061    5655 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.923604098s)
I0628 09:54:11.221563    5655 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0628 09:54:11.240133    5655 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0628 09:54:11.255959    5655 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0628 09:54:11.278917    5655 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0628 09:54:11.315807    5655 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0628 09:54:11.337104    5655 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
image-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0628 09:54:11.365255    5655 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0628 09:54:11.486030    5655 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0628 09:54:11.618570    5655 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0628 09:54:11.755709    5655 ssh_runner.go:195] Run: sudo systemctl restart docker
I0628 09:54:13.275853    5655 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.520096463s)
I0628 09:54:13.275948    5655 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0628 09:54:13.398363    5655 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0628 09:54:13.526193    5655 ssh_runner.go:195] Run: sudo systemctl start cri-docker.socket
I0628 09:54:13.547730    5655 start.go:447] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0628 09:54:13.547797    5655 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0628 09:54:13.554472    5655 start.go:468] Will wait 60s for crictl version
I0628 09:54:13.554527    5655 ssh_runner.go:195] Run: sudo crictl version
I0628 09:54:13.707103    5655 start.go:477] Version:  0.1.0
RuntimeName:  docker
RuntimeVersion:  20.10.16
RuntimeApiVersion:  1.41.0
I0628 09:54:13.707205    5655 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0628 09:54:13.745411    5655 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0628 09:54:13.789849    5655 out.go:204] * Preparing Kubernetes v1.24.1 on Docker 20.10.16 ...
* Preparing Kubernetes v1.24.1 on Docker 20.10.16 ...
I0628 09:54:13.790181    5655 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:54:13.830446    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:13.833564    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:13.833700    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:13.834190    5655 ssh_runner.go:195] Run: grep 192.168.39.1	host.minikube.internal$ /etc/hosts
I0628 09:54:13.838786    5655 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.39.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0628 09:54:13.853300    5655 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:54:13.853590    5655 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0628 09:54:13.885570    5655 docker.go:602] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.24.1
k8s.gcr.io/kube-controller-manager:v1.24.1
k8s.gcr.io/kube-proxy:v1.24.1
k8s.gcr.io/kube-scheduler:v1.24.1
k8s.gcr.io/etcd:3.5.3-0
k8s.gcr.io/pause:3.7
k8s.gcr.io/coredns/coredns:v1.8.6
gcr.io/k8s-minikube/storage-provisioner:v5

-- /stdout --
I0628 09:54:13.885619    5655 docker.go:533] Images already preloaded, skipping extraction
I0628 09:54:13.885767    5655 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0628 09:54:13.914034    5655 docker.go:602] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.24.1
k8s.gcr.io/kube-scheduler:v1.24.1
k8s.gcr.io/kube-controller-manager:v1.24.1
k8s.gcr.io/kube-proxy:v1.24.1
k8s.gcr.io/etcd:3.5.3-0
k8s.gcr.io/pause:3.7
k8s.gcr.io/coredns/coredns:v1.8.6
gcr.io/k8s-minikube/storage-provisioner:v5

-- /stdout --
I0628 09:54:13.914095    5655 cache_images.go:84] Images are preloaded, skipping loading
I0628 09:54:13.914228    5655 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0628 09:54:13.952347    5655 cni.go:95] Creating CNI manager for ""
I0628 09:54:13.952387    5655 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0628 09:54:13.952432    5655 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0628 09:54:13.952485    5655 kubeadm.go:158] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.235 APIServerPort:8443 KubernetesVersion:v1.24.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.235"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.39.235 CgroupDriver:systemd ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0628 09:54:13.952718    5655 kubeadm.go:162] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.39.235
  bindPort: 8443
bootstrapTokens:
  - groups:
      - system:bootstrappers:kubeadm:default-node-token
    ttl: 24h0m0s
    usages:
      - signing
      - authentication
nodeRegistration:
  criSocket: /var/run/cri-dockerd.sock
  name: "minikube"
  kubeletExtraArgs:
    node-ip: 192.168.39.235
  taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
  certSANs: ["127.0.0.1", "localhost", "192.168.39.235"]
  extraArgs:
    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
  extraArgs:
    allocate-node-cidrs: "true"
    leader-elect: "false"
scheduler:
  extraArgs:
    leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
  local:
    dataDir: /var/lib/minikube/etcd
    extraArgs:
      proxy-refresh-interval: "70000"
kubernetesVersion: v1.24.1
networking:
  dnsDomain: cluster.local
  podSubnet: "10.244.0.0/16"
  serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  x509:
    clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: systemd
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
  nodefs.available: "0%"
  nodefs.inodesFree: "0%"
  imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
  maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
  tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
  tcpCloseWaitTimeout: 0s

I0628 09:54:13.952895    5655 kubeadm.go:961] kubelet [Unit]
Wants=docker.socket

[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.24.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/var/run/cri-dockerd.sock --hostname-override=minikube --image-service-endpoint=/var/run/cri-dockerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.39.235 --runtime-request-timeout=15m

[Install]
 config:
{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0628 09:54:13.953021    5655 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.24.1
I0628 09:54:13.965345    5655 binaries.go:44] Found k8s binaries, skipping transfer
I0628 09:54:13.965464    5655 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0628 09:54:13.974942    5655 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (472 bytes)
I0628 09:54:13.996339    5655 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0628 09:54:14.014681    5655 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2036 bytes)
I0628 09:54:14.034096    5655 ssh_runner.go:195] Run: grep 192.168.39.235	control-plane.minikube.internal$ /etc/hosts
I0628 09:54:14.037818    5655 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.39.235	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0628 09:54:14.050676    5655 certs.go:54] Setting up /home/****/.minikube/profiles/minikube for IP: 192.168.39.235
I0628 09:54:14.052113    5655 certs.go:182] skipping minikubeCA CA generation: /home/****/.minikube/ca.key
I0628 09:54:14.052917    5655 certs.go:182] skipping proxyClientCA CA generation: /home/****/.minikube/proxy-client-ca.key
I0628 09:54:14.053011    5655 certs.go:302] generating minikube-user signed cert: /home/****/.minikube/profiles/minikube/client.key
I0628 09:54:14.053062    5655 crypto.go:68] Generating cert /home/****/.minikube/profiles/minikube/client.crt with IP's: []
I0628 09:54:14.270796    5655 crypto.go:156] Writing cert to /home/****/.minikube/profiles/minikube/client.crt ...
I0628 09:54:14.270828    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/client.crt: {Name:mkcf46dbd74ca4d6c5528c57a186bcb18418aa4a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.271330    5655 crypto.go:164] Writing key to /home/****/.minikube/profiles/minikube/client.key ...
I0628 09:54:14.271348    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/client.key: {Name:mk0d1e88324baf9378768466fe2447b9a13a31c1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.271474    5655 certs.go:302] generating minikube signed cert: /home/****/.minikube/profiles/minikube/apiserver.key.df2bd094
I0628 09:54:14.271489    5655 crypto.go:68] Generating cert /home/****/.minikube/profiles/minikube/apiserver.crt.df2bd094 with IP's: [192.168.39.235 10.96.0.1 127.0.0.1 10.0.0.1]
I0628 09:54:14.563858    5655 crypto.go:156] Writing cert to /home/****/.minikube/profiles/minikube/apiserver.crt.df2bd094 ...
I0628 09:54:14.563897    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/apiserver.crt.df2bd094: {Name:mkf13b5419c1b7e602ec63cc58a4757d2114a4eb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.564242    5655 crypto.go:164] Writing key to /home/****/.minikube/profiles/minikube/apiserver.key.df2bd094 ...
I0628 09:54:14.564258    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/apiserver.key.df2bd094: {Name:mka745e3d2c59a2a00edfd17649d23b6da9226c7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.564390    5655 certs.go:320] copying /home/****/.minikube/profiles/minikube/apiserver.crt.df2bd094 -> /home/****/.minikube/profiles/minikube/apiserver.crt
I0628 09:54:14.565004    5655 certs.go:324] copying /home/****/.minikube/profiles/minikube/apiserver.key.df2bd094 -> /home/****/.minikube/profiles/minikube/apiserver.key
I0628 09:54:14.565281    5655 certs.go:302] generating aggregator signed cert: /home/****/.minikube/profiles/minikube/proxy-client.key
I0628 09:54:14.565308    5655 crypto.go:68] Generating cert /home/****/.minikube/profiles/minikube/proxy-client.crt with IP's: []
I0628 09:54:14.893173    5655 crypto.go:156] Writing cert to /home/****/.minikube/profiles/minikube/proxy-client.crt ...
I0628 09:54:14.893201    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/proxy-client.crt: {Name:mk1e3aac95504e914c96cdc9c070f2e0c7e06d44 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.893504    5655 crypto.go:164] Writing key to /home/****/.minikube/profiles/minikube/proxy-client.key ...
I0628 09:54:14.893515    5655 lock.go:35] WriteFile acquiring /home/****/.minikube/profiles/minikube/proxy-client.key: {Name:mk981212a67cb6fc7c7d3201dc8dc22c8a0e3819 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:14.893749    5655 certs.go:388] found cert: /home/****/.minikube/certs/home/****/.minikube/certs/ca-key.pem (1675 bytes)
I0628 09:54:14.893797    5655 certs.go:388] found cert: /home/****/.minikube/certs/home/****/.minikube/certs/ca.pem (1082 bytes)
I0628 09:54:14.893826    5655 certs.go:388] found cert: /home/****/.minikube/certs/home/****/.minikube/certs/cert.pem (1127 bytes)
I0628 09:54:14.893850    5655 certs.go:388] found cert: /home/****/.minikube/certs/home/****/.minikube/certs/key.pem (1679 bytes)
I0628 09:54:14.894897    5655 ssh_runner.go:362] scp /home/****/.minikube/profiles/minikube/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0628 09:54:14.930834    5655 ssh_runner.go:362] scp /home/****/.minikube/profiles/minikube/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0628 09:54:14.961936    5655 ssh_runner.go:362] scp /home/****/.minikube/profiles/minikube/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0628 09:54:14.990447    5655 ssh_runner.go:362] scp /home/****/.minikube/profiles/minikube/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0628 09:54:15.016230    5655 ssh_runner.go:362] scp /home/****/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0628 09:54:15.043281    5655 ssh_runner.go:362] scp /home/****/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0628 09:54:15.075471    5655 ssh_runner.go:362] scp /home/****/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0628 09:54:15.101028    5655 ssh_runner.go:362] scp /home/****/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0628 09:54:15.130230    5655 ssh_runner.go:362] scp /home/****/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0628 09:54:15.156921    5655 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0628 09:54:15.176727    5655 ssh_runner.go:195] Run: openssl version
I0628 09:54:15.182916    5655 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0628 09:54:15.193269    5655 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0628 09:54:15.198010    5655 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Jun 28 01:28 /usr/share/ca-certificates/minikubeCA.pem
I0628 09:54:15.198072    5655 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0628 09:54:15.204079    5655 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0628 09:54:15.218092    5655 kubeadm.go:395] StartCluster: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[192.168.111.1:5000] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.39.235 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/****:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0628 09:54:15.218297    5655 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0628 09:54:15.254499    5655 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0628 09:54:15.265850    5655 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0628 09:54:15.278378    5655 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0628 09:54:15.288252    5655 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:

stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0628 09:54:15.288346    5655 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem"
I0628 09:54:36.507820    5655 out.go:204]   - Generating certificates and keys ...
  - Generating certificates and keys ...
I0628 09:54:36.513755    5655 out.go:204]   - Booting up control plane ...
  - Booting up control plane ...
I0628 09:54:36.517976    5655 out.go:204]   - Configuring RBAC rules ...
  - Configuring RBAC rules ...
I0628 09:54:36.521564    5655 cni.go:95] Creating CNI manager for ""
I0628 09:54:36.521599    5655 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0628 09:54:36.521660    5655 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0628 09:54:36.522099    5655 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.1/kubectl label nodes minikube.k8s.io/version=v1.26.0 minikube.k8s.io/commit=f4b412861bb746be73053c9f6d2895f12cf78565 minikube.k8s.io/name=minikube minikube.k8s.io/updated_at=2022_06_28T09_54_36_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
I0628 09:54:36.522113    5655 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.1/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0628 09:54:36.840361    5655 ops.go:34] apiserver oom_adj: -16
I0628 09:54:36.840426    5655 kubeadm.go:1045] duration metric: took 318.463432ms to wait for elevateKubeSystemPrivileges.
I0628 09:54:36.840453    5655 kubeadm.go:397] StartCluster complete in 21.622380955s
I0628 09:54:36.840496    5655 settings.go:142] acquiring lock: {Name:mk89b746975186c04d4f93439907b25163de0898 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:36.840898    5655 settings.go:150] Updating kubeconfig:  /home/****/.kube/config
I0628 09:54:36.843684    5655 lock.go:35] WriteFile acquiring /home/****/.kube/config: {Name:mk28299c4abaf4f57b926be5082ba29d19125674 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0628 09:54:37.383551    5655 kapi.go:244] deployment "coredns" in namespace "kube-system" and context "minikube" rescaled to 1
I0628 09:54:37.383663    5655 start.go:208] Will wait 6m0s for node &{Name: IP:192.168.39.235 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0628 09:54:37.387450    5655 out.go:177] * Verifying Kubernetes components...
* Verifying Kubernetes components...
I0628 09:54:37.383799    5655 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0628 09:54:37.383884    5655 addons.go:412] enableAddons start: toEnable=map[], additional=[]
I0628 09:54:37.384383    5655 config.go:178] Loaded profile config "minikube": Driver=kvm2, ContainerRuntime=docker, KubernetesVersion=v1.24.1
I0628 09:54:37.390560    5655 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0628 09:54:37.390726    5655 addons.go:65] Setting storage-provisioner=true in profile "minikube"
I0628 09:54:37.390773    5655 addons.go:65] Setting default-storageclass=true in profile "minikube"
I0628 09:54:37.390788    5655 addons.go:153] Setting addon storage-provisioner=true in "minikube"
W0628 09:54:37.390798    5655 addons.go:162] addon storage-provisioner should already be in state true
I0628 09:54:37.390801    5655 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "minikube"
I0628 09:54:37.390925    5655 host.go:66] Checking if "minikube" exists ...
I0628 09:54:37.391286    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:54:37.391338    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:54:37.391370    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:54:37.391403    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:54:37.414504    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:41931
I0628 09:54:37.414587    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:33247
I0628 09:54:37.415176    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:54:37.415230    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:54:37.415631    5655 main.go:134] libmachine: Using API Version  1
I0628 09:54:37.415645    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:54:37.415963    5655 main.go:134] libmachine: Using API Version  1
I0628 09:54:37.415976    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:54:37.416007    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:54:37.416161    5655 main.go:134] libmachine: (minikube) Calling .GetState
I0628 09:54:37.416272    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:54:37.416664    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:54:37.416692    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:54:37.435344    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:33067
I0628 09:54:37.436950    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:54:37.438098    5655 main.go:134] libmachine: Using API Version  1
I0628 09:54:37.438114    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:54:37.438446    5655 addons.go:153] Setting addon default-storageclass=true in "minikube"
W0628 09:54:37.438463    5655 addons.go:162] addon default-storageclass should already be in state true
I0628 09:54:37.438486    5655 host.go:66] Checking if "minikube" exists ...
I0628 09:54:37.438562    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:54:37.438744    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:54:37.438770    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:54:37.438777    5655 main.go:134] libmachine: (minikube) Calling .GetState
I0628 09:54:37.458314    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:37.464281    5655 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5
  - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0628 09:54:37.459387    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:37759
I0628 09:54:37.467989    5655 addons.go:345] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0628 09:54:37.468004    5655 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0628 09:54:37.468021    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:37.469923    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:54:37.470343    5655 main.go:134] libmachine: Using API Version  1
I0628 09:54:37.470355    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:54:37.470690    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:54:37.471215    5655 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:54:37.471257    5655 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:54:37.496418    5655 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:38969
I0628 09:54:37.496970    5655 main.go:134] libmachine: () Calling .GetVersion
I0628 09:54:37.497567    5655 main.go:134] libmachine: Using API Version  1
I0628 09:54:37.497581    5655 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:54:37.498524    5655 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:54:37.498598    5655 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.39.1 host.minikube.internal\n           fallthrough\n        }' | sudo /var/lib/minikube/binaries/v1.24.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0628 09:54:37.498681    5655 main.go:134] libmachine: (minikube) Calling .GetState
I0628 09:54:37.499338    5655 api_server.go:51] waiting for apiserver process to appear ...
I0628 09:54:37.499381    5655 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0628 09:54:37.512921    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:37.516533    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:37.516550    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:37.517016    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:37.522379    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:37.522712    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:37.523107    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:37.530251    5655 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:54:37.532409    5655 addons.go:345] installing /etc/kubernetes/addons/storageclass.yaml
I0628 09:54:37.532425    5655 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0628 09:54:37.532443    5655 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:54:37.576792    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:37.579185    5655 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:53:53 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:54:37.579216    5655 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:54:37.580004    5655 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:54:37.580342    5655 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:54:37.580513    5655 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:54:37.580786    5655 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:54:37.649177    5655 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0628 09:54:37.772874    5655 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0628 09:54:38.501681    5655 ssh_runner.go:235] Completed: sudo pgrep -xnf kube-apiserver.*minikube.*: (1.002246448s)
I0628 09:54:38.501743    5655 api_server.go:71] duration metric: took 1.11802494s to wait for apiserver process to appear ...
I0628 09:54:38.501758    5655 api_server.go:87] waiting for apiserver healthz status ...
I0628 09:54:38.501791    5655 api_server.go:240] Checking apiserver healthz at https://192.168.39.235:8443/healthz ...
I0628 09:54:38.503669    5655 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.39.1 host.minikube.internal\n           fallthrough\n        }' | sudo /var/lib/minikube/binaries/v1.24.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (1.005029619s)
I0628 09:54:38.503791    5655 start.go:806] {"host.minikube.internal": 192.168.39.1} host record injected into CoreDNS
I0628 09:54:38.510629    5655 api_server.go:266] https://192.168.39.235:8443/healthz returned 200:
ok
I0628 09:54:38.512026    5655 api_server.go:140] control plane version: v1.24.1
I0628 09:54:38.512177    5655 api_server.go:130] duration metric: took 10.406358ms to wait for apiserver health ...
I0628 09:54:38.512289    5655 system_pods.go:43] waiting for kube-system pods to appear ...
I0628 09:54:38.527500    5655 system_pods.go:59] 4 kube-system pods found
I0628 09:54:38.527668    5655 system_pods.go:61] "etcd-minikube" [7bef15a2-d644-472c-bfba-9b4571203957] Pending
I0628 09:54:38.527725    5655 system_pods.go:61] "kube-apiserver-minikube" [04bc0252-08b0-4b79-867a-6010436e683b] Pending
I0628 09:54:38.527787    5655 system_pods.go:61] "kube-controller-manager-minikube" [32551941-6935-4d28-ae17-d38ba2aa021d] Pending
I0628 09:54:38.527837    5655 system_pods.go:61] "kube-scheduler-minikube" [6b88af1f-fb37-4e53-8e87-ceba8ba9107d] Pending
I0628 09:54:38.527876    5655 system_pods.go:74] duration metric: took 15.506355ms to wait for pod list to return data ...
I0628 09:54:38.527918    5655 kubeadm.go:572] duration metric: took 1.14421237s to wait for : map[apiserver:true system_pods:true] ...
I0628 09:54:38.527996    5655 node_conditions.go:102] verifying NodePressure condition ...
I0628 09:54:38.533045    5655 node_conditions.go:122] node storage ephemeral capacity is 17784752Ki
I0628 09:54:38.533279    5655 node_conditions.go:123] node cpu capacity is 2
I0628 09:54:38.533378    5655 node_conditions.go:105] duration metric: took 5.327871ms to run NodePressure ...
I0628 09:54:38.533484    5655 start.go:213] waiting for startup goroutines ...
I0628 09:54:38.622506    5655 main.go:134] libmachine: Making call to close driver server
I0628 09:54:38.622552    5655 main.go:134] libmachine: (minikube) Calling .Close
I0628 09:54:38.623470    5655 main.go:134] libmachine: Making call to close driver server
I0628 09:54:38.623504    5655 main.go:134] libmachine: (minikube) Calling .Close
I0628 09:54:38.624131    5655 main.go:134] libmachine: (minikube) DBG | Closing plugin on server side
I0628 09:54:38.624137    5655 main.go:134] libmachine: (minikube) DBG | Closing plugin on server side
I0628 09:54:38.624173    5655 main.go:134] libmachine: Successfully made call to close driver server
I0628 09:54:38.624186    5655 main.go:134] libmachine: Making call to close connection to plugin binary
I0628 09:54:38.624194    5655 main.go:134] libmachine: Making call to close driver server
I0628 09:54:38.624202    5655 main.go:134] libmachine: (minikube) Calling .Close
I0628 09:54:38.624177    5655 main.go:134] libmachine: Successfully made call to close driver server
I0628 09:54:38.624256    5655 main.go:134] libmachine: Making call to close connection to plugin binary
I0628 09:54:38.624271    5655 main.go:134] libmachine: Making call to close driver server
I0628 09:54:38.624284    5655 main.go:134] libmachine: (minikube) Calling .Close
I0628 09:54:38.626159    5655 main.go:134] libmachine: Successfully made call to close driver server
I0628 09:54:38.626174    5655 main.go:134] libmachine: Making call to close connection to plugin binary
I0628 09:54:38.626185    5655 main.go:134] libmachine: Making call to close driver server
I0628 09:54:38.626186    5655 main.go:134] libmachine: (minikube) DBG | Closing plugin on server side
I0628 09:54:38.626191    5655 main.go:134] libmachine: (minikube) Calling .Close
I0628 09:54:38.626233    5655 main.go:134] libmachine: (minikube) DBG | Closing plugin on server side
I0628 09:54:38.626251    5655 main.go:134] libmachine: Successfully made call to close driver server
I0628 09:54:38.626264    5655 main.go:134] libmachine: Making call to close connection to plugin binary
I0628 09:54:38.626597    5655 main.go:134] libmachine: Successfully made call to close driver server
I0628 09:54:38.626608    5655 main.go:134] libmachine: Making call to close connection to plugin binary
I0628 09:54:38.724545    5655 out.go:177] * Enabled addons: storage-provisioner, default-storageclass
* Enabled addons: storage-provisioner, default-storageclass
I0628 09:54:38.741128    5655 addons.go:414] enableAddons completed in 1.357262174s
I0628 09:54:39.533365    5655 start.go:503] kubectl: 1.24.2, cluster: 1.24.1 (minor skew: 0)
I0628 09:54:39.580668    5655 out.go:177] * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
+ [[ 0 -eq 0 ]]
+ break
+ sudo su -l -c 'minikube stop' ****
* Stopping node "minikube"  ...
* 1 node stopped.



sudo su -l -c 'minikube start --driver=kvm2 --alsologtostderr' ****
I0628 09:59:29.048044    9441 out.go:296] Setting OutFile to fd 1 ...
I0628 09:59:29.048437    9441 out.go:343] TERM=unknown,COLORTERM=, which probably does not support color
I0628 09:59:29.048511    9441 out.go:309] Setting ErrFile to fd 2...
I0628 09:59:29.048550    9441 out.go:343] TERM=unknown,COLORTERM=, which probably does not support color
I0628 09:59:29.048814    9441 root.go:329] Updating PATH: /home/****/.minikube/bin
I0628 09:59:29.049136    9441 out.go:303] Setting JSON to false
I0628 09:59:29.050523    9441 start.go:115] hostinfo: {"hostname":"ci-test-vm-20220628134925-nxgi.openstacklocal","uptime":575,"bootTime":1656424195,"procs":153,"os":"linux","platform":"centos","platformFamily":"rhel","platformVersion":"9","kernelVersion":"5.14.0-115.el9.x86_64","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"8533cce4-4e87-42fe-9942-fd5b2aba60d3"}
I0628 09:59:29.050693    9441 start.go:125] virtualization: kvm guest
I0628 09:59:29.072869    9441 out.go:177] * minikube v1.26.0 on Centos 9 (kvm/amd64)
* minikube v1.26.0 on Centos 9 (kvm/amd64)
I0628 09:59:29.085708    9441 notify.go:193] Checking for updates...
I0628 09:59:29.086208    9441 config.go:178] Loaded profile config "minikube": Driver=kvm2, ContainerRuntime=docker, KubernetesVersion=v1.24.1
I0628 09:59:29.086924    9441 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:59:29.087166    9441 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:59:29.103472    9441 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:45167
I0628 09:59:29.104797    9441 main.go:134] libmachine: () Calling .GetVersion
I0628 09:59:29.105706    9441 main.go:134] libmachine: Using API Version  1
I0628 09:59:29.105735    9441 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:59:29.106167    9441 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:59:29.106500    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:29.106858    9441 driver.go:360] Setting default libvirt URI to qemu:///system
I0628 09:59:29.107206    9441 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:59:29.107240    9441 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:59:29.125121    9441 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:46721
I0628 09:59:29.126515    9441 main.go:134] libmachine: () Calling .GetVersion
I0628 09:59:29.127347    9441 main.go:134] libmachine: Using API Version  1
I0628 09:59:29.127362    9441 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:59:29.127678    9441 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:59:29.127847    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:29.306786    9441 out.go:177] * Using the kvm2 driver based on existing profile
* Using the kvm2 driver based on existing profile
I0628 09:59:29.326663    9441 start.go:284] selected driver: kvm2
I0628 09:59:29.326699    9441 start.go:805] validating driver "kvm2" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[192.168.111.1:5000] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.39.235 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/****:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0628 09:59:29.326877    9441 start.go:816] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0628 09:59:29.327273    9441 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0628 09:59:29.327658    9441 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/****/.minikube/bin:/home/****/.krew/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/local/go/bin
I0628 09:59:29.354226    9441 install.go:137] /home/****/.minikube/bin/docker-machine-driver-kvm2 version is 1.26.0
I0628 09:59:29.354930    9441 cni.go:95] Creating CNI manager for ""
I0628 09:59:29.354953    9441 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0628 09:59:29.354971    9441 start_flags.go:310] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.26.0-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.32@sha256:9190bd2393eae887316c97a74370b7d5dad8f0b2ef91ac2662bc36f7ef8e0b95 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[192.168.111.1:5000] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.39.235 Port:8443 KubernetesVersion:v1.24.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/****:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0628 09:59:29.355580    9441 iso.go:128] acquiring lock: {Name:mkfd1ab3d05ae9ca1111d9e050f9880c81c13514 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0628 09:59:29.378087    9441 out.go:177] * Starting control plane node minikube in cluster minikube
* Starting control plane node minikube in cluster minikube
I0628 09:59:29.381664    9441 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:59:29.381747    9441 preload.go:148] Found local preload: /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4
I0628 09:59:29.381790    9441 cache.go:57] Caching tarball of preloaded images
I0628 09:59:29.382106    9441 preload.go:174] Found /home/****/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.1-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0628 09:59:29.382172    9441 cache.go:60] Finished verifying existence of preloaded tar for  v1.24.1 on docker
I0628 09:59:29.382331    9441 profile.go:148] Saving config to /home/****/.minikube/profiles/minikube/config.json ...
I0628 09:59:29.382730    9441 cache.go:208] Successfully downloaded all kic artifacts
I0628 09:59:29.382790    9441 start.go:352] acquiring machines lock for minikube: {Name:mk405a3db8e483af8dee0a375e4dabc91c94683c Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0628 09:59:29.382902    9441 start.go:356] acquired machines lock for "minikube" in 87.69µs
I0628 09:59:29.382939    9441 start.go:94] Skipping create...Using existing machine configuration
I0628 09:59:29.382952    9441 fix.go:55] fixHost starting: 
I0628 09:59:29.383458    9441 main.go:134] libmachine: Found binary path at /home/****/.minikube/bin/docker-machine-driver-kvm2
I0628 09:59:29.383506    9441 main.go:134] libmachine: Launching plugin server for driver kvm2
I0628 09:59:29.404580    9441 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:43615
I0628 09:59:29.405403    9441 main.go:134] libmachine: () Calling .GetVersion
I0628 09:59:29.406120    9441 main.go:134] libmachine: Using API Version  1
I0628 09:59:29.406138    9441 main.go:134] libmachine: () Calling .SetConfigRaw
I0628 09:59:29.406764    9441 main.go:134] libmachine: () Calling .GetMachineName
I0628 09:59:29.407228    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:29.407582    9441 main.go:134] libmachine: (minikube) Calling .GetState
I0628 09:59:29.431603    9441 fix.go:103] recreateIfNeeded on minikube: state=Stopped err=<nil>
I0628 09:59:29.431668    9441 main.go:134] libmachine: (minikube) Calling .DriverName
W0628 09:59:29.432169    9441 fix.go:129] unexpected machine state, will restart: <nil>
I0628 09:59:29.450340    9441 out.go:177] * Restarting existing kvm2 VM for "minikube" ...
* Restarting existing kvm2 VM for "minikube" ...
I0628 09:59:29.456765    9441 main.go:134] libmachine: (minikube) Calling .Start
I0628 09:59:29.457223    9441 main.go:134] libmachine: (minikube) Ensuring networks are active...
I0628 09:59:29.477128    9441 main.go:134] libmachine: (minikube) Ensuring network default is active
I0628 09:59:30.068722    9441 main.go:134] libmachine: (minikube) Ensuring network mk-minikube is active
I0628 09:59:30.070593    9441 main.go:134] libmachine: (minikube) Getting domain xml...
I0628 09:59:30.091160    9441 main.go:134] libmachine: (minikube) Creating domain...
I0628 09:59:30.862857    9441 main.go:134] libmachine: (minikube) Waiting to get IP...
I0628 09:59:30.864332    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:30.865351    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:30.865649    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:30.865448    9576 retry.go:31] will retry after 263.082536ms: waiting for machine to come up
I0628 09:59:31.131392    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:31.132061    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:31.132083    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:31.132020    9576 retry.go:31] will retry after 381.329545ms: waiting for machine to come up
I0628 09:59:31.515446    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:31.516906    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:31.516950    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:31.516779    9576 retry.go:31] will retry after 422.765636ms: waiting for machine to come up
I0628 09:59:31.942587    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:31.943521    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:31.943564    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:31.943450    9576 retry.go:31] will retry after 473.074753ms: waiting for machine to come up
I0628 09:59:32.418894    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:32.419752    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:32.419808    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:32.419685    9576 retry.go:31] will retry after 587.352751ms: waiting for machine to come up
I0628 09:59:33.009466    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:33.010303    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:33.010338    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:33.010186    9576 retry.go:31] will retry after 834.206799ms: waiting for machine to come up
I0628 09:59:33.847134    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:33.848035    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:33.848134    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:33.848047    9576 retry.go:31] will retry after 746.553905ms: waiting for machine to come up
I0628 09:59:34.596930    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:34.598127    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:34.598278    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:34.598220    9576 retry.go:31] will retry after 987.362415ms: waiting for machine to come up
I0628 09:59:35.587990    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:35.588786    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:35.588817    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:35.588735    9576 retry.go:31] will retry after 1.189835008s: waiting for machine to come up
I0628 09:59:36.781354    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:36.782875    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:36.783005    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:36.782957    9576 retry.go:31] will retry after 1.677229867s: waiting for machine to come up
I0628 09:59:38.464573    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:38.465509    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:38.465554    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:38.465376    9576 retry.go:31] will retry after 2.346016261s: waiting for machine to come up
I0628 09:59:40.814308    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:40.814942    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:40.814981    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:40.814821    9576 retry.go:31] will retry after 3.36678925s: waiting for machine to come up
I0628 09:59:44.185705    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:44.186647    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:44.186672    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:44.186600    9576 retry.go:31] will retry after 3.11822781s: waiting for machine to come up
I0628 09:59:47.309213    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:47.310068    9441 main.go:134] libmachine: (minikube) DBG | unable to find current IP address of domain minikube in network mk-minikube
I0628 09:59:47.310101    9441 main.go:134] libmachine: (minikube) DBG | I0628 09:59:47.310022    9576 retry.go:31] will retry after 4.276119362s: waiting for machine to come up
I0628 09:59:51.589710    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.590752    9441 main.go:134] libmachine: (minikube) Found IP for machine: 192.168.39.235
I0628 09:59:51.590822    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has current primary IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.590846    9441 main.go:134] libmachine: (minikube) Reserving static IP address...
I0628 09:59:51.594746    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "minikube", mac: "52:54:00:45:cf:30", ip: "192.168.39.235"} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:51.594793    9441 main.go:134] libmachine: (minikube) Reserved static IP address: 192.168.39.235
I0628 09:59:51.594814    9441 main.go:134] libmachine: (minikube) DBG | skip adding static IP to network mk-minikube - found existing host DHCP lease matching {name: "minikube", mac: "52:54:00:45:cf:30", ip: "192.168.39.235"}
I0628 09:59:51.594838    9441 main.go:134] libmachine: (minikube) DBG | Getting to WaitForSSH function...
I0628 09:59:51.594856    9441 main.go:134] libmachine: (minikube) Waiting for SSH to be available...
I0628 09:59:51.640924    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.644284    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:51.644322    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.644664    9441 main.go:134] libmachine: (minikube) DBG | Using SSH client type: external
I0628 09:59:51.644786    9441 main.go:134] libmachine: (minikube) DBG | Using SSH private key: /home/****/.minikube/machines/minikube/id_rsa (-rw-------)
I0628 09:59:51.644829    9441 main.go:134] libmachine: (minikube) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.235 -o IdentitiesOnly=yes -i /home/****/.minikube/machines/minikube/id_rsa -p 22] /usr/bin/ssh <nil>}
I0628 09:59:51.644863    9441 main.go:134] libmachine: (minikube) DBG | About to run SSH command:
I0628 09:59:51.644875    9441 main.go:134] libmachine: (minikube) DBG | exit 0
I0628 09:59:51.768883    9441 main.go:134] libmachine: (minikube) DBG | SSH cmd err, output: <nil>: 
I0628 09:59:51.769633    9441 main.go:134] libmachine: (minikube) Calling .GetConfigRaw
I0628 09:59:51.771090    9441 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:59:51.811735    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.815178    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:51.815270    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.815771    9441 profile.go:148] Saving config to /home/****/.minikube/profiles/minikube/config.json ...
I0628 09:59:51.815993    9441 machine.go:88] provisioning docker machine ...
I0628 09:59:51.816011    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:51.816375    9441 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:59:51.816604    9441 buildroot.go:166] provisioning hostname "minikube"
I0628 09:59:51.816623    9441 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:59:51.816808    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:51.853522    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.856592    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:51.856635    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:51.856986    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:51.857272    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:51.857460    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:51.857626    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:51.857805    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:51.858126    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:51.858162    9441 main.go:134] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I0628 09:59:52.008229    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: minikube

I0628 09:59:52.008291    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:52.051694    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.055406    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:52.055486    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.055676    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:52.055940    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:52.056273    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:52.056466    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:52.056712    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:52.056960    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:52.057019    9441 main.go:134] libmachine: About to run SSH command:

		if ! grep -xq '.*\sminikube' /etc/hosts; then
			if grep -xq '127.0.1.1\s.*' /etc/hosts; then
				sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
			else 
				echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; 
			fi
		fi
I0628 09:59:52.195001    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: 
I0628 09:59:52.195080    9441 buildroot.go:172] set auth options {CertDir:/home/****/.minikube CaCertPath:/home/****/.minikube/certs/ca.pem CaPrivateKeyPath:/home/****/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/****/.minikube/machines/server.pem ServerKeyPath:/home/****/.minikube/machines/server-key.pem ClientKeyPath:/home/****/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/****/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/****/.minikube}
I0628 09:59:52.195107    9441 buildroot.go:174] setting up certificates
I0628 09:59:52.195126    9441 provision.go:83] configureAuth start
I0628 09:59:52.195157    9441 main.go:134] libmachine: (minikube) Calling .GetMachineName
I0628 09:59:52.195460    9441 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:59:52.232614    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.235486    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:52.235532    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.235990    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:52.272035    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.274782    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:52.274858    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.275054    9441 provision.go:138] copyHostCerts
I0628 09:59:52.275213    9441 exec_runner.go:144] found /home/****/.minikube/ca.pem, removing ...
I0628 09:59:52.275229    9441 exec_runner.go:207] rm: /home/****/.minikube/ca.pem
I0628 09:59:52.275340    9441 exec_runner.go:151] cp: /home/****/.minikube/certs/ca.pem --> /home/****/.minikube/ca.pem (1082 bytes)
I0628 09:59:52.275510    9441 exec_runner.go:144] found /home/****/.minikube/cert.pem, removing ...
I0628 09:59:52.275525    9441 exec_runner.go:207] rm: /home/****/.minikube/cert.pem
I0628 09:59:52.275617    9441 exec_runner.go:151] cp: /home/****/.minikube/certs/cert.pem --> /home/****/.minikube/cert.pem (1127 bytes)
I0628 09:59:52.275761    9441 exec_runner.go:144] found /home/****/.minikube/key.pem, removing ...
I0628 09:59:52.275773    9441 exec_runner.go:207] rm: /home/****/.minikube/key.pem
I0628 09:59:52.275871    9441 exec_runner.go:151] cp: /home/****/.minikube/certs/key.pem --> /home/****/.minikube/key.pem (1679 bytes)
I0628 09:59:52.275988    9441 provision.go:112] generating server cert: /home/****/.minikube/machines/server.pem ca-key=/home/****/.minikube/certs/ca.pem private-key=/home/****/.minikube/certs/ca-key.pem org=****.minikube san=[192.168.39.235 192.168.39.235 localhost 127.0.0.1 minikube minikube]
I0628 09:59:52.689224    9441 provision.go:172] copyRemoteCerts
I0628 09:59:52.689443    9441 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0628 09:59:52.689517    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:52.732242    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.734736    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:52.734803    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.735033    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:52.735553    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:52.735708    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:52.735857    9441 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:59:52.829355    9441 ssh_runner.go:362] scp /home/****/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0628 09:59:52.858570    9441 ssh_runner.go:362] scp /home/****/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0628 09:59:52.891715    9441 ssh_runner.go:362] scp /home/****/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0628 09:59:52.919419    9441 provision.go:86] duration metric: configureAuth took 724.263446ms
I0628 09:59:52.919445    9441 buildroot.go:189] setting minikube options for container-runtime
I0628 09:59:52.919696    9441 config.go:178] Loaded profile config "minikube": Driver=kvm2, ContainerRuntime=docker, KubernetesVersion=v1.24.1
I0628 09:59:52.919721    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:52.920080    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:52.956697    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.960495    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:52.960568    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:52.960720    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:52.961060    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:52.961253    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:52.961502    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:52.961703    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:52.961947    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:52.962034    9441 main.go:134] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0628 09:59:53.095799    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: tmpfs

I0628 09:59:53.096047    9441 buildroot.go:70] root file system type: tmpfs
I0628 09:59:53.096530    9441 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0628 09:59:53.096581    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:53.139898    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:53.143044    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:53.143083    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:53.143422    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:53.143832    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:53.144344    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:53.144527    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:53.144874    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:53.145065    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:53.145177    9441 main.go:134] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=kvm2 --insecure-registry 10.96.0.0/12 --insecure-registry 192.168.111.1:5000 
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0628 09:59:53.291760    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=kvm2 --insecure-registry 10.96.0.0/12 --insecure-registry 192.168.111.1:5000 
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

I0628 09:59:53.291811    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:53.330981    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:53.334116    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:53.334172    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:53.334645    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:53.335025    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:53.335331    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:53.335564    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:53.335809    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:53.336024    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:53.336071    9441 main.go:134] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0628 09:59:54.760509    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

I0628 09:59:54.760534    9441 machine.go:91] provisioned docker machine in 2.94452875s
I0628 09:59:54.760672    9441 start.go:306] post-start starting for "minikube" (driver="kvm2")
I0628 09:59:54.760717    9441 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0628 09:59:54.760767    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:54.761234    9441 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0628 09:59:54.761290    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:54.800757    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:54.803592    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:54.803660    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:54.804128    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:54.804565    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:54.804790    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:54.805082    9441 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:59:54.901543    9441 ssh_runner.go:195] Run: cat /etc/os-release
I0628 09:59:54.906458    9441 info.go:137] Remote host: Buildroot 2021.02.12
I0628 09:59:54.906483    9441 filesync.go:126] Scanning /home/****/.minikube/addons for local assets ...
I0628 09:59:54.906536    9441 filesync.go:126] Scanning /home/****/.minikube/files for local assets ...
I0628 09:59:54.906554    9441 start.go:309] post-start completed in 145.838476ms
I0628 09:59:54.906562    9441 fix.go:57] fixHost completed within 25.523611652s
I0628 09:59:54.906580    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:54.953666    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:54.956680    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:54.956713    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:54.957121    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:54.957476    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:54.957651    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:54.958029    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:54.958223    9441 main.go:134] libmachine: Using SSH client type: native
I0628 09:59:54.958423    9441 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil>  [] 0s} 192.168.39.235 22 <nil> <nil>}
I0628 09:59:54.958447    9441 main.go:134] libmachine: About to run SSH command:
date +%s.%N
I0628 09:59:55.094861    9441 main.go:134] libmachine: SSH cmd err, output: <nil>: 1656424795.078571565

I0628 09:59:55.094893    9441 fix.go:207] guest clock: 1656424795.078571565
I0628 09:59:55.094910    9441 fix.go:220] Guest: 2022-06-28 09:59:55.078571565 -0400 EDT Remote: 2022-06-28 09:59:54.906565333 -0400 EDT m=+25.928076207 (delta=172.006232ms)
I0628 09:59:55.094971    9441 fix.go:191] guest clock delta is within tolerance: 172.006232ms
I0628 09:59:55.094982    9441 start.go:81] releasing machines lock for "minikube", held for 25.71206368s
I0628 09:59:55.095053    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.095585    9441 main.go:134] libmachine: (minikube) Calling .GetIP
I0628 09:59:55.136093    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.139438    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:55.139474    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.139900    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.140257    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.140605    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.141601    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.142175    9441 main.go:134] libmachine: (minikube) Calling .DriverName
I0628 09:59:55.142383    9441 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0628 09:59:55.142503    9441 ssh_runner.go:195] Run: systemctl --version
I0628 09:59:55.142538    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:55.143108    9441 main.go:134] libmachine: (minikube) Calling .GetSSHHostname
I0628 09:59:55.198117    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.202261    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:55.202283    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.202757    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:55.202998    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:55.203282    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:55.203489    9441 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:59:55.224232    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.227386    9441 main.go:134] libmachine: (minikube) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:45:cf:30", ip: ""} in network mk-minikube: {Iface:virbr1 ExpiryTime:2022-06-28 10:59:42 -0400 EDT Type:0 Mac:52:54:00:45:cf:30 Iaid: IPaddr:192.168.39.235 Prefix:24 Hostname:minikube Clientid:01:52:54:00:45:cf:30}
I0628 09:59:55.227431    9441 main.go:134] libmachine: (minikube) DBG | domain minikube has defined IP address 192.168.39.235 and MAC address 52:54:00:45:cf:30 in network mk-minikube
I0628 09:59:55.227891    9441 main.go:134] libmachine: (minikube) Calling .GetSSHPort
I0628 09:59:55.228193    9441 main.go:134] libmachine: (minikube) Calling .GetSSHKeyPath
I0628 09:59:55.228562    9441 main.go:134] libmachine: (minikube) Calling .GetSSHUsername
I0628 09:59:55.228830    9441 sshutil.go:53] new ssh client: &{IP:192.168.39.235 Port:22 SSHKeyPath:/home/****/.minikube/machines/minikube/id_rsa Username:docker}
I0628 09:59:55.297952    9441 preload.go:132] Checking if preload exists for k8s version v1.24.1 and runtime docker
I0628 09:59:55.298432    9441 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0628 09:59:55.392712    9441 docker.go:602] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.24.1
k8s.gcr.io/kube-proxy:v1.24.1
k8s.gcr.io/kube-controller-manager:v1.24.1
k8s.gcr.io/kube-scheduler:v1.24.1
k8s.gcr.io/etcd:3.5.3-0
k8s.gcr.io/pause:3.7
k8s.gcr.io/coredns/coredns:v1.8.6
k8s.gcr.io/pause:3.6
gcr.io/k8s-minikube/storage-provisioner:v5

-- /stdout --
I0628 09:59:55.392771    9441 docker.go:533] Images already preloaded, skipping extraction
I0628 09:59:55.393044    9441 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0628 09:59:55.410365    9441 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0628 09:59:55.428979    9441 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0628 09:59:55.445479    9441 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
image-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0628 09:59:55.464855    9441 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0628 09:59:55.578360    9441 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0628 09:59:55.705130    9441 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0628 09:59:55.825239    9441 ssh_runner.go:195] Run: sudo systemctl restart docker
I0628 09:59:57.877427    9441 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.052095208s)
I0628 09:59:57.877567    9441 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0628 09:59:58.002217    9441 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0628 09:59:58.129122    9441 ssh_runner.go:195] Run: sudo systemctl start cri-docker.socket
I0628 09:59:58.143008    9441 start.go:447] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0628 09:59:58.143125    9441 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0628 09:59:58.148374    9441 start.go:468] Will wait 60s for crictl version
I0628 09:59:58.148427    9441 ssh_runner.go:195] Run: sudo crictl version
I0628 10:00:00.179268    9441 ssh_runner.go:235] Completed: sudo crictl version: (2.030804515s)
I0628 10:00:00.179353    9441 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-28T14:00:00Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0628 10:00:11.227440    9441 ssh_runner.go:195] Run: sudo crictl version
I0628 10:00:13.261424    9441 ssh_runner.go:235] Completed: sudo crictl version: (2.033847191s)
I0628 10:00:13.261551    9441 retry.go:31] will retry after 21.607636321s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-28T14:00:13Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0628 10:00:34.869638    9441 ssh_runner.go:195] Run: sudo crictl version
I0628 10:00:36.899453    9441 ssh_runner.go:235] Completed: sudo crictl version: (2.02953819s)
I0628 10:00:36.903836    9441 out.go:177] 

W0628 10:00:36.906903    9441 out.go:239] X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-28T14:00:36Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"

X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-28T14:00:36Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"

W0628 10:00:36.907161    9441 out.go:239] * 
* 

@kashifest
Copy link
Author

@spowelljr if using --driver=kvm2 is cri-dockerd still needed for minikube 1.26.0?

@kashifest
Copy link
Author

@spowelljr if using --driver=kvm2 is cri-dockerd still needed for minikube 1.26.0?

Even if it is, shouldn't I have seen the error during the initial start ?

@spowelljr
Copy link
Member

Hi @kashifest, thanks for reporting your issue with minikube.

If you don't provide a --container-runtime flag it will default to using cri-docker, but because you're using the kvm2 driver you shouldn't need to do anything to on your end to set it up.

If I have time today I'll try replicating this and see if I run into the same issue.

@spowelljr spowelljr added kind/support Categorizes issue or PR as a support question. co/kvm2-driver KVM2 driver related issues labels Jun 28, 2022
@spowelljr
Copy link
Member

So I tried this on my own machine and didn't experience what you are getting, mine restarted correctly as I would expect.

This is an odd error as it seems like crictl is no longer present in the VM.

Also, I see you're running sudo su -l, normally running minikube as root results in the following message:
🛑 The "kvm2" driver should not be used with root privileges. If you wish to continue as root, use --force.
Just curious if you're suppressing the message somehow or sudo su -l isn't running as root.

@kashifest
Copy link
Author

kashifest commented Jun 30, 2022

Also, I see you're running sudo su -l, normally running minikube as root results in the following message: 🛑 The "kvm2" driver should not be used with root privileges. If you wish to continue as root, use --force. Just curious if you're suppressing the message somehow or sudo su -l isn't running as root.

@spowelljr thanks for testing it. I guess sudo is not the issue here. I noticed that its just taking more time to get the crictl starting than the initial 3 times try. I got a workaround by adding a loop like this:

while /bin/true; do
        minikube_error=0
        sudo su -l -c 'minikube start --driver=kvm2 --alsologtostderr' "${USER}" || minikube_error=1
        if [[ $minikube_error -eq 0 ]]; then
          break
        fi
    done

I will give a small gist of the times it tried:

I0630 07:07:26.841904    9431 ssh_runner.go:195] Run: sudo crictl version
I0630 07:07:28.878086    9431 ssh_runner.go:235] Completed: sudo crictl version: (2.036152265s)
I0630 07:07:28.878136    9431 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:07:28Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:07:39.925068    9431 ssh_runner.go:195] Run: sudo crictl version
I0630 07:07:41.951446    9431 ssh_runner.go:235] Completed: sudo crictl version: (2.026330594s)
I0630 07:07:41.951630    9431 retry.go:31] will retry after 21.607636321s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:07:41Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:08:03.560565    9431 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:05.588786    9431 ssh_runner.go:235] Completed: sudo crictl version: (2.028148952s)
I0630 07:08:05.593093    9431 out.go:177] 

W0630 07:08:05.596148    9431 out.go:239] X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:I0630 07:08:54.193923   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:56.217061   10029 ssh_runner.go:235] Completed: sudo crictl version: (2.02311292s)
I0630 07:08:56.217110   10029 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:08:56Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:09:07.265727   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:09:09.295327   10029 ssh_runner.go:235] Completed: sudo crictl version: (2.02955302s)
I0630 07:09:09.295369   10029 retry.go:31] will retry after 21.607636321s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:09:09Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:09:30.904086   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:09:30.942921   10029 start.go:477] Version:  0.1.0
RuntimeName:  docker
RuntimeVersion:  20.10.16
RuntimeApiVersion:  1.41.0
time="2022-06-30T11:08:05Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"

X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1

I0630 07:08:10.568227    9895 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:12.592623    9895 ssh_runner.go:235] Completed: sudo crictl version: (2.024358437s)
I0630 07:08:12.592683    9895 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:08:12Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:08:23.640272    9895 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:25.674429    9895 ssh_runner.go:235] Completed: sudo crictl version: (2.034091685s)
I0630 07:08:25.674475    9895 retry.go:31] will retry after 21.607636321s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:08:25Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:08:47.282527    9895 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:49.315123    9895 ssh_runner.go:235] Completed: sudo crictl version: (2.032482128s)
I0630 07:08:49.319386    9895 out.go:177] 

W0630 07:08:49.341554    9895 out.go:239] X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:08:49Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"

X Exiting due to RUNTIME_ENABLE: Temporary Error: sudo crictl version: Process exited with status 1


I0630 07:08:54.193923   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:08:56.217061   10029 ssh_runner.go:235] Completed: sudo crictl version: (2.02311292s)
I0630 07:08:56.217110   10029 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:08:56Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:09:07.265727   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:09:09.295327   10029 ssh_runner.go:235] Completed: sudo crictl version: (2.02955302s)
I0630 07:09:09.295369   10029 retry.go:31] will retry after 21.607636321s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:

stderr:
time="2022-06-30T11:09:09Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/cri-dockerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
I0630 07:09:30.904086   10029 ssh_runner.go:195] Run: sudo crictl version
I0630 07:09:30.942921   10029 start.go:477] Version:  0.1.0
RuntimeName:  docker
RuntimeVersion:  20.10.16
RuntimeApiVersion:  1.41.0

So basically instead of trying 3 times only it needed to try 9 times. I believe trying 3 times only and/or the Will wait 60s for crictl version is too short for some environments.

@spowelljr
Copy link
Member

Thanks for figuring this out @kashifest!

I will say that 60 seconds seems like a long time for the container runtime to come up, but this is the first case of this I've seen reported yet, so maybe like you said it's isolated to certain environments.

@spowelljr spowelljr added the long-term-support Long-term support issues that can't be fixed in code label Aug 10, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 8, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 8, 2022
@kashifest
Copy link
Author

@spowelljr I am still seeing this issue and probably with more delay needed with newer versions, is it possible to configure the timeout?

kashifest added a commit to Nordix/metal3-dev-env that referenced this issue Dec 22, 2022
… cri to comeup which exceeds default hardcoded timeout 60 seconds. Hence we need to try in loop. Upstream issue is kubernetes/minikube#14456
furkatgofurov7 pushed a commit to Nordix/metal3-dev-env that referenced this issue Dec 23, 2022
… cri to comeup which exceeds default hardcoded timeout 60 seconds. Hence we need to try in loop. Upstream issue is kubernetes/minikube#14456
kashifest added a commit to Nordix/metal3-dev-env that referenced this issue Jan 2, 2023
… cri to comeup which exceeds default hardcoded timeout 60 seconds. Hence we need to try in loop. Upstream issue is kubernetes/minikube#14456
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

@k8s-ci-robot k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale Jan 21, 2023
@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kashifest
Copy link
Author

/reopen
/remove-lifecycle rotten
We are still seeing this issue even with docker service

@k8s-ci-robot k8s-ci-robot reopened this Oct 4, 2024
@k8s-ci-robot
Copy link
Contributor

@kashifest: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten
We are still seeing this issue even with docker service

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Oct 4, 2024
kashifest added a commit to Nordix/baremetal-operator that referenced this issue Oct 4, 2024
This is a workaround for a long existing issue in minikube which we opened long ago but it seems it has not gone away. Here is the issue for more detail kubernetes/minikube#14456

Signed-off-by: Kashif Khan <kashif.khan@est.tech>
kashifest added a commit to Nordix/baremetal-operator that referenced this issue Oct 4, 2024
This is a workaround for a long existing issue in minikube which we opened long ago but it seems it has not gone away. Here is the issue for more detail kubernetes/minikube#14456

Signed-off-by: Kashif Khan <kashif.khan@est.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/kvm2-driver KVM2 driver related issues kind/support Categorizes issue or PR as a support question. long-term-support Long-term support issues that can't be fixed in code
Projects
None yet
Development

No branches or pull requests

4 participants