kvm2: Machine didn't return an IP after 120 seconds #3566
Comments
cmdpwnd
changed the title
tstromberg
changed the title
tstromberg
changed the title
tstromberg
added
os/linux
co/kvm2-driver
|
I'm pretty confident this is a kvm/libvirt issue that we should be able to detect, but don't know how to yet. Probably also related to the use of nested VM's. Have you tried running minikube outside of VMware workstation? https://fedoraproject.org/wiki/How_to_debug_Virtualization_problems has some guidance on debugging kvm/libvirt issues, but I am especially curious what this command emits:
along with:
|
tstromberg
added
cause/nested-vm-config
|
Running nested virtualization causes, in general, more headache than needed.
For minishift we had several of these reports and most of them failed
to work properly.
…On Wed, Jan 23, 2019 at 12:48 PM Thomas Strömberg ***@***.***> wrote:
I'm pretty confident this is a kvm/libvirt issue that we should be able to detect, but don't know how to yet. Probably also related to the use of nested VM's. Have you tried running minikube outside of VMware workstation?
https://fedoraproject.org/wiki/How_to_debug_Virtualization_problems has some guidance on debugging kvm/libvirt issues, but I am especially curious what this command emits:
virt-host-validate
along with:
`virsh net-list --all
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
--
Gerard Braad | http://gbraad.nl
[ Doing Open Source Matters ]
|
|
@tstromberg I'd read through those related minishift issues before opening an issue here... I've replicated my scenario with stretch installed on hardware using the info defined in this issue, and have already ruled out For additional info though: (same output regardless of virtualization on Intel) I'm not worried about |
cmdpwnd
changed the title
|
@cmdpwnd - thanks for the update! Do you mind running a few commands to help narrow this issue down a bit further? I use kvm2 on Debian every day, but I suspect we have some environmental differences. First, let's get the virsh version:
We can roughly emulate the path the kvm driver uses to determine the IP address by first finding the name of the bridge for
From there, the kvm driver (even libvirt upstream!) parses dnsmasq status (?!?!?) to get the IP address of the interface from the bridge name we just discovered:
It seems like there should be a more straightforward way to do this with more recent releases of libvirt, since virsh has no problem with displaying the IP address here:
// my output: If you don't mind repeating the same commands, I think I can figure out how to improve the kvm driver to do the correct thing here. |
|
@tstromberg alright, I think we're getting somewhere now 👍 . My networking is totally dead. Additionally, here's the config that |
|
Hi, i have the same problem. This is my output @cmdpwnd Do you resolved this problem? If, yes, can you tell how? |
|
My logs for command:
|
|
I'm running into this same issue on Ubuntu 18.10 I'm gonna dump all my information to compare against what everyone else is experiencing. Environment: Minikube Version: 0.34.1 VM Driver: kvm2 virt-host-validate: virsh net-list --all: virsh --version: 4.6.0 virsh --connect qemu:///system dumpxml minikube | grep minikube-net: grep ip-address /var/lib/libvirt/dnsmasq/virbr1.status: 192.168.39.230 Hopefully this adds some more datapoints to help us figure out what's going on here |
|
To add another datapoint, I was running into the same issue (how I got here), but in the process of reproducing, it magically fixed itself (the worst kind of fix!). Environment: Minikube version: OS: VM Driver: Others: Initial failure was exactly the same as @cmdpwnd 's. Additional info: So how did things magically start working? After a fresh reboot of the VM, |
|
another reproducing here, checked all above [...]
(minikube) DBG | Waiting for machine to come up 19/40
(minikube) DBG | Waiting for machine to come up 20/40
(minikube) DBG | Waiting for machine to come up 21/40
(minikube) DBG | Waiting for machine to come up 22/40
(minikube) DBG | Waiting for machine to come up 23/40
(minikube) DBG | Waiting for machine to come up 24/40
(minikube) DBG | Waiting for machine to come up 25/40
(minikube) DBG | Waiting for machine to come up 26/40
(minikube) DBG | Waiting for machine to come up 27/40
(minikube) DBG | Waiting for machine to come up 28/40
(minikube) DBG | Waiting for machine to come up 29/40
(minikube) DBG | Waiting for machine to come up 30/40
(minikube) DBG | Waiting for machine to come up 31/40
(minikube) DBG | Waiting for machine to come up 32/40
(minikube) DBG | Waiting for machine to come up 33/40
(minikube) DBG | Waiting for machine to come up 34/40
(minikube) DBG | Waiting for machine to come up 35/40
(minikube) DBG | Waiting for machine to come up 36/40
(minikube) DBG | Waiting for machine to come up 37/40
(minikube) DBG | Waiting for machine to come up 38/40
(minikube) DBG | Waiting for machine to come up 39/40
(minikube) DBG | Waiting for machine to come up 40/40
I0424 17:06:53.580893 28159 start.go:384] StartHost: create: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds
I0424 17:06:53.580971 28159 utils.go:122] non-retriable error: create: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds
W0424 17:06:53.581310 28159 exit.go:99] Unable to start VM: create: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds
💣 Unable to start VM: create: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds
😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉 https://github.com/kubernetes/minikube/issues/new
Command exited with non-zero status 70
0.12user 0.02system 2:07.16elapsed 0%CPU (0avgtext+0avgdata 29468maxresident)k
16inputs+16outputs (0major+2903minor)pagefaults 0swaps
real 2m7.165s
user 0m0.128s
sys 0m0.031s
to @tstromberg I don't mind to run more debugging commands, while do you (or any other developer ) have an update on this? |
tstromberg
changed the title
|
If you run into this, please try upgrading to the most recent kvm2 machine driver and report back:
Then run |
tstromberg
added
r/2019q2
tstromberg
added
cause/firewall-or-proxy
sharifelgamal
added
the
help wanted
|
Also having this issue as described with v1.2.0 of the driver. |
|
I was having the same issue (even with kvm driver v1.2.0). Turns out that for me too conflicting Now i just have to figure out what rules to add to |
|
I found out what was causing problems in my config. The rules in my to fix that. Here's the full set of rules in my I also made sure to use |
|
It's seem issue on vmware nested virtualization,, I got it working on Vmware player 15 on Windows 10 1903 and with this setting in vmx: vhv.enable = "TRUE"
vpmc.enable = "TRUE"
vvtd.enable = "TRUE"But if I also test the minikube iso on libvirt using this command, and showing the same behaviour : virt-install --virt-type=kvm --name=test --ram 2048 --vcpus=1 --virt-type=kvm --hvm --cdrom ~/.minikube/cache/iso/minikube-v1.3.0.iso --network network=default --disk pool=default,size=20,bus=virtio,format=qcow2 Reference: |
tstromberg
added
kind/support
|
minikube v1.4 now gives a little bit more documentation around this, but I'll leave this open for others who run into this support issue. |
tstromberg
added
the
lifecycle/frozen
|
@cmdpwnd I will close this issue due to no update, and sorry to hear you had this issue, please feel free to reopen. meanwhile I recommend to give our newest driver a try in latest release minikube start --vm-driver=docker |
|
JFYI I was able to workaround this just restarting libvirtd service. Without changing anything else minukube started perfectly. |
|
I am also facing this issue and the error persists despite trying out the suggested fixes :( Posting this here in case there are other folks in my situation! My error and logs are pretty much identical to that of the earlier users who posted their logs. One thing that may be of interest is that DHCP appears empty immediately after the A few seconds later, an address is provided: I tried increasing the time for the retry (that of I went with QEMU as a driver instead, which works without any issue. |
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUGPlease provide the following details:
Environment:
VMware Workstation 11Minikube version:
minikube version: v0.33.1"DriverName": "kvm2",N/AWhat happened:
E0121 11:39:38.385150 1862 start.go:205] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds.What you expected to happen:
Success??
How to reproduce it:
Copy/Paste will doAnything else we need to know?:
If you don't clear minikube after initial failure, on rerun expect:The text was updated successfully, but these errors were encountered: