Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xhyve driver not working anymore on MacOS 10.12 sierra #646

Closed
puja108 opened this issue Sep 30, 2016 · 38 comments
Closed

xhyve driver not working anymore on MacOS 10.12 sierra #646

puja108 opened this issue Sep 30, 2016 · 38 comments
Labels
co/xhyve kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. os/macos

Comments

@puja108
Copy link
Member

puja108 commented Sep 30, 2016

Minikube version (use minikube version): 0.10.0

Environment:

  • OS (e.g. from /etc/os-release): MacOS 10.12 Sierra
  • VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): xhyve
  • Docker version (e.g. docker -v): Docker version 1.12.2-rc1, build 45bed2c, experimental (because of Sierra, but also tried previous and stable version, all Docker for Mac)
  • Install tools:
  • Others:

What happened:
minikube start --vm-driver=xhyve Starting local Kubernetes cluster... E0930 11:28:13.241885 36469 start.go:85] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting. Retrying.

What you expected to happen:
Starting local Kubernetes cluster... Running pre-create checks... Creating machine... Starting local Kubernetes cluster... or similar

How to reproduce it (as minimally and precisely as possible):
Run minikube start --vm-driver=xhyve on an environment similar to the one above.

Anything else do we need to know:
The vbox driver works, so it must be something with the xhyve or docker-machine. Also looking at the error.
Installed versions are:
docker-machine version 0.8.2, build e18a919
docker-machine-driver-xhyve-0.2.3
The latter hasn't been updated since August.

@r2d4 r2d4 added kind/bug Categorizes issue or PR as related to a bug. os/macos co/xhyve labels Sep 30, 2016
@matejkramny
Copy link

I just tried with xhyve and seems to be working for me..

Im sure its provisioning at the moment, so i think its past your stage.

$ minikube start --logtostderr --show-libmachine-logs --vm-driver=xhyve
W1001 11:55:46.197896   71588 root.go:127] Error reading config file at /Users/m/.minikube/config/config.json: open /Users/m/.minikube/config/config.json: no such file or directory
I1001 11:55:46.198040   71588 notify.go:103] Checking for updates...
Starting local Kubernetes cluster...
Running pre-create checks...
Creating machine...
(minikube) Downloading /Users/m/.minikube/cache/boot2docker.iso from file:///Users/m/.minikube/cache/iso/minikube-0.7.iso...
(minikube) Creating VM...
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) /dev/disk2                                           /Users/m/.minikube/machines/minikube/b2d-image
(minikube) "disk2" unmounted.
(minikube) "disk2" ejected.
(minikube) Generating 20000MB disk image...
(minikube) created: /Users/m/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) Generate UUID...
(minikube) Convert UUID to MAC address...
(minikube) Starting minikube...
(minikube) Waiting for VM to come online...
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys007 in order to connect to your VM
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!

@annismckenzie
Copy link

I can confirm this – had the same issue and it's the docker-machine-driver-xhyve that crashes. This is due to incompatibilities introduced with macOS Sierra and the plugin binary you have doesn't work on Sierra anymore. The repo is here: https://github.com/zchee/docker-machine-driver-xhyve. You can't go the Homebrew route because that will download the incompatible binary again. The only thing you can do is get your local Go environment up-to-date so that you can recompile the plugin from source.

Or I could upload the fixed plugin binary somewhere. Alas, that would mean you trusting me.

@shushen
Copy link

shushen commented Oct 3, 2016

I don't see this problem and can run minikube successfully with everything installed by homebrew:

  • macOS 10.12 (16A323)
  • Homebrew 1.0.5
  • xhyve: stable 0.2.0
  • docker-machine: stable 0.8.2
  • docker-machine-driver-xhyve: stable 0.2.3

Installation procedure for xhyve/docker-machine/docker-machine-driver-xhyve:

$ brew update
$ brew install --HEAD xhyve
$ brew install docker-machine-driver-xhyve
$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

Maybe it's an issue with docker-machine-driver-xhyve when installed with GO?
GO in my environment is also installed via homebrew:

  • go version go1.7.1 darwin/amd64

@puja108
Copy link
Member Author

puja108 commented Oct 4, 2016

@annismckenzie thanks, that makes sense, will try recompiling the plugin from source.

@annismckenzie
Copy link

Holler if you need help. :)

Sent from my iPhone SE

On Oct 4, 2016, at 10:00 AM, Puja notifications@github.com wrote:

@annismckenzie thanks, that makes sense, will try recompiling the plugin from source.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@puja108
Copy link
Member Author

puja108 commented Oct 4, 2016

@shushen I see, hmm, will try again, maybe something is borked in my system, as mine is also installed with brew. Maybe it's the local go version, that one seems to be outdated, will try installing go with homebrew.

@puja108
Copy link
Member Author

puja108 commented Oct 4, 2016

So after seeing that my local go version was set to 1.4.2 by Sierra I installed a recent go 1.7.1 with homebrew. After a reboot suddenly things are working. So there's no real need for a recompile, you just need a recent go version running. Thanks @shushen for the hint.

@mboersma
Copy link

None of the above workarounds (build plugin from source, ensure Go 1.7+) seem to help us. Still seeing this consistently:

$ minikube start --cpus=2 --disk-size=40g --memory=4096 --logtostderr --show-libmachine-logs --vm-driver=xhyve
W1017 12:53:45.062066    7668 root.go:127] Error reading config file at /Users/matt/.minikube/config/config.json: open /Users/matt/.minikube/config/config.json: no such file or directory
I1017 12:53:45.062272    7668 notify.go:111] Checking for updates...
Starting local Kubernetes cluster...
Running pre-create checks...
Creating machine...
(minikube) Downloading /Users/matt/.minikube/cache/boot2docker.iso from file:///Users/matt/.minikube/cache/iso/minikube-0.7.iso...
(minikube) Creating VM...
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) /dev/disk2                                           /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) "disk2" unmounted.
(minikube) "disk2" ejected.
(minikube) Generating 40000MB disk image...
(minikube) created: /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) Using Supplied UUID: 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847
(minikube) Convert UUID to MAC address...
(minikube) Starting minikube...
(minikube) Waiting for VM to come online...
E1017 12:56:07.155903    7668 start.go:87] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting. Retrying.
I1017 12:56:07.155991    7668 cluster.go:75] Machine exists!
I1017 12:56:07.180160    7668 cluster.go:82] Machine state:  Error
(minikube) Waiting for VM to come online...
E1017 12:58:08.581202    7668 start.go:87] Error starting host: Error starting stopped host: Machine didn't return an IP after 120 seconds, aborting. Retrying.
I1017 12:58:08.581262    7668 cluster.go:75] Machine exists!
I1017 12:58:08.599697    7668 cluster.go:82] Machine state:  Error
(minikube) Waiting for VM to come online...
E1017 13:00:10.014103    7668 start.go:87] Error starting host: Error starting stopped host: Machine didn't return an IP after 120 seconds, aborting. Retrying.
E1017 13:00:10.014177    7668 start.go:93] Error starting host:  Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting
Error starting stopped host: Machine didn't return an IP after 120 seconds, aborting
Error starting stopped host: Machine didn't return an IP after 120 seconds, aborting
========================================An error has occurred.  Would you like to opt in to sending anonymized crash information to minikube to help prevent future errors?(To opt out of these messages, run the command)
    minikube config set WantReportErrorPrompt false========================================
Please enter your response [Y/n]: 

Virtualbox works, but is noticeably slower than xhyve for this purpose.

Would really love to help troubleshoot or fix this, just not sure where to dig in.

@dlorenc
Copy link
Contributor

dlorenc commented Oct 17, 2016

Ah, just to confirm have you tried building the docker-machine-driver-xhyve binary from source?

@zchee what version of Go did you use to build this?

@mboersma
Copy link

mboersma commented Oct 17, 2016

have you tried building the docker-machine-driver-xhyve binary from source?

Yes, but I realize I hadn't used HEAD of xhyve itself as recommended above. So I did that and used docker-machine-driver-xhyve from brew. That fails similarly:

I1017 13:05:21.995939   10322 notify.go:111] Checking for updates...
Starting local Kubernetes cluster...
Running pre-create checks...
Creating machine...
(minikube) Downloading /Users/matt/.minikube/cache/boot2docker.iso from file:///Users/matt/.minikube/cache/iso/minikube-0.7.iso...
(minikube) Creating VM...
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) /dev/disk3                                           /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) "disk3" unmounted.
(minikube) "disk3" ejected.
(minikube) Generating 40000MB disk image...
(minikube) created: /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) Generate UUID...
(minikube) Convert UUID to MAC address...
(minikube) Starting minikube...
(minikube) Waiting for VM to come online...
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys000 in order to connect to your VM
E1017 13:07:31.309276   10322 start.go:87] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting. Retrying.
I1017 13:07:31.309382   10322 cluster.go:75] Machine exists!
I1017 13:07:31.335096   10322 cluster.go:82] Machine state:  Running
Waiting for SSH to be available...
[timeouts]

I'll try also building docker-machine-driver-xhyve from source now, in conjunction with HEAD of xhyve.

Update: no joy, still times out. Weird...

@dlorenc
Copy link
Contributor

dlorenc commented Oct 17, 2016

Ah, yeah. docker-machine-driver-xhyve fully bundles it's own version of xhyve. It's basically standalone.

@r2d4
Copy link
Contributor

r2d4 commented Oct 17, 2016

@mboersma try adding --v=5 --show-libmachine-logs to your start command. That will print out debug level logs from the driver. That might help figure out the issue

@mboersma
Copy link

$ minikube start --v=5 --logtostderr --show-libmachine-logs --vm-driver=xhyve
I1017 14:45:26.177041   17082 notify.go:111] Checking for updates...
Starting local Kubernetes cluster...
Found binary path at /Users/matt/Projects/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:57383
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) DBG | ===== Docker Machine xhyve Driver Version 0.2.3 (HEAD) =====
(minikube) DBG | 
(minikube) DBG | executing: /usr/local/bin/VBoxManage -v
(minikube) DBG | STDOUT: 5.1.6r110634
(minikube) DBG | 
(minikube) DBG | STDERR: 
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Downloading /Users/matt/.minikube/cache/boot2docker.iso from file:///Users/matt/.minikube/cache/iso/minikube-0.7.iso...
(minikube) Creating VM...
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) DBG | Mounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil attach /Users/matt/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/matt/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} attach /Users/matt/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) /dev/disk8                                           /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) DBG | Extracting vmlinuz64 into /Users/matt/.minikube/machines/minikube
(minikube) DBG | Extracting initrd.img into /Users/matt/.minikube/machines/minikube
(minikube) DBG | Unmounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil detach /Users/matt/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} detach /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) "disk8" unmounted.
(minikube) "disk8" ejected.
(minikube) Generating 20000MB disk image...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil create -megabytes 20000 -type SPARSEBUNDLE /Users/matt/.minikube/machines/minikube/root-volume] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} create -megabytes 20000 -type SPARSEBUNDLE /Users/matt/.minikube/machines/minikube/root-volume
(minikube) created: /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) DBG | /Users/matt/.minikube/machines/minikube/boot2docker.iso
(minikube) DBG | /Users/matt/.minikube/machines/minikube/config.json
(minikube) DBG | /Users/matt/.minikube/machines/minikube/id_rsa
(minikube) DBG | /Users/matt/.minikube/machines/minikube/id_rsa.pub
(minikube) DBG | /Users/matt/.minikube/machines/minikube/initrd.img
(minikube) DBG | /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) DBG | /Users/matt/.minikube/machines/minikube/vmlinuz64
(minikube) Using Supplied UUID: 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847
(minikube) Convert UUID to MAC address...
(minikube) DBG | Running command: /Users/matt/Projects/bin/docker-machine-driver-xhyve [xhyve -A -U 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk8 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -M]
(minikube) DBG | Converted MAC address: a6:4a:e0:e:cb:64
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk8 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -F /Users/matt/.minikube/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG | 
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) DBG | pci slot 5:0: unknown device "virtio-9p"
(minikube) DBG | exit status 1 &{0xc4200101e0} &{0xc420010200}
(minikube) DBG | 
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | 
... [continues]

@r2d4
Copy link
Contributor

r2d4 commented Oct 17, 2016

(minikube) DBG | pci slot 5:0: unknown device "virtio-9p"

@mboersma We enable the 9p filesystem by default with the xhyve driver. This is responsible for mounting your /Users folder in the VM. Its bundled in docker-machine-driver-xhyve, but it might not be working correctly in sierra.

Can you try rebuilding minikube with this set to false?
https://github.com/kubernetes/minikube/blob/master/pkg/minikube/cluster/cluster_darwin.go#L69

Additionally, increasing loglevel in the bootcmd will show even more logs from the xhyve driver
https://github.com/kubernetes/minikube/blob/master/pkg/minikube/cluster/cluster_darwin.go#L67

[Maybe we should pass our --v level into this loglevel, however it might be too much info for most issues debugging]

@zchee
Copy link
Contributor

zchee commented Oct 17, 2016

@dlorenc @mboersma @r2d4
Sorry for the delay. I debugged (and fixed) hyperkit...
but figured out from @mboersma logs.

short reply and probably (because browsing iPhone...)

  • macOS Sierra is not related
    • gettimeofday ABI is related, I mean for @mboersma last log issue
  • that binary has not my patched pci_virtio9p.c
  • IP not found issue caused by use fixed(hardcoded) UUID(Edit: misunderstood)

I will more detail comment later.

@puja108
Copy link
Member Author

puja108 commented Oct 18, 2016

btw, my success with that go update was only temporary. I guess I'll just wait for @zchee to make the next release.

@zchee
Copy link
Contributor

zchee commented Oct 18, 2016

@puja108 Sorry 🙇


@mboersma
As your posted error log,

Docker Machine xhyve Driver Version 0.2.3 (HEAD)

It seems its binary is not installed by the

brew install docker-machine-driver-xhyve --HEAD

Maybe you installed by go get. (or, you have two docker-machine-driver-xhyve binary)

The short answer:
I'm sorry to take up your time, retry

go get -u -d github.com/zchee/docker-machine-driver-xhyve
cd $GOPATH/src/github.com/zchee/docker-machine-driver-xhyve

# just in case
git submodule update --init

# build and install 
make install
# or if want verbose
make install V=1

I was added small trick to Version variable.

  1. If use go get, commit version uses to
    https://github.com/zchee/docker-machine-driver-xhyve/blob/master/xhyve/version.go#L8
  2. If use brew ... --HEAD, uses to
    https://github.com/Homebrew/homebrew-core/blob/master/Formula/docker-machine-driver-xhyve.rb#L38
    actually HomebrewHEAD-8b542eb(current latest).
  3. If use make, HEAD will change to commit hash.
    https://github.com/zchee/docker-machine-driver-xhyve/blob/master/Makefile#L81

So, HEAD is maybe go get.

And now, my driver has been not supported go get install if want to use 9p folder share. Needs make.
README.md has not yet been updated. It was my fault. very sorry...

And, _why does not support go get?_.
The short answer is go get does not support build submodules. In this case, lib9p.
e.g. Not support automatically run the go generate command.(maybe for the security issue)

Also, If not add -tags lib9p for go command , cgo did not built the pci_virtio_9p.c.
https://github.com/zchee/docker-machine-driver-xhyve/blob/master/vendor/github.com/zchee/libhyperkit/pci_virtio_9p.c#L1

So, I think you're binary not static linked lib9p.a object file. or not added -tags lib9p.
Therefore occur

pci slot 5:0: unknown device "virtio-9p"

Please try

nm $(which docker-machine-driver-xhyve) | grep _l9p

If anything does not appear, it is not linked.

@mboersma
Copy link

@zchee thank you so much for the help! I followed your build instructions above but things still aren't working:

$ nm $(which docker-machine-driver-xhyve) | grep _l9p
000000000438b580 T _l9p_backend_fs_init
00000000043890f0 T _l9p_connection_alloc_fid
0000000004389010 T _l9p_connection_close
0000000004388dd0 T _l9p_connection_free
0000000004388c70 T _l9p_connection_init
0000000004388e70 T _l9p_connection_on_get_response_buffer
0000000004388e40 T _l9p_connection_on_send_response
0000000004388ea0 T _l9p_connection_recv
0000000004389190 T _l9p_connection_remove_fid
000000000438aaf0 T _l9p_describe_fcall
000000000438a960 T _l9p_describe_qid
000000000438aa20 T _l9p_describe_stat
00000000043891f0 T _l9p_dispatch_request
0000000004388680 T _l9p_freefcall
00000000043887f0 T _l9p_freestat
0000000004389520 T _l9p_pack_stat
0000000004387eb0 T _l9p_pufcall
0000000004387950 T _l9p_pustat
0000000004389290 T _l9p_respond
000000000438a700 T _l9p_seek_iov
0000000004388b90 T _l9p_server_init
0000000004387b40 T _l9p_sizeof_stat
000000000438a870 T _l9p_truncate_iov
$ minikube start --v=5 --logtostderr --show-libmachine-logs --vm-driver=xhyve
W1018 10:59:23.674434     828 root.go:127] Error reading config file at /Users/matt/.minikube/config/config.json: open /Users/matt/.minikube/config/config.json: no such file or directory
I1018 10:59:23.675267     828 notify.go:111] Checking for updates...
Starting local Kubernetes cluster...
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:49646
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Creating CA: /Users/matt/.minikube/certs/ca.pem
Creating client certificate: /Users/matt/.minikube/certs/cert.pem
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) DBG | ===== Docker Machine xhyve Driver Version 0.2.3 (8b542eb) =====
(minikube) DBG | 
(minikube) DBG | executing: /usr/local/bin/VBoxManage -v
(minikube) DBG | STDOUT: 5.1.6r110634
(minikube) DBG | 
(minikube) DBG | STDERR: 
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Downloading /Users/matt/.minikube/cache/boot2docker.iso from file:///Users/matt/.minikube/cache/iso/minikube-0.7.iso...
(minikube) Creating VM...
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) DBG | Mounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil attach /Users/matt/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/matt/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} attach /Users/matt/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) /dev/disk2                                           /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) DBG | Extracting vmlinuz64 into /Users/matt/.minikube/machines/minikube
(minikube) DBG | Extracting initrd.img into /Users/matt/.minikube/machines/minikube
(minikube) DBG | Unmounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil detach /Users/matt/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} detach /Users/matt/.minikube/machines/minikube/b2d-image
(minikube) "disk2" unmounted.
(minikube) "disk2" ejected.
(minikube) Generating 20000MB disk image...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil create -megabytes 20000 -type SPARSEBUNDLE /Users/matt/.minikube/machines/minikube/root-volume] []  <nil> 0xc42002a010 0xc42002a018 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} create -megabytes 20000 -type SPARSEBUNDLE /Users/matt/.minikube/machines/minikube/root-volume
(minikube) created: /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) DBG | /Users/matt/.minikube/machines/minikube/boot2docker.iso
(minikube) DBG | /Users/matt/.minikube/machines/minikube/config.json
(minikube) DBG | /Users/matt/.minikube/machines/minikube/id_rsa
(minikube) DBG | /Users/matt/.minikube/machines/minikube/id_rsa.pub
(minikube) DBG | /Users/matt/.minikube/machines/minikube/initrd.img
(minikube) DBG | /Users/matt/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) DBG | /Users/matt/.minikube/machines/minikube/vmlinuz64
(minikube) Using Supplied UUID: 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847
(minikube) Convert UUID to MAC address...
(minikube) DBG | Running command: /usr/local/bin/docker-machine-driver-xhyve [xhyve -A -U 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk2 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -M]
(minikube) DBG | Converted MAC address: a6:4a:e0:e:cb:64
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 57FD2012-FA4A-4FF7-AEFF-26E1A1D76847 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk2 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -F /Users/matt/.minikube/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG | 
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys001 in order to connect to your VM
(minikube) DBG | 
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 4/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | rdmsr to register 0x34 on vcpu 0
(minikube) DBG | 
(minikube) DBG | Not there yet 5/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
(minikube) DBG | 
...
(minikube) DBG | Not there yet 60/60, error: IP not found for MAC a6:4a:e0:e:cb:64 in DHCP leases
E1018 11:01:40.520364     828 start.go:87] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting. Retrying.
I1018 11:01:40.520480     828 cluster.go:75] Machine exists!
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:49677
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetState
I1018 11:01:40.540580     828 cluster.go:82] Machine state:  Running
Waiting for SSH to be available...
Getting to WaitForSSH function...
(minikube) Calling .GetSSHHostname
(minikube) DBG | 
Error getting ssh command 'exit 0' : IP not found for MAC  in DHCP leases
Getting to WaitForSSH function...
(minikube) Calling .GetSSHHostname
(minikube) DBG | 
... [times out]

Let me know if there are other options I should try. (For now, I can use virtualbox, so this isn't urgent.)

@zchee
Copy link
Contributor

zchee commented Oct 18, 2016

@mboersma Okay, so it seems solved

pci slot 5:0: unknown device "virtio-9p"

Next, it seems to not found IP from /var/db/dhcpd_leases.
I think caused by hardcoded UUID (I don't know yet exactly)
https://github.com/kubernetes/minikube/blob/master/pkg/minikube/cluster/cluster_darwin.go#L26

Rebuilding minikube is sometimes slow if needs build localkube, but it's inevitable.
So, workaround is ignored in xhyve side.

diff --git a/xhyve/xhyve.go b/xhyve/xhyve.go
index 4a98fe3..4d03cd0 100644
--- a/xhyve/xhyve.go
+++ b/xhyve/xhyve.go
@@ -402,13 +402,9 @@ func (d *Driver) Create() error {
        os.Chown(d.ResolveStorePath(f.Name()), syscall.Getuid(), syscall.Getegid())
    }

-   if d.UUID == "" {
-       log.Infof("Generate UUID...")
-       d.UUID = uuidgen()
-       log.Debugf("Generated UUID: %s", d.UUID)
-   } else {
-       log.Infof("Using Supplied UUID: %s", d.UUID)
-   }
+   log.Infof("Generate UUID...")
+   d.UUID = uuidgen()
+   log.Debugf("Generated UUID: %s", d.UUID)

    log.Infof("Convert UUID to MAC address...")
    rawUUID, err := d.getMACAdress()

Please try re-build docker-machine-driver-xhyve binary with patched this diff.
Thanks.

Edit:

(For now, I can use virtualbox, so this isn't urgent.)

Ah, so try it if you have time :)

@mboersma
Copy link

I rebuilt docker-machine-xhyve-binary with the above patch, but it ends up the same:

...
(minikube) Generate UUID....
(minikube) DBG | Generated UUID: 274755CE-F166-4A09-B5FF-3500E3D33468
(minikube) DBG | Running command: /usr/local/bin/docker-machine-driver-xhyve [xhyve -A -U 274755CE-F166-4A09-B5FF-3500E3D33468 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk4 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -M]
(minikube) Convert UUID to MAC address...
(minikube) DBG | Converted MAC address: 2a:dc:42:6c:24:b3
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 274755CE-F166-4A09-B5FF-3500E3D33468 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk4 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -F /Users/matt/.minikube/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG | 
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys004 in order to connect to your VM
(minikube) DBG | 
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 4/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | rdmsr to register 0x34 on vcpu 0
(minikube) DBG | 
...

@zchee
Copy link
Contributor

zchee commented Oct 18, 2016

@mboersma Hmm... :(

OK, I'll debug it at tomorrow(today? now AM5:30 in Japan lol) will going to bed...
Sorry, please wait a moment.

@0ff
Copy link

0ff commented Oct 18, 2016

@zchee your latest version, built the way you've described (i.e. make) works fine for me with macOS 10.12 (16A323) whereas the previous version (via brew install) did not work.

minikube delete; rm -rf ~/.minikube; did the trick for me.

@mboersma
Copy link

Just to update, I'm still seeing the same xhyve problems with minikube v0.12.0.

@joshk0
Copy link

joshk0 commented Nov 4, 2016

For those looking for a solution, here's how you can get going with minikube + xhyve right now.

brew install Caskroom/cask/minikube
brew install docker-machine 
brew install docker-machine-driver-xhyve --HEAD
sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve\
sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
minikube start --vm-driver=xhyve

@puja108
Copy link
Member Author

puja108 commented Nov 4, 2016

@joshk0 does not work for me, even with latest HEAD-9d6adee right now (needed to force update with uninstall and install again). Getting the same error as @mboersma above.

@mssio
Copy link

mssio commented Nov 27, 2016

I have same problem on this issue.
I am also installing http://pow.cx/ on my Mac.
When I am uninstalling pow, it magically works.
Any idea why?

@puja108
Copy link
Member Author

puja108 commented Nov 27, 2016

I also still have this problem, sadly, I couldn't find what causes it, yet. Still forced to use vbox

@pcgeek86
Copy link

I'm having this problem as well, but the error I'm getting is quite a bit different, I think. The "SSH command" seems to be misinterpreted as exit 0.

Error Log

➜  ~ minikube start --vm-driver=xhyve --v=5 --show-libmachine-logs
Starting local Kubernetes cluster...
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:52020
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetState
Waiting for SSH to be available...
Getting to WaitForSSH function...
(minikube) Calling .GetSSHHostname
(minikube) DBG |
Error getting ssh command 'exit 0' : IP not found for MAC  in DHCP leases

Homebrew Version of Docker Machine Xhyve Driver

➜  ~ brew info docker-machine-driver-xhyve
docker-machine-driver-xhyve: stable 0.3.1 (bottled), HEAD
Docker Machine driver for xhyve
https://github.com/zchee/docker-machine-driver-xhyve
/usr/local/Cellar/docker-machine-driver-xhyve/0.3.1 (3 files, 13.2M) *
  Poured from bottle on 2016-12-14 at 09:12:09
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/docker-machine-driver-xhyve.rb
==> Dependencies
Build: go ✘
Required: opam ✔, libev ✔
Recommended: docker-machine ✔
==> Requirements
Build: git ✔
Required: macOS >= 10.10 ✔
==> Options
--without-docker-machine
	Build without docker-machine support
--without-qcow2
	Do not support qcow2 disk image format
--HEAD
	Install HEAD version
==> Caveats
This driver requires superuser privileges to access the hypervisor. To
enable, execute
    sudo chown root:wheel /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
    sudo chmod u+s /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

@puja108
Copy link
Member Author

puja108 commented Feb 6, 2017

For me this has been solved by now with some latest updates. Not sure which one actually fixed it, but my current versions are:

minikube version: v0.16.0
docker-machine version 0.9.0, build 15fd4c7
docker-machine-driver-xhyve: stable 0.3.1 (bottled), HEAD
xhyve: stable 0.2.0 (bottled), HEAD```

@shakefu
Copy link

shakefu commented Feb 24, 2017

I can't get this to work at all... On OS X 10.11 El Capitan.

It seems to try to start VirtualBox no matter what I pass for --vm-driver. Is this expected?

$ minikube version
minikube version: v0.16.0

$ brew ls --versions | grep -E 'xhyve|docker'
docker-machine 0.9.0
docker-machine-driver-xhyve 0.3.1
xhyve 0.2.0

$ ls -lah /usr/local/opt/docker-machine-driver-xhyve/bin/
-r-sr-xr-x  1 root         wheel    13M Dec  4 23:25 docker-machine-driver-xhyve

$ minikube config view
- WantReportError: true
- v: 10
- vm-driver: xhyve

$ minikube start --show-libmachine-logs

--show-libmachine-logs is deprecated.
Please use --v=3 to show libmachine logs, and --v=7 for debug level libmachine logs

Starting local Kubernetes cluster...
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:49706
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) DBG | ===== Docker Machine xhyve Driver Version 0.3.1 (Homebrewab0aeba) =====
(minikube) DBG |
(minikube) DBG | executing: /usr/local/bin/VBoxManage -v
(minikube) DBG | STDOUT:
(minikube) DBG | STDERR: /usr/local/bin/VBoxManage: line 2: /Applications/VirtualBox.app/Contents/MacOS/VBoxManage: No such file or directory
(minikube) DBG | /usr/local/bin/VBoxManage: line 2: exec: /Applications/VirtualBox.app/Contents/MacOS/VBoxManage: cannot execute: No such file or directory
(minikube) DBG |
E0224 14:14:04.438626    5020 start.go:107] Error starting host: Error creating host: Error with pre-create check: "Error detecting VBox version: exit status 126".

 Retrying.
E0224 14:14:04.439327    5020 start.go:113] Error starting host:  Error creating host: Error with pre-create check: "Error detecting VBox version: exit status 126"

@simonguest
Copy link

simonguest commented Mar 1, 2017

@shakefu I had the same error message, which was caused by an incomplete VirtualBox uninstallation. Removing /usr/local/bin/VBoxManage resolved my problem, and I was able to use the Xhyve driver successfully.

@imsky
Copy link

imsky commented Apr 9, 2017

@simonguest that was my issue, i had to delete /usr/bin/VBoxManage however

@dnwe
Copy link

dnwe commented Apr 18, 2017

I was having this same issue. Clearing out .minikube ( rm -rf ~/.minikube ) fixed it for me. I was then able to minikube start with the xhyve driver.

@jicowan
Copy link

jicowan commented Jul 3, 2017

I am also getting an the error described above. I couldn't tell whether there was a fix from reading through the thread.

System Version: macOS 10.12.5 (16F73)
Docker Version: 17.06.0-ce, build 02c1d87
Docker Machine Version: 0.12.1, build 45c69ad
docker-machine-driver-xhyve: stable 0.3.3 (bottled)
kubectl: stable v1.7.0

c4b301c6a74b:~ jicowan$ minikube start --vm-driver=xhyve --v=10
Starting local Kubernetes v1.6.4 cluster...
Starting VM...
Downloading Minikube ISO
 90.95 MB / 90.95 MB [==============================================] 100.00% 0s
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:50840
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
(minikube) Calling .DriverName
Creating CA: /Users/jicowan/.minikube/certs/ca.pem
Creating client certificate: /Users/jicowan/.minikube/certs/cert.pem
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) DBG | ===== Docker Machine xhyve Driver Version 0.3.3 (Homebrew7d92f74) =====
(minikube) DBG |
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Downloading /Users/jicowan/.minikube/cache/boot2docker.iso from file:///Users/jicowan/.minikube/cache/iso/minikube-v0.20.0.iso...
(minikube) DBG | Mounting boot2docker.iso
(minikube) Creating VM...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil attach /Users/jicowan/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/jicowan/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42000e018 0xc42000e020 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} attach /Users/jicowan/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/jicowan/.minikube/machines/minikube/b2d-image
(minikube) /dev/disk8          	                               	/Users/jicowan/.minikube/machines/minikube/b2d-image
(minikube) DBG | Extracting Kernel Options...
(minikube) DBG | Extracted Options "loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=minikube"
(minikube) DBG | Extracting /Users/jicowan/.minikube/machines/minikube/b2d-image/boot/bzImage into /Users/jicowan/.minikube/machines/minikube/bzImage
(minikube) DBG | Extracting /Users/jicowan/.minikube/machines/minikube/b2d-image/boot/initrd into /Users/jicowan/.minikube/machines/minikube/initrd
(minikube) DBG | Unmounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil detach /Users/jicowan/.minikube/machines/minikube/b2d-image] []  <nil> 0xc42000e018 0xc42000e020 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} detach /Users/jicowan/.minikube/machines/minikube/b2d-image
(minikube) "disk8" unmounted.
(minikube) "disk8" ejected.
(minikube) Generating 20000MB disk image...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil create -megabytes 20000 -type SPARSEBUNDLE /Users/jicowan/.minikube/machines/minikube/root-volume] []  <nil> 0xc42000e018 0xc42000e020 [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>} create -megabytes 20000 -type SPARSEBUNDLE /Users/jicowan/.minikube/machines/minikube/root-volume
(minikube) created: /Users/jicowan/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/boot2docker.iso
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/bzImage
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/config.json
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/id_rsa
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/id_rsa.pub
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/initrd
(minikube) DBG | /Users/jicowan/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Generate UUID...
(minikube) DBG | Generated UUID: 3E0511AC-641D-499F-932E-BB8EDC699278
(minikube) Convert UUID to MAC address...
(minikube) DBG | Running command: /usr/local/bin/docker-machine-driver-xhyve [xhyve -A -U 3E0511AC-641D-499F-932E-BB8EDC699278 -c 2 -m 2048M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/jicowan/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk8 -f kexec,/Users/jicowan/.minikube/machines/minikube/bzImage,/Users/jicowan/.minikube/machines/minikube/initrd,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=minikube -M]
(minikube) DBG | Converted MAC address: 9a:0:ec:e2:1a:41
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 3E0511AC-641D-499F-932E-BB8EDC699278 -c 2 -m 2048M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/jicowan/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk8 -f kexec,/Users/jicowan/.minikube/machines/minikube/bzImage,/Users/jicowan/.minikube/machines/minikube/initrd,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=minikube -F /Users/jicowan/.minikube/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG |
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC 9a:0:ec:e2:1a:41 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys003 in order to connect to your VM
(minikube) DBG |
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC 9a:0:ec:e2:1a:41 in DHCP leases
(minikube) DBG |
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC 9a:0:ec:e2:1a:41 in DHCP leases

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 31, 2017
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jan 30, 2018
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@prisar
Copy link

prisar commented Oct 3, 2018

@shushen it worked for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/xhyve kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. os/macos
Projects
None yet
Development

No branches or pull requests