hyperkit: IP address is not set / mount.nfs: Connection timed out #3366

antonmarin · 2018-11-27T05:47:46Z

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG or HELP WANTED

How to reproduce it (as minimally and precisely as possible): I start as:

minikube start --loglevel 0 --logtostderr -v=10 --stderrthreshold=0 \
        --memory=4096 --cpus=2 \
        --disk-size "30g" \
        --kubernetes-version="v1.9.9" \
        --vm-driver "hyperkit" \
        --nfs-share ${HOME_DIR} --nfs-shares-root /

sometimes it's ok, so just minikube delete

What happened: I receive

E1127 08:35:36.211915   27977 start.go:159] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: ssh command error:
command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
err     : exit status 32
output  : mount.nfs: Connection timed out
.

 Retrying.
E1127 08:35:36.211982   27977 start.go:165] Error starting host:  Error creating host: Error creating machine: Error in driver during machine creation: ssh command error:
command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
err     : exit status 32
output  : mount.nfs: Connection timed out

Logs

I1127 08:31:05.771463   27977 start.go:96] Viper configuration:
Aliases:
map[string]string{}
Override:
map[string]interface {}{"v":"10"}
PFlags:
map[string]viper.FlagValue{"container-runtime":viper.pflagValue{flag:(*pflag.Flag)(0xc420418dc0)}, "dns-domain":viper.pflagValue{flag:(*pflag.Flag)(0xc420418b40)}, "docker-env":viper.pflagValue{flag:(*pflag.Flag)(0xc420418820)}, "vm-driver":viper.pflagValue{flag:(*pflag.Flag)(0xc4204181e0)}, "cache-images":viper.pflagValue{flag:(*pflag.Flag)(0xc420419040)}, "docker-opt":viper.pflagValue{flag:(*pflag.Flag)(0xc4204188c0)}, "feature-gates":viper.pflagValue{flag:(*pflag.Flag)(0xc420418fa0)}, "memory":viper.pflagValue{flag:(*pflag.Flag)(0xc420418280)}, "xhyve-disk-driver":viper.pflagValue{flag:(*pflag.Flag)(0xc420418640)}, "kvm-network":viper.pflagValue{flag:(*pflag.Flag)(0xc4204185a0)}, "nfs-shares-root":viper.pflagValue{flag:(*pflag.Flag)(0xc420418780)}, "apiserver-ips":viper.pflagValue{flag:(*pflag.Flag)(0xc420418aa0)}, "apiserver-names":viper.pflagValue{flag:(*pflag.Flag)(0xc420418a00)}, "keep-context":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bdd60)}, "network-plugin":viper.pflagValue{flag:(*pflag.Flag)(0xc420418e60)}, "profile":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bcfa0)}, "registry-mirror":viper.pflagValue{flag:(*pflag.Flag)(0xc420418c80)}, "apiserver-name":viper.pflagValue{flag:(*pflag.Flag)(0xc420418960)}, "disable-driver-mounts":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bdf40)}, "disk-size":viper.pflagValue{flag:(*pflag.Flag)(0xc4204183c0)}, "extra-config":viper.pflagValue{flag:(*pflag.Flag)(0xc4204190e0)}, "host-only-cidr":viper.pflagValue{flag:(*pflag.Flag)(0xc420418460)}, "insecure-registry":viper.pflagValue{flag:(*pflag.Flag)(0xc420418be0)}, "mount-string":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bdea0)}, "cpus":viper.pflagValue{flag:(*pflag.Flag)(0xc420418320)}, "iso-url":viper.pflagValue{flag:(*pflag.Flag)(0xc4204180a0)}, "mount":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bde00)}, "bootstrapper":viper.pflagValue{flag:(*pflag.Flag)(0xc4204bd040)}, "hyperv-virtual-switch":viper.pflagValue{flag:(*pflag.Flag)(0xc420418500)}, "kubernetes-version":viper.pflagValue{flag:(*pflag.Flag)(0xc420418d20)}, "nfs-share":viper.pflagValue{flag:(*pflag.Flag)(0xc4204186e0)}}
Env:
map[string]string{}
Key/Value Store:
map[string]interface {}{}
Config:
map[string]interface {}{"wantreporterrorprompt":false}
Defaults:
map[string]interface {}{"wantkubectldownloadmsg":true, "wantnonedriverwarning":true, "v":"0", "reminderwaitperiodinhours":24, "wantreporterror":false, "wantreporterrorprompt":true, "showdriverdeprecationnotification":true, "showbootstrapperdeprecationnotification":true, "alsologtostderr":"false", "log_dir":"", "wantupdatenotification":true}
Starting local Kubernetes v1.9.9 cluster...
Starting VM...
I1127 08:31:05.771706   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/pause-amd64:3.0 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/pause-amd64_3.0
I1127 08:31:05.771706   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/etcd-amd64:3.0.17 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/etcd-amd64_3.0.17
I1127 08:31:05.771749   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.1 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.8.1
I1127 08:31:05.771775   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kube-addon-manager:v6.5 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v6.5
I1127 08:31:05.771799   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.4 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.4
I1127 08:31:05.771818   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.4 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.4
I1127 08:31:05.771820   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kube-scheduler-amd64:v1.9.9 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kube-scheduler-amd64_v1.9.9
I1127 08:31:05.771826   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kube-controller-manager-amd64:v1.9.9 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kube-controller-manager-amd64_v1.9.9
I1127 08:31:05.771843   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.4 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.4
I1127 08:31:05.771857   27977 cluster.go:69] Machine does not exist... provisioning new machine
I1127 08:31:05.771788   27977 cache_images.go:292] Attempting to cache image: gcr.io/k8s-minikube/storage-provisioner:v1.8.0 at /Users/antonmarin/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.0
I1127 08:31:05.771846   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kube-apiserver-amd64:v1.9.9 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kube-apiserver-amd64_v1.9.9
I1127 08:31:05.771858   27977 cache_images.go:292] Attempting to cache image: k8s.gcr.io/kube-proxy-amd64:v1.9.9 at /Users/antonmarin/.minikube/cache/images/k8s.gcr.io/kube-proxy-amd64_v1.9.9
I1127 08:31:05.771867   27977 cluster.go:70] Provisioning machine with config: {MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v0.26.0.iso Memory:4096 CPUs:2 DiskSize:30000 VMDriver:hyperkit XhyveDiskDriver:ahci-hd DockerEnv:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: KvmNetwork:default Downloader:{} DockerOpt:[] DisableDriverMounts:false NFSShare:[/Users/antonmarin] NFSSharesRoot:/ UUID:}
I1127 08:31:05.771919   27977 cache_images.go:79] Successfully cached all images.
I1127 08:31:05.771962   27977 downloader.go:56] Not caching ISO, using https://storage.googleapis.com/minikube/iso/minikube-v0.26.0.iso
Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:54336
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Reading certificate data from /Users/antonmarin/.minikube/certs/ca.pem
Decoding PEM data...
Parsing certificate...
Reading certificate data from /Users/antonmarin/.minikube/certs/cert.pem
Decoding PEM data...
Parsing certificate...
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Downloading /Users/antonmarin/.minikube/cache/boot2docker.iso from file:///Users/antonmarin/.minikube/cache/iso/minikube-v0.26.0.iso...
(minikube) DBG | Writing magic tar header
(minikube) DBG | Writing SSH key tar header
(minikube) Using UUID a2f80226-f205-11e8-a1b0-c4b301c75221
(minikube) Generated MAC 5e:93:ea:aa:48:ca
(minikube) Starting with cmdline: loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=minikube
(minikube) Setting up NFS mounts
(minikube) 192.168.64.31
Password:
(minikube) DBG | Using SSH client type: external
(minikube) DBG | Using SSH private key: /Users/antonmarin/.minikube/machines/minikube/id_rsa (-rw-------)
(minikube) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.64.31 -o IdentitiesOnly=yes -i /Users/antonmarin/.minikube/machines/minikube/id_rsa -p 22] /usr/bin/ssh <nil>}
(minikube) DBG | About to run SSH command:
(minikube) DBG | echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
(minikube) DBG | SSH cmd err, output: exit status 32: mount.nfs: Connection timed out
(minikube) DBG | 
(minikube) DBG | NFS setup failed: ssh command error:
(minikube) DBG | command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
(minikube) DBG | err     : exit status 32
(minikube) DBG | output  : mount.nfs: Connection timed out
(minikube) DBG | 
E1127 08:35:36.211915   27977 start.go:159] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: ssh command error:
command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
err     : exit status 32
output  : mount.nfs: Connection timed out
.

 Retrying.
E1127 08:35:36.211982   27977 start.go:165] Error starting host:  Error creating host: Error creating machine: Error in driver during machine creation: ssh command error:
command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/antonmarin\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/antonmarin ///Users/antonmarin\n" | sh
err     : exit status 32
output  : mount.nfs: Connection timed out

Please provide the following details:

Environment:
macOS High Sierra

Minikube version (use minikube version): minikube version: v0.27.0

OS (e.g. from /etc/os-release): macOS High Sierra
VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): "DriverName": "hyperkit",
ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): "Boot2DockerURL": "file:///Users/antonmarin/.minikube/cache/iso/minikube-v0.26.0.iso"
Install tools:
Others:
The above can be generated in one go with the following commands (can be copied and pasted directly into your terminal):

minikube version
echo "";
echo "OS:";
cat /etc/os-release
echo "";
echo "VM driver:"; 
grep DriverName ~/.minikube/machines/minikube/config.json
echo "";
echo "ISO version";
grep -i ISO ~/.minikube/machines/minikube/config.json

Output of minikube logs (if applicable):

▶ minikube logs
F1127 08:45:27.770591   29902 logs.go:50] Error getting cluster bootstrapper: getting kubeadm bootstrapper: getting ssh client: Error dialing tcp via ssh client: dial tcp :22: getsockopt: connection refused

Anything else do we need to know:

▶ docker-machine -s ~/.minikube --debug ls
Docker Machine Version:  0.15.0, build b48dc28d
Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:54673
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
timeout set to 10s
(minikube) Calling .GetURL
(minikube) Calling .GetState
(minikube) Calling .DriverName
NAME       ACTIVE   DRIVER     STATE     URL   SWARM   DOCKER    ERRORS
minikube   -        hyperkit   Running                 Unknown   IP address is not set
Making call to close driver server
(minikube) Calling .Close
Successfully made call to close driver server
Making call to close connection to plugin binary

The text was updated successfully, but these errors were encountered:

antonmarin · 2018-12-10T04:42:02Z

More info: I use dnsmasq and nfs mounting, and there is 192.168.64.1 in my dnsmasq.conf

antonmarin · 2018-12-10T04:53:22Z

Seems like clearing /etc/exports helps, testing

antonmarin · 2018-12-14T04:46:14Z

sudo bash -c "echo '' > /etc/exports"; minikube delete;
// start again

Helps. Is there any idea why it happens?

tstromberg · 2019-01-23T20:11:26Z

Does this problem go away if dnsmasq is not running?

antonmarin · 2019-01-24T06:12:01Z

I’ll try today evening

antonmarin · 2019-01-30T10:01:34Z

Tried for 2 days and i can not reproduce this behaviour. 😞

antonmarin · 2019-01-30T11:07:08Z

Going to test inside another network, where it reproduces more often

antonmarin · 2019-01-30T12:14:33Z

Retrying.
E0130 15:11:35.684917    4730 start.go:165] Error starting host:  Error creating host: Error creating machine: Error in driver during machine creation: ssh command error:
command : echo -e "#/bin/bash\nsudo mkdir -p ///Users/g.bessonov\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users/g.bessonov ///Users/g.bessonov\n" | sh
err     : exit status 32
output  : mount.nfs: Connection timed out

Stopping dnsmasq changed nothing

antonmarin · 2019-02-21T13:28:04Z

Updated to v0.34.1. Got the same error, but now

minikube ip
💣  Error getting IP: IP address is not set

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new

fejta-bot · 2019-05-22T13:39:58Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

tstromberg · 2019-05-22T22:27:16Z

I'm closing this issue as it hasn't seen activity in awhile, and it's unclear if this issue still exists. If this issue does continue to exist in the most recent release of minikube, please feel free to re-open it.

Make sure to upgrade to the latest hyperkit driver:

curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-hyperkit \
&& sudo install -o root -g wheel -m 4755 docker-machine-driver-hyperkit /usr/local/bin/

Thank you for opening the issue!

antonmarin · 2019-05-23T04:41:14Z

Ok. I haven't seen such behaviour for a month (after moved to v1.0).

tstromberg added kind/bug Categorizes issue or PR as related to a bug. os/macos co/hyperkit Hyperkit related issues area/networking networking issues labels Dec 4, 2018

tstromberg changed the title ~~Docker-machine with hyperkit sometimes don't acquire ip address~~ hyperkit: IP address is not set / mount.nfs: Connection timed out Dec 4, 2018

tstromberg added the priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. label Jan 23, 2019

tstromberg added the help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. label Jan 24, 2019

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 22, 2019

tstromberg closed this as completed May 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hyperkit: IP address is not set / mount.nfs: Connection timed out #3366

hyperkit: IP address is not set / mount.nfs: Connection timed out #3366

antonmarin commented Nov 27, 2018

antonmarin commented Dec 10, 2018 •

edited

Loading

antonmarin commented Dec 10, 2018

antonmarin commented Dec 14, 2018 •

edited

Loading

tstromberg commented Jan 23, 2019

antonmarin commented Jan 24, 2019

antonmarin commented Jan 30, 2019

antonmarin commented Jan 30, 2019 •

edited

Loading

antonmarin commented Jan 30, 2019 •

edited

Loading

antonmarin commented Feb 21, 2019

fejta-bot commented May 22, 2019

tstromberg commented May 22, 2019

antonmarin commented May 23, 2019

hyperkit: IP address is not set / mount.nfs: Connection timed out #3366

hyperkit: IP address is not set / mount.nfs: Connection timed out #3366

Comments

antonmarin commented Nov 27, 2018

antonmarin commented Dec 10, 2018 • edited Loading

antonmarin commented Dec 10, 2018

antonmarin commented Dec 14, 2018 • edited Loading

tstromberg commented Jan 23, 2019

antonmarin commented Jan 24, 2019

antonmarin commented Jan 30, 2019

antonmarin commented Jan 30, 2019 • edited Loading

antonmarin commented Jan 30, 2019 • edited Loading

antonmarin commented Feb 21, 2019

fejta-bot commented May 22, 2019

tstromberg commented May 22, 2019

antonmarin commented May 23, 2019

antonmarin commented Dec 10, 2018 •

edited

Loading

antonmarin commented Dec 14, 2018 •

edited

Loading

antonmarin commented Jan 30, 2019 •

edited

Loading

antonmarin commented Jan 30, 2019 •

edited

Loading