Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minikube start with ssh as driver ignores --ssh-key parameter #10289

Closed
songford opened this issue Jan 28, 2021 · 12 comments
Closed

Minikube start with ssh as driver ignores --ssh-key parameter #10289

songford opened this issue Jan 28, 2021 · 12 comments
Labels
co/generic-driver kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence.

Comments

@songford
Copy link

Steps to reproduce the issue:

  1. minikube start --driver=ssh --ssh-ip-address=192.168.233.5 --ssh-key='~/.ssh/id_rsa' --ssh-user=songford --alsologtostderr -v=4

Full output of failed command:

😄  minikube v1.17.0 on Darwin 11.1
✨  Using the ssh driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🏃  Updating the running ssh "minikube" bare metal machine ...
🤦  StartHost failed, but will try again: provision: fast detect: OS type not recognized
🔄  Restarting existing ssh bare metal machine for "minikube" ...

With --alsologtostderr -v=4:

I0128 11:29:34.855887   80187 out.go:229] Setting OutFile to fd 1 ...
I0128 11:29:34.856501   80187 out.go:281] isatty.IsTerminal(1) = true
I0128 11:29:34.856510   80187 out.go:242] Setting ErrFile to fd 2...
I0128 11:29:34.856516   80187 out.go:281] isatty.IsTerminal(2) = true
I0128 11:29:34.856621   80187 root.go:280] Updating PATH: /Users/I342328/.minikube/bin
I0128 11:29:34.857016   80187 out.go:236] Setting JSON to false
I0128 11:29:34.903806   80187 start.go:106] hostinfo: {"hostname":"C02CP0ZYMD6R","uptime":487923,"bootTime":1611316651,"procs":641,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"11.1","kernelVersion":"20.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"82e2050c-5936-3d24-ad3b-ec4ec5c09e11"}
W0128 11:29:34.903942   80187 start.go:114] gopshost.Virtualization returned error: not implemented yet
I0128 11:29:34.977337   80187 out.go:119] 😄  minikube v1.17.0 on Darwin 11.1
😄  minikube v1.17.0 on Darwin 11.1
I0128 11:29:34.977609   80187 notify.go:126] Checking for updates...
I0128 11:29:34.979161   80187 ssh_runner.go:149] Run: systemctl --version
I0128 11:29:34.979310   80187 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0128 11:29:35.260651   80187 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0128 11:29:35.800978   80187 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0128 11:29:36.460258   80187 retry.go:31] will retry after 791.196345ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0128 11:29:37.252724   80187 driver.go:315] Setting default libvirt URI to qemu:///system
I0128 11:29:37.320599   80187 out.go:119] ✨  Using the ssh driver based on existing profile
✨  Using the ssh driver based on existing profile
I0128 11:29:37.320650   80187 start.go:279] selected driver: ssh
I0128 11:29:37.320660   80187 start.go:702] validating driver "ssh" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:ssh HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress:192.168.233.5 SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.233.5 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
I0128 11:29:37.320851   80187 start.go:713] status for ssh: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0128 11:29:37.321171   80187 start_flags.go:390] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:ssh HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress:192.168.233.5 SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.233.5 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
I0128 11:29:37.364779   80187 out.go:119] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I0128 11:29:37.364842   80187 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0128 11:29:37.364962   80187 preload.go:105] Found local preload: /Users/I342328/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4
I0128 11:29:37.364979   80187 cache.go:54] Caching tarball of preloaded images
I0128 11:29:37.365010   80187 preload.go:131] Found /Users/I342328/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0128 11:29:37.365017   80187 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0128 11:29:37.365158   80187 profile.go:148] Saving config to /Users/I342328/.minikube/profiles/minikube/config.json ...
I0128 11:29:37.366246   80187 cache.go:185] Successfully downloaded all kic artifacts
I0128 11:29:37.366289   80187 start.go:313] acquiring machines lock for minikube: {Name:mked90d01b554410cfff82b594e94a4e6ef2fff1 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0128 11:29:37.366420   80187 start.go:317] acquired machines lock for "minikube" in 92.879µs
I0128 11:29:37.366470   80187 start.go:93] Skipping create...Using existing machine configuration
I0128 11:29:37.366484   80187 fix.go:54] fixHost starting:
I0128 11:29:37.367223   80187 fix.go:107] recreateIfNeeded on minikube: state=Running err=<nil>
W0128 11:29:37.367239   80187 fix.go:133] unexpected machine state, will restart: <nil>
I0128 11:29:37.434783   80187 out.go:119] 🏃  Updating the running ssh "minikube" bare metal machine ...
🏃  Updating the running ssh "minikube" bare metal machine ...
I0128 11:29:37.434936   80187 machine.go:88] provisioning docker machine ...
I0128 11:29:37.434967   80187 main.go:119] libmachine: Waiting for SSH to be available...
I0128 11:29:37.434987   80187 main.go:119] libmachine: Getting to WaitForSSH function...
I0128 11:29:37.435182   80187 main.go:119] libmachine: Using SSH client type: native
I0128 11:29:37.435262   80187 main.go:119] libmachine: &{{{<nil> 0 [] [] []} root [] 0x440c880 <nil>  [] 0s} 192.168.233.5 22 <nil> <nil>}
I0128 11:29:37.435275   80187 main.go:119] libmachine: About to run SSH command:
exit 0
I0128 11:29:37.467165   80187 main.go:119] libmachine: Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain
I0128 11:29:40.504417   80187 main.go:119] libmachine: Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain

But I am able to use ssh -i ~/.ssh/id_rsa songford@192.168.233.5 to connect to this VM.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 28, 2021

Thanks for reporting, it seems like this also happens on Windows - but haven't really seen it on Linux...

Since this is the second start ("updating the running"), the logs might not have all the details though.

@afbjorklund afbjorklund added co/generic-driver kind/bug Categorizes issue or PR as related to a bug. priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. labels Jan 28, 2021
@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 28, 2021

Note that ~/.ssh/id_rsa is actually the default. So the parameter should not even be needed.

EDIT: Does it work better with "$HOME/.ssh/id_rsa" ? It seems that it is not expanding the tilde.

✨  Using the ssh driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🤦  StartHost failed, but will try again: creating host: create: precreate: SSH key does not exist: "~/.ssh/id_rsa"
😿  Failed to start ssh bare metal machine. Running "minikube delete" may fix it: creating host: create: precreate: SSH key does not exist: "~/.ssh/id_rsa"

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 28, 2021

These messages are "normal":

Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
It takes a while for minikube to call the Config, but eventually it does and then it moves on (after a couple of retries).

Couldn't copy SSH public key : unable to copy ssh key: open private_key.pub: no such file or directory
There is not always a public key available for copying, but it is basically only the private key that matters anyway*.

* assuming the VM is set up...

And it should give errors, if there was something wrong with the key:

precreate: SSH key does not exist: "~/.ssh/id_rsa"

ssh: no key found

There is an old (2016) TODO, to validate the key in PreCreateCheck...

@afbjorklund
Copy link
Collaborator

I can confirm that it does seem to be a little stupid, when not using the external ssh but the "native" ssh.

creating host: create: creating: usermod: NewSession: new client: new client: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain

There is supposed to be a publickey (from file or from agent) in there, otherwise it will be hard to get in...

Unfortunately the NativeClient is 1) stupid and 2) hardcoded, so it doesn't add the default auth methods.

@afbjorklund afbjorklund added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. labels Jan 28, 2021
@songford
Copy link
Author

@afbjorklund thanks for following up with this issue. I've tried expanding the path into a full one manually which didn't help either. I am currently using docker driver to bypass this issue, but if what I'm experiencing is universal, it seems the ssh driver on Mac is somewhat broken...Hard to imagine any serious SSH target uses none as its authentication method.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 28, 2021

Hard to imagine any serious SSH target uses none as its authentication method.

It's the default, but usually we add some more too :-)

Broken on all platforms, but you should be able to use --ssh-key "$HOME/.ssh/id_rsa"

https://github.com/kubernetes/minikube/blob/v1.17.0/pkg/minikube/sshutil/sshutil.go#L43_L46

When fixed, it will support ssh-agent and ~/.ssh/id_rsa by default (without param)

Basically the same as when using ssh (OpenSSH) CLI ?

@afbjorklund afbjorklund added priority/backlog Higher priority than priority/awaiting-more-evidence. and removed priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. labels Jan 28, 2021
@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 28, 2021

@songford

thanks for following up with this issue. I've tried expanding the path into a full one manually which didn't help either. I am currently using docker driver to bypass this issue, but if what I'm experiencing is universal, it seems the ssh driver on Mac is somewhat broken..

The current workaround is: (possible to change user and port too)

minikube start --driver=ssh --ssh-ip-address=vm.example.com \
--ssh-key=$HOME/.ssh/id_rsa --native-ssh=false

Because using tilde and native ssh client doesn't work at the moment.

@songford
Copy link
Author

songford commented Jan 29, 2021

@afbjorklund

I342328@C02CP0ZYMD6R ~ % minikube start --driver=ssh --ssh-ip-address=192.168.233.5 --ssh-key=$HOME/.ssh/id_rsa --native-ssh=false --alsologtostderr -v=4 --ssh-user=fizzytidus
I0129 15:40:34.875632   16624 out.go:229] Setting OutFile to fd 1 ...
I0129 15:40:34.876341   16624 out.go:281] isatty.IsTerminal(1) = true
I0129 15:40:34.876351   16624 out.go:242] Setting ErrFile to fd 2...
I0129 15:40:34.876356   16624 out.go:281] isatty.IsTerminal(2) = true
I0129 15:40:34.876439   16624 root.go:280] Updating PATH: /Users/I342328/.minikube/bin
I0129 15:40:34.876853   16624 out.go:236] Setting JSON to false
I0129 15:40:34.915438   16624 start.go:106] hostinfo: {"hostname":"C02CP0ZYMD6R","uptime":15551,"bootTime":1611890483,"procs":551,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"11.1","kernelVersion":"20.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"82e2050c-5936-3d24-ad3b-ec4ec5c09e11"}
W0129 15:40:34.915583   16624 start.go:114] gopshost.Virtualization returned error: not implemented yet
I0129 15:40:34.984178   16624 out.go:119] 😄  minikube v1.17.0 on Darwin 11.1
😄  minikube v1.17.0 on Darwin 11.1
I0129 15:40:34.986424   16624 ssh_runner.go:149] Run: systemctl --version
I0129 15:40:34.986530   16624 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0129 15:40:35.264430   16624 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0129 15:40:35.808612   16624 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0129 15:40:36.465552   16624 retry.go:31] will retry after 791.196345ms: new client: new client: Error creating new ssh host from driver: Error getting ssh host name for driver: IP address is not set
I0129 15:40:37.257649   16624 driver.go:315] Setting default libvirt URI to qemu:///system
I0129 15:40:37.324809   16624 out.go:119] ✨  Using the ssh driver based on existing profile
✨  Using the ssh driver based on existing profile
I0129 15:40:37.324851   16624 start.go:279] selected driver: ssh
I0129 15:40:37.324858   16624 start.go:702] validating driver "ssh" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:ssh HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress:192.168.233.5 SSHUser:root SSHKey:/Users/I342328/.ssh/id_rsa SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
I0129 15:40:37.324988   16624 start.go:713] status for ssh: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0129 15:40:37.325261   16624 start_flags.go:390] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:ssh HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress:192.168.233.5 SSHUser:root SSHKey:/Users/I342328/.ssh/id_rsa SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
I0129 15:40:37.349863   16624 out.go:119] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I0129 15:40:37.349919   16624 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0129 15:40:37.349982   16624 preload.go:105] Found local preload: /Users/I342328/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4
I0129 15:40:37.349998   16624 cache.go:54] Caching tarball of preloaded images
I0129 15:40:37.350023   16624 preload.go:131] Found /Users/I342328/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0129 15:40:37.350031   16624 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0129 15:40:37.350211   16624 profile.go:148] Saving config to /Users/I342328/.minikube/profiles/minikube/config.json ...
I0129 15:40:37.370127   16624 cache.go:185] Successfully downloaded all kic artifacts
I0129 15:40:37.370187   16624 start.go:313] acquiring machines lock for minikube: {Name:mked90d01b554410cfff82b594e94a4e6ef2fff1 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0129 15:40:37.370362   16624 start.go:317] acquired machines lock for "minikube" in 152.639µs
I0129 15:40:37.370403   16624 start.go:93] Skipping create...Using existing machine configuration
I0129 15:40:37.370413   16624 fix.go:54] fixHost starting:
I0129 15:40:37.371767   16624 fix.go:107] recreateIfNeeded on minikube: state=Running err=<nil>
W0129 15:40:37.371788   16624 fix.go:133] unexpected machine state, will restart: <nil>
I0129 15:40:37.437289   16624 out.go:119] 🏃  Updating the running ssh "minikube" bare metal machine ...
🏃  Updating the running ssh "minikube" bare metal machine ...
I0129 15:40:37.437402   16624 machine.go:88] provisioning docker machine ...
I0129 15:40:37.437497   16624 main.go:119] libmachine: Waiting for SSH to be available...
I0129 15:40:37.437531   16624 main.go:119] libmachine: Getting to WaitForSSH function...
I0129 15:40:37.437809   16624 main.go:119] libmachine: Using SSH client type: external
I0129 15:40:37.437938   16624 main.go:119] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.233.5 -p 22] /usr/local/bin/ssh <nil>}
I0129 15:40:37.438052   16624 main.go:119] libmachine: About to run SSH command:
exit 0
I0129 15:40:37.499032   16624 main.go:119] libmachine: SSH cmd err, output: exit status 255:
I0129 15:40:37.499080   16624 main.go:119] libmachine: Error getting ssh command 'exit 0' : ssh command error:
command : exit 0
err     : exit status 255
output  :
I0129 15:40:40.503665   16624 main.go:119] libmachine: Getting to WaitForSSH function...
I0129 15:40:40.503996   16624 main.go:119] libmachine: Using SSH client type: external
I0129 15:40:40.504089   16624 main.go:119] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.233.5 -p 22] /usr/local/bin/ssh <nil>}
I0129 15:40:40.504121   16624 main.go:119] libmachine: About to run SSH command:
exit 0
I0129 15:40:40.569600   16624 main.go:119] libmachine: SSH cmd err, output: exit status 255:
I0129 15:40:40.569644   16624 main.go:119] libmachine: Error getting ssh command 'exit 0' : ssh command error:
command : exit 0
err     : exit status 255
output  :

Unfortunately, using --native-ssh=false doesn't seem to help either. Don't know how I should go about the log lines, but it sounds like somehow it thinks the SSH client is exit 0?

@afbjorklund
Copy link
Collaborator

What is the version of /usr/local/bin/ssh ?

Can you try to run the command with -v ?

It looks like it is using the wrong user, but maybe something else is off.

--ssh-ip-address=192.168.233.5 --ssh-key=$HOME/.ssh/id_rsa --ssh-user=fizzytidus

root@192.168.233.5 -p 22 (and no sign of the -i parameter either, so not "working")

Unfortunately, using --native-ssh=false doesn't seem to help either.

It did call the external ssh program as requested, so that part is good...

it sounds like somehow it thinks the SSH client can be called with exit 0

It should be possible:

minikube --alsologtostderr ssh exit 0


🏃 Updating the running ssh "minikube" bare metal machine ...

It probably used the previous profile/config ?

  • ~/.minikube/profiles/minikube/config.json
  • ~/.minikube/machines/minikube/config.json

@songford
Copy link
Author

songford commented Jan 29, 2021

What is the version of /usr/local/bin/ssh ?

ssh on my distro (macOS, hence darwin/BSD?) doesn't take -v. If I look into man ssh, it says

     OpenSSH is a derivative of the original and free ssh 1.2.12 release by Tatu Ylonen.  Aaron Campbell, Bob Beck, Markus
     Friedl, Niels Provos, Theo de Raadt and Dug Song removed many bugs, re-added newer features and created OpenSSH.  Markus
     Friedl contributed the support for SSH protocol versions 1.5 and 2.0.

BSD                            January 29, 2021                            BSD

Sorry I wasn't clear about what I meant by

it sounds like somehow it thinks the SSH client can be called with exit 0

I meant to say it thinks my SSH client is exit 0 somehow?

Also this line
I0129 15:40:37.437938 16624 main.go:119] libmachine: &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.233.5 -p 22] /usr/local/bin/ssh <nil>} I0129 15:40:37.438052 16624 main.go:119] libmachine: About to run SSH command:

ls it how minikube calls the ssh client? I tried to run

/usr/local/bin/ssh -F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.233.5 -p 22

as shown in the log on my system, and it doesn't return anything to either STDOUT or STDERR. (Probably due to setting LogLevel=quiet) echo $? returns 255, which corresponds to the log.

If I add the missing -i parameter to my ssh private key and changing the user logging into my VM as the one provided by --ssh-user into the command, then I can SSH into my VM.

So I suspect if I change the code handling SSH to honour the user and key supplied, the issue should be resolved. Can I attempt to do that? I can try to submit a PR to fix it.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 29, 2021

Usually -v (or -vvv) is used to increase ssh verbosity. Like you mention, removing quiet is probably also a good idea.

    -V      Display the version number and exit.

     -v      Verbose mode.  Causes ssh to print debugging messages about its progress.  This is
             helpful in debugging connection, authentication, and configuration problems.  Multiple
             -v options increase the verbosity.  The maximum is 3.

You can either delete the old "ssh" driver profile (it's just the connection parameters anyway), or start a new one (-p)

@afbjorklund
Copy link
Collaborator

Closed in #10293 and #10298, it will now validate any given key and default to ~/.ssh/id_rsa if none given.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/generic-driver kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence.
Projects
None yet
Development

No branches or pull requests

2 participants