Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

windows wsl2: Fix invoking kubeadm failing when spaces in PATH for none driver #12617

Merged
merged 1 commit into from
Sep 29, 2021

Conversation

marwatk
Copy link
Contributor

@marwatk marwatk commented Sep 28, 2021

Experimenting with minikube on bare metal in WSL2 breaks if Windows has put any items in your path containing spaces (e.g. Program Files):

❌  Exiting due to GUEST_START: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem": exit status 127
stdout:

stderr:
env: ‘Files’: No such file or directory

This simple patch corrects the behavior.

Because of the simplicity of the change I didn't alter any tests but I'm happy to if necessary.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 28, 2021
@k8s-ci-robot
Copy link
Contributor

Welcome @marwatk!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot
Copy link
Contributor

Hi @marwatk. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 28, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: marwatk
To complete the pull request process, please assign medyagh after the PR has been reviewed.
You can assign the PR to them by writing /assign @medyagh in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@medyagh
Copy link
Member

medyagh commented Sep 28, 2021

@marwatk is this for None Driver ?

do you mind echoing an example of full Env that breaks minikube ?

@medyagh
Copy link
Member

medyagh commented Sep 28, 2021

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 28, 2021
@marwatk
Copy link
Contributor Author

marwatk commented Sep 28, 2021

@medyagh

is this for the None Driver

Yes, none driver.

do you mind echoing an example of full Env that breaks minikube ?

Here you go (this is on actual Linux, not WSL2):

# PATH="$PATH:with space" out/minikube start --driver=none
😄  minikube v1.23.2 on Rocky 8.4
✨  Using the none driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing none bare metal machine for "minikube" ...
ℹ️  OS release is Rocky Linux 8.4 (Green Obsidian)
🐳  Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
💢  initialization failed, will try again: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.22.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem": exit status 127
stdout:

stderr:
env: ‘space’: No such file or directory


💣  Error starting cluster: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.22.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem": exit status 127
stdout:

stderr:
env: ‘space’: No such file or directory


╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

❌  Exiting due to GUEST_START: wait: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.22.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem": exit status 127
stdout:

stderr:
env: ‘space’: No such file or directory


╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12617) |
+----------------+----------+---------------------+
| minikube start | 48.5s    | 48.0s               |
| enable ingress | 33.4s    | 32.7s               |
+----------------+----------+---------------------+

Times for minikube (PR 12617) start: 47.8s 47.3s 47.5s 49.2s 47.9s
Times for minikube start: 50.5s 47.9s 48.1s 48.3s 47.4s

Times for minikube ingress: 31.9s 33.3s 34.7s 33.2s 33.7s
Times for minikube (PR 12617) ingress: 31.8s 32.8s 32.9s 32.8s 33.3s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12617) |
+----------------+----------+---------------------+
| minikube start | 21.8s    | 21.4s               |
| enable ingress | 32.2s    | 32.3s               |
+----------------+----------+---------------------+

Times for minikube (PR 12617) start: 21.1s 21.1s 20.8s 21.4s 22.4s
Times for minikube start: 22.4s 20.8s 21.8s 21.9s 22.0s

Times for minikube ingress: 34.4s 27.9s 35.9s 35.5s 27.4s
Times for minikube (PR 12617) ingress: 26.9s 36.4s 27.9s 35.9s 34.4s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12617) |
+----------------+----------+---------------------+
| minikube start | 41.1s    | 40.2s               |
| enable ingress | 33.7s    | 36.2s               |
+----------------+----------+---------------------+

Times for minikube start: 30.9s 43.5s 44.0s 43.1s 44.1s
Times for minikube (PR 12617) start: 25.7s 44.9s 43.5s 43.9s 43.0s

Times for minikube ingress: 28.4s 33.9s 36.4s 33.0s 36.9s
Times for minikube (PR 12617) ingress: 37.4s 62.4s 19.9s 28.9s 32.5s

@medyagh
Copy link
Member

medyagh commented Sep 28, 2021

@medyagh

is this for the None Driver

Yes, none driver.

do you mind echoing an example of full Env that breaks minikube ?

Here you go (this is on actual Linux, not WSL2):

# PATH="$PATH:with space" out/minikube start --driver=none

`

@marwatk I am curious, what would be a real world example that a path would have space in it ? was it a corrupt path env?

@marwatk
Copy link
Contributor Author

marwatk commented Sep 28, 2021

@medyagh WSL2 adds your Windows path to your linux path to launch windows apps from the linux shell. Some of these can have spaces (like Program Files).

--driver=none is so far the only option that offers an equivalent experience to what you get with Docker Desktop's in-built k8s.

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Windows TestStartStop/group/embed-certs/serial/Pause (gopogh) 13.89 (chart)
Docker_Windows TestStoppedBinaryUpgrade/MinikubeLogs (gopogh) 28.91 (chart)
Docker_Windows TestAddons/Setup (gopogh) 43.75 (chart)
Docker_Windows TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 57.14 (chart)
Docker_Windows TestNetworkPlugins/group/bridge/Start (gopogh) 60.34 (chart)
Docker_Windows TestNetworkPlugins/group/kubenet/Start (gopogh) 66.38 (chart)
Docker_Windows TestNetworkPlugins/group/custom-weave/Start (gopogh) 70.69 (chart)
Docker_Windows TestNetworkPlugins/group/kindnet/DNS (gopogh) 70.97 (chart)
Docker_Windows TestNetworkPlugins/group/calico/Start (gopogh) 90.52 (chart)
Docker_Windows TestCertOptions (gopogh) 99.22 (chart)
Docker_Linux_containerd TestScheduledStopUnix (gopogh) 100.00 (chart)

To see the flake rates of all tests by environment, click here.

@medyagh
Copy link
Member

medyagh commented Sep 29, 2021

@medyagh WSL2 adds your Windows path to your linux path to launch windows apps from the linux shell. Some of these can have spaces (like Program Files).

--driver=none is so far the only option that offers an equivalent experience to what you get with Docker Desktop's in-built k8s.

That makes sense thanks for the explanation

@medyagh
Copy link
Member

medyagh commented Sep 29, 2021

And thank you for fixing this for everyone else

@medyagh medyagh merged commit 1b955d8 into kubernetes:master Sep 29, 2021
@medyagh medyagh changed the title Fix invoking kubeadm failing when spaces in PATH Windows wsl2: Fix invoking kubeadm failing when spaces in PATH Sep 29, 2021
@medyagh medyagh changed the title Windows wsl2: Fix invoking kubeadm failing when spaces in PATH windows wsl2: Fix invoking kubeadm failing when spaces in PATH for none driver Sep 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants