More documentation around vm-driver=none for local use

[jakefeasel]

Is this a BUG REPORT or FEATURE REQUEST? (choose one): FEATURE REQUEST

Please provide the following details:

Environment:

Minikube version (use minikube version): v0.25.0

• OS (e.g. from /etc/os-release): "Ubuntu 16.04.3 LTS (Xenial Xerus)"
• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): none
• ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): n/a
• Install tools:
• Others: Docker version 18.02.0-ce

What happened: I spent a long time struggling to get things working properly with vm-driver=none

What you expected to happen: Find documentation which provides coverage for this very valuable use-case.

Anything else do we need to know:

I have a linux laptop that I want to use k8s on. I do not want to run a VM, since that is an inefficient use of my laptop's memory (always either too much or too little memory allocated for the given docker containers I want to run) and slower due to the extra virtualization. Also it shouldn't be necessary - I have a Linux environment, why should I have to run another VM just to run Linux? That's why I bought this laptop.

The use of vm-driver=none is hardly documented at all. What little there is does not seem to consider the value for developer machines.

Yesterday I filed #2571, but this was actually not the full story. I had a red-herring with IP addresses which led me to believe that the docker0 interface was the root of the problem. It turns out, the real problem was whatever IP address I happened to have bound to my ethernet interface would be used in the construction of the cluster. If that IP address changed for any reason (as often happens on laptops) then the whole environment was inaccessible.

The workaround was not to specify a bridge IP for docker, as I had thought. Instead you need to start minikube like so:

minikube start --vm-driver=none --apiserver-ips 127.0.0.1 --apiserver-name localhost

And then go and edit ~/.kube/config, replacing the server IP that was detected from the main network interface with "localhost". For example, mine now looks like this:

- cluster:
    certificate-authority: /home/jfeasel/.minikube/ca.crt
    server: https://localhost:8443
  name: minikube

With this configuration, I can access my local cluster all of the time, even if the main network interface is disabled.

Also, we should note that it is required to have "socat" installed on the Linux environment. See this issue for details: kubernetes/kubernetes#19765 I saw this when I tried to use helm to connect to my local cluster; I got errors with port-forwarding. Since I'm using Ubuntu all I had to do was sudo apt-get install socat and then everything worked as expected.

[harpratap]

@jakefeasel Kinda unrelated but do you know how to let minikube pull images I created locally? All the docs mention using minikube docker-env but it returns none in case of using docker driver.

[ncabatoff]

@harpratap see #2443, your locally available images are also available to kubernetes with vm-driver=none.

[harpratap]

@ncabatoff Do I need to specify some additional parameter when starting minkube to let it access those locally created images? I get this error -
Failed to pull image "mylocalnginx": rpc error: code = Unknown desc = Error response from daemon: pull access denied for mylocalnginx, repository does not exist or may require 'docker login'

[ncabatoff]

Not that I'm aware of. Note that you don't even have to pull them: with vm-driver=none it's the same docker daemon as on your desktop, i.e. the one you've been populating when you build images. So just skip the pull you're doing now, reference the images normally in your container manifests, and it should just work. At least that's been my experience.

[harpratap]

@ncabatoff Yes that is what I did and I got the image pull error, didn't do any manual pulling of images.

Edit: Nevermind, got it working by setting imagePullPolicy to Never. Thanks!

Edit Again: Don't change the imagePullPolicy, you just need to tag your local images with a version other than latest, so I just built my image mylocalnginx:v1 and it worked.

[bennettellis]

Seems like the right place to add a couple of items that I've discovered along the way with vm-driver=none:

1. minikube mount not supported
2. minikube ssl not supported and doesn't seem to be any way to ssh to the minikube "node"?? This is particularly a problem when using something like Terraform file provisioner to move content to the node to share with pods. However, see item 3 below.
3. mounting a host volume to a pod mounts it from the underlying host, not the "node" that is(?) minikube. Maybe put another way is that the host behind minikube is also in effect the node in some respects, at least when the vm-driver is none.

[bw2]

Also having issues similar to https://brokenco.de/2018/09/04/minikube-vmdriver-none.html