none driver can't be used with non-Docker runtimes (looks for "docker" executable)

[afbjorklund]

There is something wrong in the runtime detection of the precreate:

$ sudo minikube start --vm-driver=none --container-runtime=cri-o
😄  minikube v1.4.0 on Ubuntu 16.04
🤹  Running on localhost (CPUs=4, Memory=7800MB, Disk=138379MB) ...
🔄  Retriable failure: create: precreate: exec: "docker": executable file not found in $PATH
🤹  Running on localhost (CPUs=4, Memory=7800MB, Disk=138379MB) ...
🔄  Retriable failure: create: precreate: exec: "docker": executable file not found in $PATH
🤹  Running on localhost (CPUs=4, Memory=7800MB, Disk=138379MB) ...
🔄  Retriable failure: create: precreate: exec: "docker": executable file not found in $PATH
^C

For some reason it is calling the wrong Available function ?

func (r *Docker) Available() error {
	_, err := exec.LookPath("docker")
	return err
}

func (r *CRIO) Available() error {
	return r.Runner.Run("command -v crio")
}

And the docker runtime seems to be checking locally ?
(I guess us using docker machine makes it always there)

It also forgot to look for crictl, but that is another story.
(and interesting how this is regarded as a "retriable failure")

[afbjorklund]

Interestingly, we are calling NewDriver twice. Once with correct values, once with empty ones:

I1006 18:22:15.786795   11858 none.go:67] none: {minikube /home/anders/.minikube cri-o}
I1006 18:22:15.786826   11858 cruntime.go:89] cruntime: {cri-o  0x3027db0  }
I1006 18:22:15.786953   11858 none.go:67] none: {  }
I1006 18:22:15.786973   11858 cruntime.go:89] cruntime: {  0x3027db0  }

This is how we end up with Docker for the runtime, even though it was supposed to be CRIO.

	case "", "docker":
		return &Docker{Socket: c.Socket, Runner: c.Runner}, nil
	case "crio", "cri-o":
		return &CRIO{Socket: c.Socket, Runner: c.Runner, ImageRepository: c.ImageRepository, KubernetesVersion: c.KubernetesVersion}, nil

[afbjorklund]

Unfortunately this suggests that something is fundamentally wrong in the Driver "registry" ?

First the config is happily created by "createNoneHost", only to get clobbered by "init"'s empty

[afbjorklund]

Hopefully someone else understands how the "registry" is supposed to work, and can fix this ?

An interesting story on RHEL 8 is that installing podman-docker makes it segfault instead.
Mostly due to the "unexpected" advertisement on stdout, about docker compatibility issues:

# docker version
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Version:       1.0.5
Go Version:    go1.11.6
OS/Arch:       linux/amd64

[Skybladev2]

I see this error when deleting and starting minikube cluster again (no args).
How to fix this?

[afbjorklund]

Well, the workaround is to make something/anything called "docker" available.
But the real fix means improving the minikube driver registry, which is broken...

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[afbjorklund]

Still broken, in 1.8.1.

💣  Unable to start VM. Please investigate and run 'minikube delete' if possible: creating host: create: precreate: exec: "docker": executable file not found in $PATH

😿  minikube is exiting due to an error. If the above message is not useful, open an issue:
👉  https://github.com/kubernetes/minikube/issues/new/choose

/remove-lifecycle stale

[mazzystr]

$ sudo minikube start --vm-driver=none --container-runtime=cri-o works for me on Fedora 31. I am also reverted to cgroups v1 per docker procedure.

minikube start --container-runtime=cri-o --network-plugin=cni --enable-default-cni --driver=none --alsologtostderr --v=5
....
blah
🏄  Done! kubectl is now configured to use "minikube"
I0424 14:47:29.871172  533397 start.go:454] kubectl: 1.18.1, cluster: 1.18.0 (minor skew: 0)

# kubectl get nodes
NAME    STATUS   ROLES    AGE     VERSION
cube0   Ready    master   5m16s   v1.18.0

# docker version
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Version:            1.9.0
RemoteAPI Version:  1
Go Version:         go1.13.9
OS/Arch:            linux/amd64

[afbjorklund]

Nice that the workaround works, but user should not have to install podman-docker emulator.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[x7upLime]

This issue has to do with how the none driver is designed,
relative to how createHost[pkg/minikube/machine/start.go] is designed:

func createHost(api libmachine.API, cfg *config.ClusterConfig, n *config.Node) (*host.Host, error) {
       // inside this function we have access to the cluster config.
       // ....
	def := registry.Driver(cfg.Driver)
       // ....
	dd, err := def.Config(*cfg, *n)
       // ....
	data, err := json.Marshal(dd)
       // ....

	h, err := api.NewHost(cfg.Driver, data)
       // ...
	if err := timedCreateHost(h, api, cfg.StartHostTimeout); err != nil {
      // .....

timedCreateHost() in turn, calls api.Create, passing the *host.Host, which contains the driver we've *initialized" inside api.NewHost.

The thing is that NewHost (which doesn't have access to the cluster config) does this annoying thing:

	1. def := registry.Driver(drvName)
	2. d := def.Init()
	3. err := json.Unmarshal(rawDriver, d)
	   where rawDriver is the local name for the passed 'data'

After the json.Unmarshal, that's the driver we're ending with.

That's how the none driver looks like:

type Driver struct {
	*drivers.BaseDriver
	*pkgdrivers.CommonDriver
	URL     string
	runtime cruntime.Manager
	exec    command.Runner
}

it relies on runtime and exec, which have to be initialized,
but nothing that is to be initialized would pass the marshal/unmarshal thing in api.NewHost.

So what is happening is that when def.Init() is called inside api.NewHost:

// pkg/minikube/registry/drvs/none/none.go
		Init:     func() drivers.Driver { return none.NewDriver(none.Config{}) },

a none.NewDriver call is issued, with an empty none.Config..

func NewDriver(c Config) *Driver {
	runtime, err := cruntime.New(cruntime.Config{Type: c.ContainerRuntime, Runner: runner})
        // ...
	return &Driver{
               // ...
		runtime: runtime,
	}
}

(makes sense, 'cause we shouldn't know anything about the driver during initialization.. that's what config should be for)

So that we're initializing a cruntime with ac.ContainerRuntime == "",
which defaults to docker: case "", "docker":, inside pkg/minikube/cruntime/cruntime.go
So no container-runtime config should work with the none driver during this step.

And after our unmarshal...
(raw json looking like this:

"{\"IPAddress\":\"\",\"MachineName\":\"minikube\",\"SSHUser\":\"\",\"SSHPort\":0,\"SSHKeyPath\":\"\",\"StorePath\":\"/home/ubuntu/.minikube\",\"SwarmMaster\":false,\"SwarmHost\":\"\",\"SwarmDiscovery\":\"\",\"URL\":\"\"}"

..the runtime field of the struct is not even exported, so there is no way even for the runtime config to be passed on.)

the runtime remains docker.

[x7upLime]

The other drivers doesn't seem to have this issue: none of them seems to rely on initialized stuff after config.

• qemu -- it doesn't have anything that has to be initialized inside the Driver struct
• kvm -- same
• hyperkit -- same
• kic -- it keeps the config passed to NewDriver, inside the driver itself.. and does its "initialization", directly inside (*Driver).Create()

Only hyperkit does an explicit (*Driver).PrecreateCheck()

[afbjorklund]

Thanks for figuring out the registry bug, it was somewhat puzzling.

For reference, the upstream driver looks like this:

func NewDriver(hostName, storePath string) *Driver {
        return &Driver{
                BaseDriver: &drivers.BaseDriver{
                        MachineName: hostName,
                        StorePath:   storePath,
                },
        }
}

With your changes, the none2 is more similar to it.

[afbjorklund]

This would probably fix some similar issues, with the "ssh" driver (generic2)

Since upstream does dynamic flags, it is much simpler:

func NewDriver(hostName, storePath string) drivers.Driver {
        return &Driver{
                EnginePort: engine.DefaultPort,
                BaseDriver: &drivers.BaseDriver{
                        MachineName: hostName,
                        StorePath:   storePath,
                },
        }
}

But in minikube, the Config is linked statically (not runtime)