minikube support?

[jkroepke]

I would like to know if its possible to connect to the kubernetes pod and service network the same way as to the docker network?

Scope: minikube on Docker for Desktop

[gregnr]

Hey @jkroepke! Just to clarify, are you running minikube specifically, or Docker Desktop's Kubernetes option:

I haven't attempted docker-mac-net-connect with either of these yet, but Docker Desktop's Kubernetes option will most likely be the easiest to hook into this tool.

[jkroepke]

I'm talking about minikube. I'm aware of the Docker Desktop option, but I use the profile functionallity of minikube.

By default, minikube runs a dedicated docker daemon inside a container which is accessable by minikube docker-env

[gregnr]

Got it, thanks for confirming. I will take a look at this when I get a moment. In the mean time, can you confirm if you've tried this yourself? You will need a /var/run/docker.sock available on the macOS side. Is this automatically created when you install minikube? What does minikube docker-env output for you?

[jkroepke]

Hi @gregnr

on my machine, minikube runs on top of an existing docker daemon. For example, minikube will re-use the Docker Daemon from Docker Desktop (mounted as /var/run/docker.sock) and will create one container.

This minikube container will spin up a dedicated Docker Daemon (real docker in docker) and a Kubernetes environment

% minikube docker-env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://127.0.0.1:53936"
export DOCKER_CERT_PATH="/Users/jok/.minikube/certs"
export MINIKUBE_ACTIVE_DOCKERD="minikube"

# To point your shell to minikube's docker-daemon, run:
# eval $(minikube -p minikube docker-env)

An alternative approach would be k8s-mac-net-connect where a Kubernetes Pod is used instead a docker container.

Additionally, I may need multiple instances of docker-mac-net-connect (one for docker, one for kubernetes) however at this moment, only one instance is supported because the port is hardcoded.

[gregnr]

@jkroepke you're right, we should allow port to be configurable. Adding this to #24.

Thanks for the explanation. Understood about dind. Out of curiosity, do you know what would happen if you didn't have Docker Desktop installed?

After we make the Docker daemon API endpoint configurable (#24), in theory you could then point it to your dind endpoint (ie. tcp://127.0.0.1:53936 above) to get it to work. But I imagine we may face some more complications with dind on the networking side (how does dind create bridge networks, etc?).

Re: k8s-mac-net-connect - this could be an option, but I imagine lots of the logic would be duplicated, especially if k8s is using Docker as its container runtime. We could abstract common code somewhere else between the projects, but this adds extra maintenance that may not be worth introducing at this point. If k8s isn't using Docker as its container runtime (ie. using containerd), the logic will change a bit more since we can no longer watch for networking changes via Docker daemon, etc.

[jkroepke]

The idea of k8s-mac-net-connect was to expose the Kubernetes Pod CIDR and Service CIDR and not the underlaying docker networks.

The Pod/Service CIDRs are static.

Minikube is only of one of very few distributione hat using docker as runtime. Containerd ist most common here.

If docker desktop is not avalible, minikube is setup a hyperkit virtual machine with docker daemon.

https://minikube.sigs.k8s.io/docs/drivers/

[mingmingshiliyu]

@jkroepke you're right, we should allow port to be configurable. Adding this to #24.

Thanks for the explanation. Understood about dind. Out of curiosity, do you know what would happen if you didn't have Docker Desktop installed?

After we make the Docker daemon API endpoint configurable (#24), in theory you could then point it to your dind endpoint (ie. tcp://127.0.0.1:53936 above) to get it to work. But I imagine we may face some more complications with dind on the networking side (how does dind create bridge networks, etc?).

Re: k8s-mac-net-connect - this could be an option, but I imagine lots of the logic would be duplicated, especially if k8s is using Docker as its container runtime. We could abstract common code somewhere else between the projects, but this adds extra maintenance that may not be worth introducing at this point. If k8s isn't using Docker as its container runtime (ie. using containerd), the logic will change a bit more since we can no longer watch for networking changes via Docker daemon, etc.

can't connect to the k8s cluster running on docker-desktop on Mac,I tried to ping node ip with kubectl get node -o wide

[RafalSkolasinski]

Not sure if exactly the same as needs of original poster, but I was struggling to get a nice access to the LoadBalancers of K8s services after moving to Mac. Everything used to work like charm on Linux using MetalLB.

This project was a god-sent and solved most of my issues. I wrote down my findings as a gist here - sharing hoping that others find it useful.