Local kind cluster ip change might cause issues

[jorgemoralespou]

Describe the bug

I have a local setup with a local educates kind cluster as well as the resolver, registry and 3 mirrors.
All of the educates, registry and mirrors are attached to the kind network.
When Docker Desktop is restarted, the IP assigned to these containers might change because they depend on the startup time, as all are marked as Restart: Always. So, if you have a cluster that was created and has an internal ip of 172.18.0.2 it might start now and get a different ip (e.g. 172.18.0.3) and then pods will be seen as Running but will not be marked as running. I noticed this in k9s as pods show up in red and nothing works.

To check the ip of the container do:

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' educates-control-plane

To see the IP that was configured in the previous run do:

docker exec educates-control-plane grep "server:" /etc/kubernetes/kubelet.conf

If there's an ip mismatch then you have hit this problem. You can see what container has squatted the ip by doing:

docker network inspect kind -f '{{range .Containers}}{{.Name}} has IP {{.IPv4Address}}{{println}}{{end}}'

To fix the issue you need to:

disconnect the squatter container:

# Disconnect the container currently holding the IP
docker network disconnect kind <squatter-container>

Fix the educates-control-plane container ip:

# Example only - replace with your OLD IP found in Step 2
docker network disconnect kind educates-control-plane
docker network connect --ip 172.18.0.2 kind educates-control-plane
docker restart educates-control-plane

Connect back the squatter container:

# Disconnect the container currently holding the IP
docker network connect kind <squatter-container>

Solution proposed

Add a specific and high ip to the educates-control-plane container when run. (e.g.
docker run -d
--name
--network kind
--ip 172.18.0.200

This needs to be done from within the Educates CLI. We need to verify the subnet first. If your kind network is 172.18.0.0/16, then 172.18.0.200 is perfectly safe

Registry and mirror containers IP are not a problem (AFAICT) but we could implement the same on these, just to make sure that everything is consistent.

### Additional information

_No response_

[GrahamDumpleton]

Is there perhaps some way to rely on internal Docker DNS and fact that container name is available as a hostname?

I can't remember exactly what the reason for the bridge network was, but is the real problem that the docker container IP for registry is added to a EndPointSlice in default namespace. That is what becomes out of date if the IP changes and thus cannot be routed. If instead of mapping to IP, could use aliasing to have it refer to Docker DNS hostname for the container in some way when creating headless Service in default namespace.

But then, I may have tried that long ago and why ended up with mess have now, as couldn't find anything else that worked.

[jorgemoralespou]

As it seems, you can not set a fixed ip for the control-plane on kind as it's not configurable, so the solution boils down to set a fixed ip for every other container that is attached to the kind network, so that they don't take the control-plane's ip.

This is a classic "race condition" problem in Docker networking. Since Kind generates TLS certificates based on the IP address assigned at creation (e.g., 172.18.0.2), the cluster breaks if the container restarts and receives a different IP (e.g., 172.18.0.4) because another container grabbed the original IP first.

You cannot natively set a fixed nodeIP in the kind-config.yaml because Kind relies on the Docker daemon to assign IPs. However, you can solve this by forcing all "neighbor" containers to use specific, non-conflicting IPs, effectively reserving the lower IP range for Kind.

Here is the step-by-step solution to fix the race condition.
The Strategy: "High-IP" Reservation

Since you cannot force Kind to take a static IP, you must force every other container on that network to take a specific, "high" IP address (e.g., .200, .201). This leaves the default lower IPs (like .2 or .3) permanently available for Kind, regardless of startup order.