none driver uses k8s.io/apimachinery/pkg/util/net.ChooseHostInterface() for apiserver IP, causing problems for laptops on DHCP

[maxb]

What Happened?

I'm running minikube using the none driver on an Ubuntu laptop.

minikube's none driver uses k8s.io/apimachinery/pkg/util/net.ChooseHostInterface() to determine the IP address it should use in various contexts, including:

• where to configure as control-plane.minikube.internal in /etc/hosts
• apiserver URL to place in kubeconfig

This is problematic when using a laptop which will move between wifi networks, and/or acquire a different IP address from DHCP after being rebooted or hibernated.

When this happens, existing minikube clusters fail to operate correctly because the apiserver cannot be communicated with.

No configuration option is currently available to override the choice.

Whilst the current behaviour is quite reasonable for people running test labs on servers with stable networking, it's fairly unhelpful for the "personal development laptop" scenario.

I propose that there should be a way to override the IP address used by the none driver to represent the local machine in minikube config ... - it might even be appropriate for it to default to a localhost IP, and only use the current autodetection on user request.

I did come up with a rather arcane workaround, of creating a route to 0.0.0.0/32 (which is not a default route, because of the mask, but looks enough like a default route for k8s.io/apimachinery/pkg/util/net to mistake it as such) on a dummy network interface, with a dummy static IP assigned - but it's quite a hacky approach.

Attach the log file

n/a

Operating System

Ubuntu

Driver

None (Baremetal)

[afbjorklund]

The none driver is not recommended for laptops, so it is mostly used for custom virtual machines (with static IPs)

A parameter to choose the interface (or address) is still reasonable, preferrably it should only listen to a local address.

[klaases]

Hi @maxb, were you able to take a look at @afbjorklund's comment and did it help resolve the situation?

[klaases]

Hi @maxb – is this issue still occurring? Are additional details available? If so, please feel free to re-open the issue by commenting with /reopen. This issue will be closed as additional information was unavailable and some time has passed.

Additional information that may be helpful:

• Whether the issue occurs with the latest minikube release

• The exact minikube start command line used

• Attach the full output of minikube logs, run minikube logs --file=logs.txt to create a log file

Thank you for sharing your experience!

[maxb]

/reopen

Sorry for going quiet on this one - initially I wasn't sure how to usefully and effectively challenge "The none driver is not recommended for laptops", so I took some time to think about it ... and then life got in the way and I never got back to this issue.

I can see the point that the none driver isn't the right starting point for everyone on a laptop - especially now you need to set up cri-dockerd for it to work with Docker.

That said, for the more investigatively minded software developer / sysadmin hybrid, who wants to learn more about Kubernetes infrastructure too, as well as just use it, and is willing to put in a bit more up-front setup work to get more insight into the details, and a better end result development experience --- then I think the none driver on a laptop is a great way to do it.

In particular, I've found it useful that in a none driver scenario:

• There's no difference between the host Docker instance and the minikube internal Docker instance, so there's no need to explicitly transport images between them.
• Since Kubernetes is running directly on the host laptop, you can directly access Kubernetes service IPs from your regular client tools and web browser.

I think it could be reasonable to keep this issue open to track implementing a minikube config parameter that a user could use to configure this IP, rather than having to resort to tricking the autodetection code?

For reference, here is the rather clumsy workaround I came up with:

/etc/systemd/network/kube0.netdev

[NetDev]
Name=kube0
Kind=dummy
```

/etc/systemd/network/kube0.network
```
[Match]
Name=kube0

[Network]
Address=192.0.2.0/32
Address=2001:db8::/128
DNS=10.96.0.10
Domains=~cluster.local
LinkLocalAddressing=no

[Route]
Destination=0.0.0.0
```

Thus both tricking the current autodetection code for selecting the API server IP on the host, but also instructing systemd-resolved based systems to resolve *.cluster.local DNS names via the local Kubernetes DNS service - in this way you can browse directly to Kubernetes services by DNS name in your web browser, which is really quite nice.

</details>

[k8s-ci-robot]

@maxb: Reopened this issue.

In response to this:

/reopen

Sorry for going quiet on this one - initially I wasn't sure how to usefully and effectively challenge "The none driver is not recommended for laptops", so I took some time to think about it ... and then life got in the way and I never got back to this issue.

I can see the point that the none driver isn't the right starting point for everyone on a laptop - especially now you need to set up cri-dockerd for it to work with Docker.

That said, for the more investigatively minded software developer / sysadmin hybrid, who wants to learn more about Kubernetes infrastructure too, as well as just use it, and is willing to put in a bit more up-front setup work to get more insight into the details, and a better end result development experience --- then I think the none driver on a laptop is a great way to do it.

In particular, I've found it useful that in a none driver scenario:

• There's no difference between the host Docker instance and the minikube internal Docker instance, so there's no need to explicitly transport images between them.
• Since Kubernetes is running directly on the host laptop, you can directly access Kubernetes service IPs from your regular client tools and web browser.

I think it could be reasonable to keep this issue open to track implementing a minikube config parameter that a user could use to configure this IP, rather than having to resort to tricking the autodetection code?

For reference, here is the rather clumsy workaround I came up with:

/etc/systemd/network/kube0.netdev

[NetDev]
Name=kube0
Kind=dummy
```

/etc/systemd/network/kube0.network
```
[Match]
Name=kube0

[Network]
Address=192.0.2.0/32
Address=2001:db8::/128
DNS=10.96.0.10
Domains=~cluster.local
LinkLocalAddressing=no

[Route]
Destination=0.0.0.0
```

Thus both tricking the current autodetection code for selecting the API server IP on the host, but also instructing systemd-resolved based systems to resolve *.cluster.local DNS names via the local Kubernetes DNS service - in this way you can browse directly to Kubernetes services by DNS name in your web browser, which is really quite nice.

</details>

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[maxb]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.