virtualbox: change default `host-only-cidr` #12811

nbusseneau · 2021-10-28T16:01:37Z

We change the default --host-only-cidr to 192.168.59.1/24 in order to be compatible with the new default host-only networking restrictions implemented in VirtualBox 6.1.28.

This fixes access denied errors on minikube start when using VirtualBox >= 6.1.28:

VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available)
VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp

More details:

VirtualBox 6.1.28 introduced new restrictions on host-only networking: https://www.virtualbox.org/wiki/Changelog-6.1#v28
Manual: https://www.virtualbox.org/manual/ch06.html#network_hostonly

On Linux, Mac OS X and Solaris Oracle VM VirtualBox will only allow IP addresses in 192.68.56.0/21 range to be assigned to host-only adapters. For IPv6 only link-local addresses are allowed. If other ranges are desired, they can be enabled by creating /etc/vbox/networks.conf and specifying allowed ranges there. For example, to allow 10.0.0.0/8 and 192.168.0.0/16 IPv4 ranges as well as 2001::/64 range put the following lines into /etc/vbox/networks.conf:
 * 10.0.0.0/8 192.168.0.0/16
 * 2001::/64
Lines starting with the hash # are ignored. Next example allows any addresses, effectively disabling range control:
 * 0.0.0.0/0 ::/0

These new restrictions manifest in the form of the following issue on minikube start due to the default --host-only-cidr used by the VirtualBox driver being 192.168.99.1/24:

# minikube start --driver=virtualbox
😄  minikube v1.23.2 on Ubuntu 18.04
✨  Using the virtualbox driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🔥  Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🔥  Deleting "minikube" in virtualbox ...
🤦  StartHost failed, but will try again: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.99.1 --netmask 255.255.255.0 failed:
VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available)
VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp

🔥  Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
😿  Failed to start virtualbox VM. Running "minikube delete" may fix it: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet1 --ip 192.168.99.1 --netmask 255.255.255.0 failed:
VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available)
VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp

❌  Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet1 --ip 192.168.99.1 --netmask 255.255.255.0 failed:
VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available)
VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

While the above is the primary error, other errors will be reported if the adapter already exists (this happens when the adapter was created before updating to VirtualBox 6.1.28, on a version of VirtualBox without the new restrictions). Some examples:

❌ Exiting due to IF_VBOX_NOT_VISIBLE: Failed to start host: creating host: create: creating: Error setting up host only network on machine start: The host-only adapter we just created is not visible. This is a well known VirtualBox bug. You might want to uninstall it and reinstall at least version 5.0.12 that is is supposed to fix this issue

❌  minikube is unable to connect to the VM: dial tcp 192.168.99.112:22: i/o timeout

	This is likely due to one of two reasons:

	- VPN or firewall interference
	- virtualbox network configuration issue

	Suggested workarounds:

	- Disable your local VPN or firewall software
	- Configure your local VPN or firewall to allow access to 192.168.99.112
	- Restart or reinstall virtualbox
	- Use an alternative --vm-driver
	- Use --force to override this connectivity check
	

❌  Exiting due to GUEST_PROVISION: Failed to validate network: dial tcp 192.168.99.112:22: i/o timeout

When switching to a valid CIDR, minikube start works as usual:

# minikube start --driver=virtualbox --host-only-cidr "192.168.59.1/24"
😄  minikube v1.23.2 on Ubuntu 18.04
✨  Using the virtualbox driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🔥  Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎  Verifying Kubernetes components...
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

fixes #12765

k8s-ci-robot · 2021-10-28T16:01:45Z

Welcome @nbusseneau!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2021-10-28T16:01:46Z

Hi @nbusseneau. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

minikube-bot · 2021-10-28T16:03:04Z

Can one of the admins verify this patch?

nbusseneau · 2021-10-28T16:13:07Z

Output of minikube start using a local build:

# ./out/minikube start --driver=virtualbox
😄  minikube v1.23.2 on Ubuntu 18.04
✨  Using the virtualbox driver based on user configuration
💿  Downloading VM boot image ...
    > minikube-v1.23.1-1633115168...: 65 B / 65 B [----------] 100.00% ? p/s 0s
    > minikube-v1.23.1-1633115168...: 225.55 MiB / 225.55 MiB  100.00% 5.30 MiB
👍  Starting control plane node minikube in cluster minikube
🔥  Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🔎  Verifying Kubernetes components...
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Behaviour is now identical to passing --host-only-cidr "192.168.59.1/24" as above.

@ reviewers: I'm a bit confused as I ran the commands as instructed in https://github.com/kubernetes/community/blob/b9266ed1a2365d4f83b559c521a2b6ee73bbe20f/contributors/guide/pull-requests.md#run-local-verifications, however only make test did run properly:

# make verify
make: *** No rule to make target 'verify'. Stop.
# make test
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.23.2 -X k8s.io/minikube/pkg/version.isoVersion=v1.23.1-1633115168-12081 -X k8s.io/minikube/pkg/version.gitCommitID="c2385ebfbc10f1eff07769f76422b1c99c4786c1" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v5" ./test.sh
= make lint =============================================================
ok
= go mod ================================================================
ok
= boilerplate ===========================================================
ok
= schema_check ==========================================================
ok
= go test ===============================================================
ok  	k8s.io/minikube/cmd/minikube/cmd	0.246s	coverage: 19.1% of statements
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.106s	coverage: 21.0% of statements
ok  	k8s.io/minikube/pkg/addons	0.113s	coverage: 25.2% of statements
ok  	k8s.io/minikube/pkg/drivers	0.018s	coverage: 18.9% of statements
ok  	k8s.io/minikube/pkg/drivers/hyperkit	0.034s	coverage: 77.3% of statements
ok  	k8s.io/minikube/pkg/drivers/kic/oci	6.096s	coverage: 20.0% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.019s	coverage: 4.1% of statements
ok  	k8s.io/minikube/pkg/minikube/assets	0.081s	coverage: 20.5% of statements
ok  	k8s.io/minikube/pkg/minikube/audit	0.023s	coverage: 72.8% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	0.798s	coverage: 48.1% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil	0.108s	coverage: 61.6% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/bsutil/ktmpl	0.052s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/images	0.016s	coverage: 97.2% of statements
ok  	k8s.io/minikube/pkg/minikube/cluster	0.104s	coverage: 13.3% of statements
ok  	k8s.io/minikube/pkg/minikube/command	0.103s	coverage: 12.4% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.076s	coverage: 71.9% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.073s	coverage: 28.8% of statements
ok  	k8s.io/minikube/pkg/minikube/docker	0.087s	coverage: 20.8% of statements
ok  	k8s.io/minikube/pkg/minikube/download	1.045s	coverage: 25.3% of statements
ok  	k8s.io/minikube/pkg/minikube/driver	0.016s	coverage: 49.0% of statements
ok  	k8s.io/minikube/pkg/minikube/driver/auxdriver	0.041s	coverage: 19.0% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.035s	coverage: 59.5% of statements
ok  	k8s.io/minikube/pkg/minikube/image	0.019s	coverage: 5.1% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.028s	coverage: 81.5% of statements
ok  	k8s.io/minikube/pkg/minikube/localpath	0.015s	coverage: 47.4% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.078s	coverage: 0.8% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	0.138s	coverage: 20.4% of statements
ok  	k8s.io/minikube/pkg/minikube/mustload	0.074s	coverage: 10.5% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.098s	coverage: 83.3% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.097s	coverage: 66.4% of statements
ok  	k8s.io/minikube/pkg/minikube/out/register	0.028s	coverage: 55.4% of statements
ok  	k8s.io/minikube/pkg/minikube/perf	4.026s	coverage: 21.2% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.101s	coverage: 68.7% of statements
ok  	k8s.io/minikube/pkg/minikube/reason	0.011s	coverage: 70.0% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.014s	coverage: 77.0% of statements
ok  	k8s.io/minikube/pkg/minikube/registry/drvs/docker	0.066s	coverage: 20.2% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.033s	coverage: 84.2% of statements
ok  	k8s.io/minikube/pkg/minikube/shell	0.026s	coverage: 94.4% of statements
ok  	k8s.io/minikube/pkg/minikube/storageclass	0.021s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/style	0.015s	coverage: 100.0% of statements
ok  	k8s.io/minikube/pkg/minikube/sysinit	0.045s	coverage: 4.5% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.052s	coverage: 45.5% of statements
ok  	k8s.io/minikube/pkg/minikube/tunnel	1.452s	coverage: 63.8% of statements
ok  	k8s.io/minikube/pkg/util	0.543s	coverage: 75.7% of statements
ok  	k8s.io/minikube/pkg/util/lock	0.003s	coverage: 22.2% of statements
ok  	k8s.io/minikube/pkg/util/retry	0.002s	coverage: 0.0% of statements
ok
# make test-integration
make: *** No rule to make target 'test-integration'. Stop.

So I instead ran make integration as instructed in https://minikube.sigs.k8s.io/docs/contrib/testing/. Is it perhaps the case that the contributing guide above has not updated / should point to the contribution guide on the website instead? 🤔

spowelljr · 2021-10-28T16:25:40Z

/ok-to-test

minikube-pr-bot · 2021-10-28T17:17:15Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 48.1s    | 47.4s               |
| enable ingress | 32.2s    | 31.9s               |
+----------------+----------+---------------------+

Times for minikube start: 49.2s 46.8s 47.8s 48.1s 48.4s
Times for minikube (PR 12811) start: 47.8s 46.5s 48.1s 47.8s 46.9s

Times for minikube ingress: 32.3s 32.8s 31.3s 32.3s 32.2s
Times for minikube (PR 12811) ingress: 33.8s 31.3s 31.9s 31.2s 31.3s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 21.7s    | 21.6s               |
| enable ingress | 31.0s    | 34.1s               |
+----------------+----------+---------------------+

Times for minikube (PR 12811) start: 21.0s 21.2s 22.5s 22.2s 21.3s
Times for minikube start: 21.7s 21.2s 22.0s 22.3s 21.4s

Times for minikube ingress: 27.9s 27.9s 34.4s 28.5s 36.4s
Times for minikube (PR 12811) ingress: 35.4s 35.9s 28.4s 35.4s 35.5s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 37.4s    | 41.4s               |
| enable ingress | 34.7s    | 33.6s               |
+----------------+----------+---------------------+

Times for minikube start: 26.6s 43.2s 29.5s 43.9s 43.7s
Times for minikube (PR 12811) start: 43.1s 43.5s 39.8s 43.9s 36.8s

Times for minikube ingress: 34.9s 33.9s 33.9s 33.9s 36.9s
Times for minikube (PR 12811) ingress: 33.4s 19.4s 32.4s 41.9s 40.6s

minikube-pr-bot · 2021-10-28T18:43:15Z

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
KVM_Linux	TestStoppedBinaryUpgrade/MinikubeLogs (gopogh)	0.75 (chart)
KVM_Linux	TestStoppedBinaryUpgrade/Upgrade (gopogh)	0.75 (chart)
Docker_Linux_containerd	TestStartStop/group/embed-certs/serial/AddonExistsAfterStop (gopogh)	1.33 (chart)
Docker_Linux_containerd	TestStartStop/group/embed-certs/serial/EnableAddonWhileActive (gopogh)	2.00 (chart)
KVM_Linux	TestRunningBinaryUpgrade (gopogh)	2.24 (chart)
KVM_Linux_containerd	TestPause/serial/PauseAgain (gopogh)	4.35 (chart)
Docker_Linux	TestStartStop/group/old-k8s-version/serial/Pause (gopogh)	6.67 (chart)
Docker_Linux	TestFunctional/serial/ComponentHealth (gopogh)	8.00 (chart)
Docker_Windows	TestMountStart/serial/StartWithMountFirst (gopogh)	69.11 (chart)
Docker_Windows	TestMountStart/serial/StartWithMountSecond (gopogh)	69.11 (chart)
Docker_Windows	TestMountStart/serial/VerifyMountFirst (gopogh)	69.11 (chart)
Docker_Windows	TestMountStart/serial/VerifyMountPostDelete (gopogh)	69.11 (chart)
Docker_Windows	TestMountStart/serial/VerifyMountSecond (gopogh)	69.11 (chart)
Docker_Windows	TestNetworkPlugins/group/kindnet/DNS (gopogh)	69.47 (chart)
Docker_Windows	TestNetworkPlugins/group/custom-weave/Start (gopogh)	70.21 (chart)
Docker_Windows	TestNetworkPlugins/group/enable-default-cni/DNS (gopogh)	74.82 (chart)
Docker_Windows	TestNetworkPlugins/group/calico/Start (gopogh)	78.72 (chart)
Docker_Windows	TestNetworkPlugins/group/kubenet/DNS (gopogh)	80.29 (chart)
Docker_Windows	TestNetworkPlugins/group/bridge/DNS (gopogh)	80.43 (chart)
Docker_Linux_containerd	TestScheduledStopUnix (gopogh)	100.00 (chart)
Docker_Windows	TestCertOptions (gopogh)	100.00 (chart)
Docker_Windows	TestMountStart/serial/RestartStopped (gopogh)	100.00 (chart)
Docker_Windows	TestMountStart/serial/Stop (gopogh)	100.00 (chart)
Docker_Windows	TestMountStart/serial/VerifyMountPostStop (gopogh)	100.00 (chart)
Docker_Windows	TestPause/serial/VerifyDeletedResources (gopogh)	100.00 (chart)

To see the flake rates of all tests by environment, click here.

nbusseneau · 2021-10-28T19:40:23Z

Question: do you want me to add a note in the virtualbox.md documentation with some more details about the host CIDR restrictions, so that users do not unknowingly pass in a --host-only-cidr value without knowing they should first allow it in VirtualBox configuration?

I suppose this would avoid some users reporting issues in here thinking it has to do with Minikube when it is not the case.

spowelljr · 2021-11-01T16:52:03Z

Hi @nbusseneau, how does this change work with older versions of VirtualBox? ie. Will this break working with older versions or will it still work fine?

sharifelgamal · 2021-11-01T21:24:27Z

I can confirm this change works with Virtualbox 6.1.8, which I think is sufficiently old.

sharifelgamal · 2021-11-01T23:40:35Z

Question: do you want me to add a note in the virtualbox.md documentation with some more details about the host CIDR restrictions, so that users do not unknowingly pass in a --host-only-cidr value without knowing they should first allow it in VirtualBox configuration?

I suppose this would avoid some users reporting issues in here thinking it has to do with Minikube when it is not the case.

This would be great yeah. Extra documentation is always a good idea.

We change the default `--host-only-cidr` to `192.168.59.1/24` in order to be compatible with the new default host-only networking restrictions implemented in VirtualBox 6.1.28. This fixes access denied errors on `minikube start` when using VirtualBox >= 6.1.28: ``` VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available) VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp ``` More details: VirtualBox 6.1.28 introduced new restrictions on host-only networking: https://www.virtualbox.org/wiki/Changelog-6.1#v28 Manual: https://www.virtualbox.org/manual/ch06.html#network_hostonly > On Linux, Mac OS X and Solaris Oracle VM VirtualBox will only allow IP > addresses in 192.68.56.0/21 range to be assigned to host-only > adapters. For IPv6 only link-local addresses are allowed. If other > ranges are desired, they can be enabled by creating > /etc/vbox/networks.conf and specifying allowed ranges there. For > example, to allow 10.0.0.0/8 and 192.168.0.0/16 IPv4 ranges as well as > 2001::/64 range put the following lines into /etc/vbox/networks.conf: > > * 10.0.0.0/8 192.168.0.0/16 > * 2001::/64 > > Lines starting with the hash # are ignored. Next example allows any > addresses, effectively disabling range control: > > * 0.0.0.0/0 ::/0 These new restrictions manifest in the form of the following issue on `minikube start` due to the default `--host-only-cidr` used by the VirtualBox driver being `192.168.99.1/24`: ```console 😄 minikube v1.23.2 on Ubuntu 18.04 ✨ Using the virtualbox driver based on user configuration 👍 Starting control plane node minikube in cluster minikube 🔥 Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🔥 Deleting "minikube" in virtualbox ... 🤦 StartHost failed, but will try again: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.99.1 --netmask 255.255.255.0 failed: VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available) VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp 🔥 Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 😿 Failed to start virtualbox VM. Running "minikube delete" may fix it: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet1 --ip 192.168.99.1 --netmask 255.255.255.0 failed: VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available) VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp ❌ Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: Error setting up host only network on machine start: /usr/bin/VBoxManage hostonlyif ipconfig vboxnet1 --ip 192.168.99.1 --netmask 255.255.255.0 failed: VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied (extended info not available) VBoxManage: error: Context: "EnableStaticIPConfig(Bstr(pszIp).raw(), Bstr(pszNetmask).raw())" at line 242 of file VBoxManageHostonly.cpp ╭───────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────────╯ ``` While the above is the primary error, other errors will be reported if the adapter already exists (this happens when the adapter was created before updating to VirtualBox 6.1.28, on a version of VirtualBox without the new restrictions). Some examples: ``` ❌ Exiting due to IF_VBOX_NOT_VISIBLE: Failed to start host: creating host: create: creating: Error setting up host only network on machine start: The host-only adapter we just created is not visible. This is a well known VirtualBox bug. You might want to uninstall it and reinstall at least version 5.0.12 that is is supposed to fix this issue ``` ``` ❌ minikube is unable to connect to the VM: dial tcp 192.168.99.112:22: i/o timeout This is likely due to one of two reasons: - VPN or firewall interference - virtualbox network configuration issue Suggested workarounds: - Disable your local VPN or firewall software - Configure your local VPN or firewall to allow access to 192.168.99.112 - Restart or reinstall virtualbox - Use an alternative --vm-driver - Use --force to override this connectivity check ❌ Exiting due to GUEST_PROVISION: Failed to validate network: dial tcp 192.168.99.112:22: i/o timeout ``` When switching to a valid CIDR, `minikube start` works as usual: ```console 😄 minikube v1.23.2 on Ubuntu 18.04 ✨ Using the virtualbox driver based on user configuration 👍 Starting control plane node minikube in cluster minikube 🔥 Creating virtualbox VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🐳 Preparing Kubernetes v1.22.2 on Docker 20.10.8 ... ▪ Generating certificates and keys ... ▪ Booting up control plane ... ▪ Configuring RBAC rules ... ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5 🔎 Verifying Kubernetes components... 🌟 Enabled addons: storage-provisioner, default-storageclass 🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default ``` Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com>

nbusseneau · 2021-11-02T14:36:33Z

Hi @nbusseneau, how does this change work with older versions of VirtualBox? ie. Will this break working with older versions or will it still work fine?

This should have no impact on older VirtualBox versions as we only change the default CIDR used by Minikube so that it is compatible with new restrictions in 6.1.28, and there were no such restrictions in prior versions. So there is no "restriction conflict" and no need to handle a different default value per VirtualBox version, in case that was your question ^^

I can confirm at least that my local build of Minikube with the PR changes using this new CIDR by default works fine after downgrading to VirtualBox 6.0.24.

This would be great yeah. Extra documentation is always a good idea.

Added, PTAL :)

nbusseneau · 2021-11-02T14:43:35Z

Question: are the CI issues expected? Did the CIDR change mess up some kind of test expectation which I'd missed?

minikube-pr-bot · 2021-11-02T15:26:43Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 47.4s    | 47.3s               |
| enable ingress | 30.9s    | 31.2s               |
+----------------+----------+---------------------+

Times for minikube ingress: 30.3s 30.8s 30.7s 30.8s 31.8s
Times for minikube (PR 12811) ingress: 32.2s 31.3s 30.3s 30.8s 31.3s

Times for minikube start: 49.8s 47.0s 47.7s 46.3s 46.2s
Times for minikube (PR 12811) start: 46.7s 48.8s 46.6s 46.8s 47.6s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 21.8s    | 21.3s               |
| enable ingress | 33.5s    | 31.1s               |
+----------------+----------+---------------------+

Times for minikube start: 21.8s 22.2s 22.0s 22.1s 21.1s
Times for minikube (PR 12811) start: 20.5s 22.0s 20.6s 21.0s 22.2s

Times for minikube ingress: 35.9s 26.4s 34.4s 35.9s 34.9s
Times for minikube (PR 12811) ingress: 35.9s 28.0s 27.9s 27.9s 35.9s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 12811) |
+----------------+----------+---------------------+
| minikube start | 40.5s    | 43.6s               |
| enable ingress | 42.0s    | 39.6s               |
+----------------+----------+---------------------+

Times for minikube (PR 12811) start: 43.5s 42.9s 44.3s 43.6s 43.7s
Times for minikube start: 26.7s 43.2s 44.6s 44.5s 43.3s

Times for minikube ingress: 34.9s 75.9s 32.9s 36.9s 29.4s
Times for minikube (PR 12811) ingress: 36.9s 66.9s 23.4s 33.9s 36.9s

site/content/en/docs/drivers/virtualbox.md

minikube-pr-bot · 2021-11-02T16:51:28Z

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
Docker_Linux_containerd	TestStartStop/group/embed-certs/serial/Pause (gopogh)	3.42 (chart)
Docker_Linux_containerd	TestScheduledStopUnix (gopogh)	100.00 (chart)

To see the flake rates of all tests by environment, click here.

sharifelgamal · 2021-11-02T18:56:37Z

Question: are the CI issues expected? Did the CIDR change mess up some kind of test expectation which I'd missed?

Yeah, the CI failures are test flakes, not due to this change at all.

k8s-ci-robot · 2021-11-02T20:38:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nbusseneau, sharifelgamal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sharifelgamal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 28, 2021

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 28, 2021

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 28, 2021

k8s-ci-robot requested review from prezha and spowelljr October 28, 2021 16:02

nbusseneau force-pushed the pr/fix-virtualbox-ip branch from fdc3ec8 to c2385eb Compare October 28, 2021 16:03

k8s-ci-robot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Oct 28, 2021

nbusseneau mentioned this pull request Oct 28, 2021

VBoxManage: error: Code E_ACCESSDENIED (0x80070005) - Access denied #12765

Closed

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 28, 2021

ChristopherHX mentioned this pull request Nov 2, 2021

> macOS CI broken VirtualBox >= 6.1.28 nektos/act#858

Closed

nbusseneau force-pushed the pr/fix-virtualbox-ip branch from c2385eb to 02a43c5 Compare November 2, 2021 14:36

nbusseneau commented Nov 2, 2021

View reviewed changes

site/content/en/docs/drivers/virtualbox.md Show resolved Hide resolved

sharifelgamal approved these changes Nov 2, 2021

View reviewed changes

sharifelgamal merged commit 3342d3a into kubernetes:master Nov 2, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 2, 2021

medyagh mentioned this pull request Nov 3, 2021

Minikube wont start 6.2 VirtualBox #12850

Closed

This was referenced Nov 15, 2021

minikube start with virtual box not working after upgrading Mac OS Monterey #12943

Closed

Windows 8: VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component MachineWrap, interface IMachine #12889

Closed

nshalman mentioned this pull request Nov 30, 2021

[Issue #112] OS X Monterey and vbox 6.1.28 compatibility changes to Vagrantfile tinkerbell/playground#114

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

virtualbox: change default `host-only-cidr` #12811

virtualbox: change default `host-only-cidr` #12811

nbusseneau commented Oct 28, 2021 •

edited

Loading

k8s-ci-robot commented Oct 28, 2021

k8s-ci-robot commented Oct 28, 2021

minikube-bot commented Oct 28, 2021

nbusseneau commented Oct 28, 2021 •

edited

Loading

spowelljr commented Oct 28, 2021

minikube-pr-bot commented Oct 28, 2021

minikube-pr-bot commented Oct 28, 2021

nbusseneau commented Oct 28, 2021 •

edited

Loading

spowelljr commented Nov 1, 2021 •

edited

Loading

sharifelgamal commented Nov 1, 2021 •

edited

Loading

sharifelgamal commented Nov 1, 2021

nbusseneau commented Nov 2, 2021

nbusseneau commented Nov 2, 2021

minikube-pr-bot commented Nov 2, 2021

minikube-pr-bot commented Nov 2, 2021

sharifelgamal commented Nov 2, 2021

k8s-ci-robot commented Nov 2, 2021

virtualbox: change default host-only-cidr #12811

virtualbox: change default host-only-cidr #12811

Conversation

nbusseneau commented Oct 28, 2021 • edited Loading

k8s-ci-robot commented Oct 28, 2021

k8s-ci-robot commented Oct 28, 2021

minikube-bot commented Oct 28, 2021

nbusseneau commented Oct 28, 2021 • edited Loading

spowelljr commented Oct 28, 2021

minikube-pr-bot commented Oct 28, 2021

minikube-pr-bot commented Oct 28, 2021

nbusseneau commented Oct 28, 2021 • edited Loading

spowelljr commented Nov 1, 2021 • edited Loading

sharifelgamal commented Nov 1, 2021 • edited Loading

sharifelgamal commented Nov 1, 2021

nbusseneau commented Nov 2, 2021

nbusseneau commented Nov 2, 2021

minikube-pr-bot commented Nov 2, 2021

minikube-pr-bot commented Nov 2, 2021

sharifelgamal commented Nov 2, 2021

k8s-ci-robot commented Nov 2, 2021

virtualbox: change default `host-only-cidr` #12811

virtualbox: change default `host-only-cidr` #12811

nbusseneau commented Oct 28, 2021 •

edited

Loading

nbusseneau commented Oct 28, 2021 •

edited

Loading

nbusseneau commented Oct 28, 2021 •

edited

Loading

spowelljr commented Nov 1, 2021 •

edited

Loading

sharifelgamal commented Nov 1, 2021 •

edited

Loading