starting minikube on arm64 Mac with Parallels #11219
Labels
co/parallels-driver
Parallels driver issues
help wanted
Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
kind/feature
Categorizes issue or PR as related to a new feature.
lifecycle/stale
Denotes an issue or PR has remained open with no activity and has become stale.
os/macos
priority/important-longterm
Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Comments
|
Is // supportedDrivers is a list of supported drivers on Darwin.
var supportedDrivers []string = func() []string {
if runtime.GOARCH == "arm64" {
// on darwin/arm64 only docker and ssh are supported yet
return []string{
Docker,
SSH,
}
}
return []string{
VirtualBox,
Parallels,
VMwareFusion,
HyperKit,
VMware,
Docker,
SSH,
}
}() |
afbjorklund
added
co/parallels-driver
|
This also needs an ISO to be available for arm64, but that is something that is planned. You should be able to start an Ubuntu VM with Parallels, and then use the SSH driver ? If Parallels supports Vagrant (as a provider) you could use that with the Vagrantfile (#10089 (comment)) Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/focal64"
config.vm.network "private_network", ip: "192.168.50.4"
config.vm.provider "virtualbox" do |vb|
vb.cpus = 2
vb.memory = "2048"
# focal64 bug: https://bugs.launchpad.net/cloud-images/+bug/1829625
vb.customize [ "modifyvm", :id, "--uartmode1", "file", File::NULL ]
end
config.vm.provision "shell", inline: <<-SHELL
apt-get update
apt-get install -y conntrack
SHELL
endEDIT: Apparently, you need to use "bento/ubuntu-20.04" for provider "parallels"
Otherwise the regular Ubuntu LTS Server installation ISO and clicking through the wizard should work too. https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.2-live-server-arm64.iso Currently you still have to install the container runtime and some other requirements ( In the future, minikube is supposed to be able to provision those itself (but it is currenly broken) #10883 |
|
atm, only docker driver is supported on M1 |
ilya-zuyev
added
the
priority/backlog
|
blocked on #9228 for the minikube.iso |
spowelljr
added
the
kind/feature
This was referenced Jul 30, 2021
|
We could need some help from Parallels users and developers, to support this VM driver. There are no integration tests for VMware or Parallels, so they rely on community support... |
|
@afbjorklund I am on an M1 + Parallels so am happy to help. Would you mind catching me up a bit on what gaps remain? |
|
@mprimeaux if you can verify if this ISO works (after expanding), then we know we can use it as the base for the new: https://github.com/afbjorklund/minimal-buildroot/releases/download/0.2/boot-arm64.iso.gz (it requires UEFI to boot) |
|
@afbjorklund Thanks for the gz. I created an ISO with the files and booted in Parallels. Here's what I see.
|
|
Not so much sure about "create", it was more like "insert" (CD) ? Just meant that you needed to Like so: https://kb.parallels.com/en/122842 (using the Anyway, it is supposed to show you the GRUB menu first, then boot the kernel, and then dump you at a prompt:
You can login as |
afbjorklund
changed the title
|
@afbjorklund Thanks for the detail. Apologies. Perhaps I misunderstood. The link to the arm64 ISO you provided decompresses to a folder structure that doesn't contain an ISO and so I thought you preferred me to create and ISO using the linked gz as the source for |
|
Ah, maybe it got clever and mounted the ISO rather than just decompressing the file (from .gz). The file structure on the CD looks something like this: But the file is supposed to be used as-is by the VM, it is a bootable Live CD image (just compressed) Here is what it looks like in a more GUI tool like VirtualBox (here booting the boot-amd64.iso version) virtualbox-amd64.mp4 |
|
I created a 'blank' virtual machine and then attempted to attach the compressed ISO as per the Parallels KB article you linked above. Here's the 'play by play' of the failed attempt.
The ISO structure that I have, of course, matches the one you show in your previous reply. I'll dig around more. |
|
You will need to decompress the file.
I think next time I will not compress it π (the kernel and initrd are already compressed) |
|
@afbjorklund Progress :) Boot-1.movNot very exciting but at least grub is coming up. Just hangs after that. I'll let it run a bit more to see if anything 'times out' but not holding my breath. Also, I can confirm that 'CD/DVD' is first in the boot order. |
|
But it seems like something is not working with the boot sequence, that is a bit disappointing but we can wait for the iso. So for some reason, EFI is failing to start Linux - while running in Parallels. |
|
A bit disappointing for sure. Regarding,
Which ISO? |
See #9228 The real minikube.iso (for arm64), currently struggling a bit with getting the amd64 variant built in CI. I think most of the packages have been prepared, so it's more a matter of integrating this new ISO... i.e. split the buildroot config file and the linux config file into two separate ones, for amd64 and arm64 The current ISO uses if you want to continue on the parallels side, the driver needs building for both architectures now and maybe you (or someone else) will be able to find out the issue with the "minimal" iso boot ? We will see what the test results on Apple M1 when running the ISO with Docker HyperKit is. And there is supposed to be HVF hardware support for arm64 too in a QEMU release "soon" |
|
Sounds great! If possible, it would be nice to know if the amd64 UEFI image works when running on Intel ? https://github.com/afbjorklund/minimal-buildroot/releases/download/0.2/boot-amd64.iso.gz As I mentioned, I tested it with VirtualBox (after ticking the "EFI" box*), so I think it works with Parallels too... * this box, I think it is enabled by default in Parallels (says https://kb.parallels.com/en/122735 at least)
I'm not sure yet if we will change both architectures to use GRUB and UEFI, or if we still stick with It should also be possible to keep a hybrid BIOS/UEFI image, if we want to be compatible (for the x86_64). |
|
@afbjorklund I can confirm the boot-amd64.iso.gz works in Parallels 17 on macOS (Intel). As per the KB, I needed to set boot-amd64.movInterestingly, when setting the same boot flag in Parallels 17 on Apple Silicon for the ARM image, I receive the following error, which causes a Technical Data Report to be generated.
VM Log |
spowelljr
added
priority/important-longterm
|
Is there any testing that I can assist with for this issue? |
The qemu2 driver is the supported one (also on M1) |
|
There is a DP of virtualbox emulating arm on m1: https://www.virtualbox.org/wiki/Downloads |
|
As far as I can tell, it still only emulates Intel: https://forums.virtualbox.org/viewtopic.php?f=8&t=107344 It would be more interesting with a vz driver, than looking at the oracle virtualbox or docker hyperkit legacy. |
I don't know who will be working on this in the future, but it is something for the 2023 roadmap to be sure... It needs updating to 2022.02 (possibly 2023.02 ?), and there is a lot of cleanup and deduplication needed as well. |
|
@afbjorklund I'm investigating |
|
/assign |
|
I'm interested in investigating this issue. I deleted minikube check for Darwin+ARM support and replaced 
How can I get logs/errors instead of penguins? My logs for minikube: |
|
Apparently Parallells no longer uses this file for ARM, so it needs a code change in the driver. |
|
@afbjorklund Looks like it's not our case β we run Linux, not macOS inside Parallels. And I see another IPs in that file. |
|
You might want to open another issue for Parallels running on a Linux host. This was for Parallels Desktop for Mac: https://github.com/Parallels/docker-machine-parallels Where the new VMs ended up in https://github.com/machine-drivers/docker-machine-parallels (this is the driver fork that minikube is building from) |
|
Yes, I found that fork and patched it from Which project is responsible for ISO image? |
|
There are (at least) two images, one minikube.iso and one generic boot2docker.iso https://github.com/docker/machine | https://github.com/boot2docker/boot2docker When using minikube, it is always the minikube-machine and the minikube-iso used. https://minikube.sigs.k8s.io/docs/contrib/building/iso/ (based on coreos/buildroot) |
|
Got it, thanks! How can I debug the run process for the image (downloaded or built)? Maybe something is wrong with the Parallels configuration, maybe with the image itself. I can't even iterate by Alt-(1-6). My current state is on the screenshot and minikube logs here |
|
You would have to ask Parallels, where they keep the serial log Currently only QEMU is getting tested (not VMware/Parallels) |
|
I successfully run Minikube in Parallels on Apple Silicone. The main problem for me was the Serial Console in VM β current I'll prepare PRs for both repos: https://github.com/kubernetes/minikube and https://github.com/minikube-machine/machine-driver-parallels Full log: |
|
@afbjorklund Who can help with checking and merging this PR and releasing new version? minikube-machine/machine-driver-parallels#1 It's our currently used fork: |
|
As far as I know, both Parallells and VMware drivers are abandoned upstream? Then again, so is machine itself... Anyone in @kubernetes/minikube-maintainers |
|
Do you propose to fix it differently? Like not using |
|
No, patching is fine (I can still merge it)... Upstream seems to only have added a warning:
|
|
I don't understand why they added this warning because it works fine, except this bug with Serial Console and no Do you need a PR to Anyway full diff if need |
|
Well, there is no docker-machine for arm64 and there is no boot2docker.iso for arm64 either? But as long as you recompile those for the other architecture, then it should still work I think. i.e. it works OK with the qemu driver (used boot2docker-debian for the OS) |
Please do a PR, and confirm that it works for your local testing (since there is no Parallells in CI) |
|
As I understand, it's not exactly And despite the word
Used ISO: I'll create PR, thank you for help with it! |
|
@afbjorklund |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce the issue:
1.β― minikube start
π minikube v1.19.0 on Darwin 11.2.3 (arm64)
β¨ Automatically selected the parallels driver
β Exiting due to DRV_UNSUPPORTED_OS: The driver 'parallels' is not supported on darwin/arm64
2.
3.
Full output of failed command:
β― minikube start --driver=parallels
π minikube v1.19.0 on Darwin 11.2.3 (arm64)
β¨ Using the parallels driver based on user configuration
β Exiting due to DRV_UNSUPPORTED_OS: The driver 'parallels' is not supported on darwin/arm64
Full output of
minikube startcommand used, if not already included:Optional: Full output of
minikube logscommand:Installation logs:
β― brew install minikube
==> Downloading https://ghcr.io/v2/homebrew/core/minikube/manifests/1.19.0
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/minikube/blobs/sha256:8704e50e9352938
==> Downloading from https://pkg-containers-az.githubusercontent.com/ghcr1/blobs/sha25
######################################################################## 100.0%
==> Pouring minikube--1.19.0.arm64_big_sur.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
/opt/homebrew/share/zsh/site-functions
==> Summary
πΊ /opt/homebrew/Cellar/minikube/1.19.0: 8 files, 58.8MB
The text was updated successfully, but these errors were encountered: