lima fails to start on a fresh OS, fresh install on M3 Pro: `Error: Image at 0013FD41000 start failed: Not Found` ... `Instruction abort: Translation fault, first level`

[nbibler]

Tip

(EDIT by @AkihiroSuda)

Workaround

Specify --vm-type=vz in limactl create

limactl create --vm-type=vz ...

Proper fix

Expected to be fixed in Lima v0.19.1

• qemu, aarch64: use EDK2 from "Firmware/edk2 20231213 patches" (needed for QEMU v8.2.0) #2061

---

Description

Using Homebrew to install lima on MacOS 14.1.1 (23B2082), Apple M3 Pro, the installation completes successfully, but starting up lima freezes at "INFO[0000] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"" and eventually fails. The log files seem to indicate some type of build error: "Error: Image at 0013FD41000 start failed: Not Found", "Instruction abort: Translation fault, first level".

$ brew info lima
==> lima: stable 0.18.0 (bottled), HEAD
Linux virtual machines
https://github.com/lima-vm/lima
/opt/homebrew/Cellar/lima/0.18.0 (104 files, 169.8MB) *
  Poured from bottle using the formulae.brew.sh API on 2023-11-09 at 16:21:16
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/l/lima.rb
License: Apache-2.0

$ brew install lima

$ limactl create --tty=false
INFO[0020] Attempting to download the image              arch=aarch64 digest="sha256:78d35a2f551d281912ee7e5202660d94d713aa1b5de86a17e261328cc2f093d4" location="https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img"
INFO[0062] Downloaded the image from "https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img" 
INFO[0062] Attempting to download the nerdctl archive    arch=aarch64 digest="sha256:ea30ab544c057e3a0457194ecd273ffbce58067de534bdfaffe4edf3a4da6357" location="https://github.com/containerd/nerdctl/releases/download/v1.6.2/nerdctl-full-1.6.2-linux-arm64.tar.gz"
INFO[0076] Downloaded the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v1.6.2/nerdctl-full-1.6.2-linux-arm64.tar.gz" 
INFO[0076] Run `limactl start default` to start the instance. 

$ limactl start default
INFO[0000] Using the existing instance "default"        
INFO[0000] QEMU binary "/opt/homebrew/bin/qemu-system-aarch64" seems properly signed with the "com.apple.security.hypervisor" entitlement 
INFO[0000] [hostagent] hostagent socket created at /Users/nbibler/.lima/default/ha.sock 
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/nbibler/.lima/default/serial*.log") 
INFO[0000] SSH Local Port: 60022                        
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 5: "ssh" 

Checking the logs (serial.log):

UEFI firmware (version edk2-stable202302-for-qemu built at 17:14:55 on Mar 15 2023)
Error: Image at 0013FD41000 start failed: Not Found


Synchronous Exception at 0x000000013FD37E08
PC 0x00013FD37E08 (0x00013FD35000+0x00002E08) [ 0] ArmCpuDxe.dll
PC 0x00013FD37E08 (0x00013FD35000+0x00002E08) [ 0] ArmCpuDxe.dll
PC 0x00013FD37D00 (0x00013FD35000+0x00002D00) [ 0] ArmCpuDxe.dll
PC 0x00013FD380BC (0x00013FD35000+0x000030BC) [ 0] ArmCpuDxe.dll
PC 0x0000476C36C0 (0x0000476BB000+0x000086C0) [ 1] DxeCore.dll
PC 0x0000476C8938 (0x0000476BB000+0x0000D938) [ 1] DxeCore.dll
PC 0x0000476C2F44 (0x0000476BB000+0x00007F44) [ 1] DxeCore.dll
PC 0x00013FD38DA0 (0x00013FD35000+0x00003DA0) [ 2] ArmCpuDxe.dll
PC 0x0000476C2848 (0x0000476BB000+0x00007848) [ 3] DxeCore.dll
PC 0x0000476CFF84 (0x0000476BB000+0x00014F84) [ 3] DxeCore.dll
PC 0x0000476C6340 (0x0000476BB000+0x0000B340) [ 3] DxeCore.dll

[ 0] /home/kraxel/projects/qemu/roms/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPkg/Drivers/CpuDxe/CpuDxe/DEBUG/ArmCpuDxe.dll
[ 1] /home/kraxel/projects/qemu/roms/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
[ 2] /home/kraxel/projects/qemu/roms/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPkg/Drivers/CpuDxe/CpuDxe/DEBUG/ArmCpuDxe.dll
[ 3] /home/kraxel/projects/qemu/roms/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll

  X0 0x0000000047FFE020   X1 0x000000013F0D7003   X2 0x0000000000100000   X3 0x0000000000000000
  X4 0x0000000000000200   X5 0x0000000000000004   X6 0x0060000000000000   X7 0xFF9F000000000F3F
  X8 0x000000013FFFF008   X9 0x0000000400000000  X10 0x000000013F0D5000  X11 0x000000013FA37FFF
 X12 0x0000000000000000  X13 0x0000000000000008  X14 0x0000000000000000  X15 0x0000000000000000
 X16 0x000000013FD36208  X17 0x0000000047702000  X18 0x0000000000000000  X19 0x0000000100000000
 X20 0x0000000047FFE000  X21 0x0000000000000001  X22 0x0000000000000020  X23 0x000000013F0D7003
 X24 0x000000003FFFFFFF  X25 0x0000000000000000  X26 0x000000013CAA0000  X27 0x0000000000000002
 X28 0x000000013FD3D020   FP 0x00000000476BA8A0   LR 0x000000013FD37E08  

  V0 0x0000000000000000 0000000000000000   V1 0x0000000000000000 0000000000000000
  V2 0x0000000000000000 0000000000000000   V3 0x0000000000000000 0000000000000000
  V4 0x0000000000000000 0000000000000000   V5 0x0000000000000000 0000000000000000
  V6 0x0000000000000000 0000000000000000   V7 0x0000000000000000 0000000000000000
  V8 0x0000000000000000 0000000000000000   V9 0x0000000000000000 0000000000000000
 V10 0x0000000000000000 0000000000000000  V11 0x0000000000000000 0000000000000000
 V12 0x0000000000000000 0000000000000000  V13 0x0000000000000000 0000000000000000
 V14 0x0000000000000000 0000000000000000  V15 0x0000000000000000 0000000000000000
 V16 0x0000000000000000 0000000000000000  V17 0x0000000000000000 0000000000000000
 V18 0x0000000000000000 0000000000000000  V19 0x0000000000000000 0000000000000000
 V20 0x0000000000000000 0000000000000000  V21 0x0000000000000000 0000000000000000
 V22 0x0000000000000000 0000000000000000  V23 0x0000000000000000 0000000000000000
 V24 0x0000000000000000 0000000000000000  V25 0x0000000000000000 0000000000000000
 V26 0x0000000000000000 0000000000000000  V27 0x0000000000000000 0000000000000000
 V28 0x0000000000000000 0000000000000000  V29 0x0000000000000000 0000000000000000
 V30 0x0000000000000000 0000000000000000  V31 0x0000000000000000 0000000000000000

  SP 0x00000000476BA8A0  ELR 0x000000013FD37E08  SPSR 0x80000205  FPSR 0x00000000
 ESR 0x86000005          FAR 0x000000013FD37E08

 ESR : EC 0x21  IL 0x1  ISS 0x00000005

Instruction abort: Translation fault, first level

Stack dump:
  00000476BA7A0: 000000013F0D7000 000000013F0D54F8 0060000000000000 FF9F000000000F3F
  00000476BA7C0: 000000013CAA0000 000000013CAA0000 000000003CA00000 0000000C00000001
  00000476BA7E0: 0000000000000002 0000000000000F28 00000000476BA8A0 000000013FD37D00
  00000476BA800: 0000000100000000 0000000047FFE000 0000000000000001 0000000000000020
  00000476BA820: 000000013F0D7000 000000003FFFFFFF 000000013F0D7000 000000013CAA0000
  00000476BA840: 0000000000000002 000000013FD3D020 0000000047FFD000 000000013F0D7F28
  00000476BA860: 0060000000000000 FF9F000000000F3F 000000013CC00000 000000013CAA0000
  00000476BA880: 0000000000000000 0000001500000001 0000000000000001 000000013F0D5000
> 00000476BA8A0: 00000000476BA950 000000013FD37D00 0000000047FFA000 0000000047FFF000
  00000476BA8C0: 0000000000000000 0000000000000000 0000000047FFE000 0000007FFFFFFFFF
  00000476BA8E0: 000000013F0D8228 000000013CAA0000 0000000000000001 000000013FD3D020
  00000476BA900: 00000000476BA940 0000000047FFE020 0060000000000000 FF9F000000000F3F
  00000476BA920: 0000000140000000 000000013CAA0000 0000000147FFA000 0000001E00000001
  00000476BA940: 0060000000000000 000000013F0D7000 00000000476BAA00 000000013FD380BC
  00000476BA960: 002000000000041C 0000000047FFA000 0000000000004000 00000000F4AA6000
  00000476BA980: 0000000047FFF000 0000000000000001 000000013F0D8228 00000000476E5648
ASSERT [ArmCpuDxe] /home/kraxel/projects/qemu/roms/edk2/ArmPkg/Library/DefaultExceptionHandlerLib/AArch64/DefaultExceptionHandler.c(333): ((BOOLEAN)(0==1))

[AkihiroSuda]

• Is this apparently specific to M3? Or macOS 14?
• Does --vm-type=vz work?

[nbibler]

@AkihiroSuda : --vm-type=vz does appear to work! ❤️

I can't easily downgrade any of my machines to test on MacOS <14. And the only Apple Silicon I have is an M3. So, I can't give much useful information to the other questions.

Thank you! I didn't even see that --vm-type was an option to try. 😄

[ethanzway]

@AkihiroSuda
my m3 pro same problem
when use qemu stuck at "Waiting for the essential requirement 1 of 5: "ssh"" but use vz everything is fine
serial.log report error "Error: Image at 0013FD41000 start failed: Not Found"

[AkihiroSuda]

@ethanzway Could you report to https://gitlab.com/qemu-project/qemu/-/issues ?
Is this likely to be specific to M3 (not M2) ?

[jandubois]

rancher-sandbox/rancher-desktop#5943 is another report of this issue with an M3 Max chip:

UEFI firmware (version edk2-stable202302-for-qemu built at 17:14:55 on Mar 15 2023)
Error: Image at 001BFD41000 start failed: Not Found

@AkihiroSuda I realize that this seems to be an upstream problem, but shouldn't we keep this issue here open until it is resolved in QEMU, and we have information on which version is required to fix it?

At least it makes it obvious to other people running into this problem that it is still unresolved.