qemu: Intel on ARM: change the default CPU from `qemu64` to something newer

[AkihiroSuda]

Found a comparison table: https://www.qemu.org/docs/master/system/i386/cpu.html

Surprised to know that qemu64 sticks to x86_64-v1 by default 👀

Originally posted by @AkihiroSuda in #3047 (comment)

[AkihiroSuda]

We also have some workarounds for qemu64 (why specifically for qemu64?)

lima/pkg/qemu/qemu.go

Lines 547 to 558 in badd9c4

	if strings.HasPrefix(cpu, "qemu64") && runtime.GOOS != "windows" {
	// use q35 machine with vmware io port disabled.
	args = appendArgsIfNoConflict(args, "-machine", "q35,vmport=off")
	// use tcg accelerator with multi threading with 512MB translation block size
	// https://qemu-project.gitlab.io/qemu/devel/multi-thread-tcg.html?highlight=tcg
	// https://qemu-project.gitlab.io/qemu/system/invocation.html?highlight=tcg%20opts
	// this will make sure each vCPU will be backed by 1 host user thread.
	args = appendArgsIfNoConflict(args, "-accel", "tcg,thread=multi,tb-size=512")
	// This will disable CPU S3/S4 state.
	args = append(args, "-global", "ICH9-LPC.disable_s3=1")
	args = append(args, "-global", "ICH9-LPC.disable_s4=1")
	} else if runtime.GOOS == "windows" && accel == "whpx" {

[AkihiroSuda]

Looks like we had Haswell-v4 as the default CPU in the past and then switched to qemu64 in:

• Fix #471: Improve x86_64 emulation on arm64 platform #494

[afbjorklund]

Sounds like a good idea, the default models seem to be discouraged by QEMU anyway?

https://www.qemu.org/docs/master/system/i386/cpu.html#default-x86-cpu-models

The default QEMU CPU models are designed such that they can run on all hosts. If an application does not wish to do perform any host compatibility checks before launching guests, the default is guaranteed to work.

The default CPU models will, however, leave the guest OS vulnerable to various CPU hardware flaws, so their use is strongly discouraged.

Strange that -cpu host doesn't work, but we have seen it break before (e.g. Windows)

---

The v2 versus v3 is bound to become very annoying, if it requires duplicating the architecture...

However, according to same documentation - RHEL 9 shouldn't boot on v1 either, did that work?

EDIT: It (c9s) booted fine in emulation, even though it didn't have the required microarch (v2)???"

Name: QEMU Virtual CPU version 2.5+
Vendor String: AuthenticAMD
Vendor ID: AMD
PhysicalCores: 4
Threads Per Core: 1
Logical Cores: 4
CPU Family 15 Model: 107 Stepping: 1
Features: CMOV,CMPXCHG8,CX16,FXSR,FXSROPT,HYPERVISOR,LAHF,MMX,NX,OSXSAVE,SSE,SSE2,SSE3,SVM,SYSCALL,SYSEE,X87,XSAVE
Microarchitecture level: 1
Cacheline bytes: 64
L1 Instruction Cache: 65536 bytes
L1 Data Cache: 65536 bytes
L2 Cache: 524288 bytes
L3 Cache: -1 bytes

https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level#recommendations_for_rhel_9

UPDATE: Duh, I started Ubuntu by mistake. When actually using centos-stream9, it crashes just fine:

[    3.128571] Run /init as init process
Fatal glibc error: CPU does not support x86-64-v2
[    3.132458] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

[afbjorklund]

For CentOS Stream 10, the error is similar but different:

Fatal glibc error: CPU does not support x86-64-v3

[arixmkii]

It should be possible to boost qemu64 to x86-64-v3 level, like it is done in proxmox https://github.com/proxmox/qemu-server/blob/3aa377c9fe2e55d5434d7aa81a2c8c94b86f914d/PVE/QemuServer/CPUConfig.pm#L52 It might be sane to use this, when (1) hardware acceleration is in play and (2) support for both Intel and AMD is required w/o code/config changes. When using TCG it is better to just emulate some CPU of the desired level.

[AkihiroSuda]

It should be possible to boost qemu64 to x86-64-v3 level, like it is done in proxmox https://github.com/proxmox/qemu-server/blob/3aa377c9fe2e55d5434d7aa81a2c8c94b86f914d/PVE/QemuServer/CPUConfig.pm#L52 It might be sane to use this, when (1) hardware acceleration is in play and (2) support for both Intel and AMD is required w/o code/config changes. When using TCG it is better to just emulate some CPU of the desired level.

qemu64 is only used for TCG mode.
On Intel hosts, the default CPU type is set to host (w/ ,-pdpe1gb):

lima/pkg/limayaml/defaults.go

Lines 57 to 84 in a713c61

	func defaultCPUType() CPUType {
	cpuType := map[Arch]string{
	AARCH64: "cortex-a72",
	ARMV7L: "cortex-a7",
	// Since https://github.com/lima-vm/lima/pull/494, we use qemu64 cpu for better emulation of x86_64.
	X8664: "qemu64",
	RISCV64: "rv64", // FIXME: what is the right choice for riscv64?
	}
	for arch := range cpuType {
	if IsNativeArch(arch) && IsAccelOS() {
	if HasHostCPU() {
	cpuType[arch] = "host"
	} else if HasMaxCPU() {
	cpuType[arch] = "max"
	}
	}
	if arch == X8664 && runtime.GOOS == "darwin" {
	switch cpuType[arch] {
	case "host", "max":
	// Disable pdpe1gb on Intel Mac
	// https://github.com/lima-vm/lima/issues/1485
	// https://stackoverflow.com/a/72863744/5167443
	cpuType[arch] += ",-pdpe1gb"
	}
	}
	}
	return cpuType
	}