PVM host kernel crash when start PVM VM on AMD virtual machine #4
Description
Description
I try to run PVM VM on AMD Zen2 virtual machine (which PCID is disabled), when PVM VM boot into init, AMD virtual machine panic.
[34657.105528] general protection fault: 0000 [#1] PREEMPT SMP NOPTI
[34657.106109] CPU: 7 PID: 954129 Comm: vcpu0 Kdump: loaded Not tainted 6.7.0-rc6-pvm-alex+ #7
[34657.106730] Hardware name: <>, BIOS <>
[34657.107360] RIP: 0010:entry_SYSRETQ_switcher_unsafe_stack+0x55/0x134
[34657.107886] Code: 81 7f 40 33 00 2b 00 0f 85 a9 00 00 00 48 8b 4c 24 20 65 48 89 0d 27 27 20 7e 65 80 35 e7 26 20 7e 03 65 48 8b 0d ef 26 20 7e <0f> 22 d9 0f 01 f8 48 8b 4f 50 48 c1 e1 10 48 c1 f9 10 f3 48 0f ae
[34657.109266] RSP: 0018:fffffe000019ffd0 EFLAGS: 00010002
[34657.109694] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff
[34657.110224] RDX: 0000000000000001 RSI: 000000000005b000 RDI: ffff888207e02000
[34657.110752] RBP: 00007ffe8c827500 R08: 00007f5f0bf3b650 R09: 00007f5f0bfbae30
[34657.111279] R10: 00007f5f0b8f8000 R11: 0000000000000286 R12: 0000000000000000
[34657.111807] R13: 00007f5f0b903428 R14: 00007f5f0bee99a0 R15: 00007f5f0bee99a0
[34657.112339] FS: 00007f5f0b734400(0000) GS:ffff88881fbc0000(0000) knlGS:fffff0003ec00000
[34657.112930] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[34657.113362] CR2: 00007f5f0b92a05e CR3: 0000000163c35000 CR4: 00000000003506f0
[34657.113889] Call Trace:
[34657.114087] <ENTRY_TRAMPOLINE>
[34657.114337] ? show_regs+0x65/0x70
[34657.114608] ? __die_body+0x23/0x70
[34657.114881] ? die_addr+0x41/0x70
[34657.115146] ? exc_general_protection+0x1fd/0x440
[34657.115508] ? asm_exc_general_protection+0x36/0x60
[34657.115880] ? entry_SYSRETQ_switcher_unsafe_stack+0x55/0x134
[34657.116315] </ENTRY_TRAMPOLINE>
Step to reproduce
-
Build PVM host kernel and PVM guest kernel
Following the guide pvm-get-started-with-kata.md, install PVM host kernel in AMD Zen 2 virtual machine. -
PVM VM resource from Guide
cloud-hypervisor v37
VM image from Guide -
Start PVM VM
Start PVM VM on AMD Zen 2 virtual machine
cloud-hypervisor.v37
--api-socket ch.sock \
--log-file vmm.log \
--cpus boot=1 \
--kernel vmlinux.virt-pvm-guest \
--cmdline 'console=ttyS0 root=/dev/vda1 rw clocksource=kvm-clock pti=off' \
--memory size=1G,hugepages=off,shared=false,prefault=off \
--disk id=disk_0,path=ubuntu-22.04-pvm-kata.raw \
-v --console off --serial tty -
The AMD Zen2 virtual machine panic
The crash is caused by switcher try to do fast switch，but using an invalid UMOD CR3 (RCX: ffffffffffffffff), from the debug message found the panic happens after PVM_HC_TLB_FLUSH_CURRENT hypercall, which free the prev_roots, but the fast switch failed to check state of prev_roots, then set an invalid UMOD CR3, and make AMD Zen2 virtual machine panic in PVM switcher code.
[34657.105488] kvm_pvm: vcpu 0 reason 0x20000 rip 0xffffd97f818efcd9 info1 0x0000000000000400 info2 0x0000000000000000 intr_info 0x00000400 error_code 0x00000000
[34657.105490] kvm_pvm: handle_exit_syscall 2112 17088204
[34657.105518] kvm_pvm: entry smod_cr3 163c35000 umod_cr3 ffffffffffffffff
[34657.105528] general protection fault: 0000 [#1] PREEMPT SMP NOPTI
Maybe the fix
So there should be valid check for prev_root, like check_switch_cr3, so my fix is like this, which could boot PVM VM on AMD virtual machine successfully.
--- a/arch/x86/kvm/pvm/pvm.c
+++ b/arch/x86/kvm/pvm/pvm.c
@@ -765,8 +765,9 @@ static void pvm_set_host_cr3_for_guest_without_host_pcid(struct vcpu_pvm *pvm)
{
u64 root_hpa = pvm->vcpu.arch.mmu->root.hpa;
u64 switch_root = 0;
+ u64 prev_root_hpa = pvm->vcpu.arch.mmu->prev_roots[0].hpa;
- if (pvm->vcpu.arch.mmu->prev_roots[0].pgd == pvm->msr_switch_cr3) {
+ if (VALID_PAGE(prev_root_hpa) && (pvm->vcpu.arch.mmu->prev_roots[0].pgd == pvm->msr_switch_cr3)) {
switch_root = pvm->vcpu.arch.mmu->prev_roots[0].hpa;
pvm->switch_flags &= ~SWITCH_FLAGS_NO_DS_CR3;
} else {