ci: Linux/MSHV restore-processors marks restored tsc-early unstable via clocksource watchdog #253

Activity

1. Pedro Henrique Penna (ppenna)

   added

   bugSomething isn't workingSomething isn't working

   on Sep 29, 2026
2. Pedro Henrique Penna (ppenna)

   added a commit that references this issue on Sep 29, 2026

   ci: record restored TSC downtime in restore logs

   ...

   9f3cf42
3. ppenna commented on Sep 29, 2026

   

   Pedro Henrique Penna (ppenna)

   on Sep 29, 2026

   ContributorAuthor

   More actions

   Reproduced on Linux/MSHV; #251 avoids it in CI

   I reproduced this deterministically on azure-mshv-7 (vmss000006, Xeon Platinum 8573C) and on a bare-metal Linux/MSHV lab host (2× Xeon Silver 4114), following the first suggested step. #251 now makes MSHV captures wait until tsc replaces tsc-early, as WHP captures already did.

   Method

   I used a test-only variant of the restore-processors capture:

   • A trivial SMP probe keeps hvc0 busy, so the guest reads the host's snapshot trigger promptly.
   • After reading the trigger, the controller waits until a fixed guest uptime, then runs nvx-snapshot.
   • The snapshot has 8 VPs and maxcpus=1. The untargeted restore starts 0.15 s after the capture process exits.
   • OpenVMM's adjusted restored vCPU TSC debug event reports the applied downtime.
   • After resume, the guest sleeps 1.5 s, then reads dmesg and current_clocksource.

   Results

   Capture uptime	Clocksource at capture	Host	Downtime	TSC marked unstable	Skew
   0.51–0.57 s, before the baseline	tsc-early	azure-mshv-7	166–167 ms	0 of 4	—
   0.60 s	tsc-early	azure-mshv-7	167–168 ms	4 of 4	169–170 ms
   0.80 s	tsc-early	azure-mshv-7	167–168 ms	5 of 5	170 ms
   1.00 s	tsc-early	azure-mshv-7	166–168 ms	5 of 5	170 ms
   1.00 s	tsc-early	Lab host	188–219 ms	5 of 5	190–220 ms
   1.10 s, after the first comparison	tsc-early	azure-mshv-7	168 ms	0 of 4	—
   1.14–1.18 s, with #251's wait	tsc	Both	166–213 ms	0 of 15	—

   The failures logged the same messages as run 36526970004. For example, a 1.00 s capture on azure-mshv-7 logged:

   [    1.250542] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc-early' as unstable because the skew is too large:
   [    1.250548] clocksource:                       'refined-jiffies' wd_nsec: 500000000 wd_now: ffff8b3c wd_last: ffff8b0a mask: ffffffff
   [    1.250550] clocksource:                       'tsc-early' cs_nsec: 670061098 cs_now: b1b8b3e8 cs_last: 55dcc177 mask: ffffffffffffffff
   [    1.250551] clocksource:                       Clocksource 'tsc-early' skewed 170061098 ns (170 ms) over watchdog 'refined-jiffies' interval of 500000000 ns (500 ms)
   [    1.250557] tsc: Marking TSC unstable due to clocksource watchdog
   [    1.250619] clocksource: Switched to clocksource refined-jiffies
   

   What the results add to the hypothesis

   • The window closes at the watchdog's first comparison, about 1.08 s after boot, rather than at the switch to tsc about 1.12 s after boot. A capture at 1.10 s, still on tsc-early, passed. The first successful comparison makes tsc-early valid for high-resolution mode, and the one-shot tick then recovers the jiffies after the restore. Run 36526970004 captured just before that comparison, which ran at wd_now = 109 jiffies, right after the restore.
   • The skew was the downtime rounded up to the next 10 ms tick. Across the 19 failures, the skew exceeded the downtime by 0.7 to 7.2 ms. That puts run 36526970004's downtime between about 132 and 142 ms, rather than the skew plus one tick.
   • In a normal restore-processors run on azure-mshv-7, the applied downtime grew from 124 ms at target 1 to 306 ms at target 2 and 686 ms at target 4, because every target restores the same snapshot. From about target 4 onward, the spanning watchdog interval likely exceeds 1 s, so the check is skipped. Targets 1 and 2 remain exposed.

   What #251 changes

   • MSHV captures that use the SMP probe now wait up to 5 s, before the probe and the snapshot request, until Linux replaces tsc-early. This is the first contract option, applied to the harness. It covers restore-processors, restore-tsc-sync, smp-snapshot, and MSHV benchmark captures.
   • Restore logs now include OpenVMM's adjusted restored vCPU TSC event with the applied downtime, as the second step asks.
   • With the wait, restore-processors, restore-tsc-sync, and smp-snapshot passed 30 of 30 runs on each host. Full test-microvm runs passed 3 of 3 on azure-mshv-7 and 2 of 2 on the lab host.

   #251 doesn't strengthen the guard (the third step) or decide the restore clock contract for snapshots taken outside the harness (the fourth step). A snapshot captured in this window by other tooling still loses the TSC clocksource after restore.
4. Pedro Henrique Penna (ppenna)

   self-assigned this

   on Sep 30, 2026
5. Pedro Henrique Penna (ppenna)

   added the

   Bug

   issue type on Sep 30, 2026
6. Pedro Henrique Penna (ppenna)

   added

   bugSomething isn't workingSomething isn't working

   and removed

   bugSomething isn't workingSomething isn't working

   on Sep 30, 2026
7. ppenna commented on Sep 30, 2026

   

   Pedro Henrique Penna (ppenna)

   on Sep 30, 2026

   ContributorAuthor

   More actions

   Design suggestions

   #251 keeps CI captures out of the tsc-early window. The guard (third step) and the restore clock contract for other capture paths (fourth step) are still open. This comment explains why the guest stays on tsc-early for a second, then proposes a contract and guard options and lists the alternatives I considered. I prototyped the recommendation and validated it on bare-metal Linux/MSHV, Linux/KVM, and Windows/WHP hosts and on azure-mshv-7 (vmss000006). The results are at the end; the prototype isn't submitted.

   Why MSHV and WHP guests stay on tsc-early

   • OpenVMM appends tsc_early_khz=<backend TSC kHz> to every microVM boot that can publish a snapshot (prepare_cold_boot_command_line), so the guest uses the exact frequency that the machine contract records.
   • With that parameter, Linux skips x86_platform.calibrate_tsc() (tsc.c). On Intel, that callback reads CPUID leaf 0x15, which OpenVMM fills in, and marks the frequency as known (tsc.c). The parameter that pins the frequency exactly therefore also keeps Linux from treating it as known.
   • With an unknown frequency, init_tsc_clocksource() starts a refinement instead of registering tsc (tsc.c). The refinement registers tsc one second later. It compares against an HPET or ACPI PM timer, and the microVM has neither, so it registers tsc with the frequency unchanged (tsc.c). The second on tsc-early gains nothing.
   • KVM guests are unaffected. kvmclock's calibrate_cpu callback runs even with tsc_early_khz and marks the frequency as known (kvmclock.c, L323-L324), so tsc is registered at boot. The watchdog's reference on KVM is kvm-clock, which restore advances.

   Why the 1.10 s capture passed

   The comment above attributes this to the first successful comparison making tsc-early valid for high-resolution mode. That can't happen here: the watchdog sets CLOCK_SOURCE_VALID_FOR_HRES only when its reference is continuous (clocksource.c), and refined-jiffies isn't (jiffies.c). The tick stays periodic until tsc is registered.

   The 1.10 s capture passed because the refinement, due at about 1.12 s of guest time, ran after the restore and before the watchdog's next comparison. It removed tsc-early from the watchdog, the one-shot tick caught jiffies up (tick-sched.c), and tsc started from a fresh baseline. The failing window is unchanged: from the watchdog's first reading of tsc-early (about 0.58 s) to its first comparison (about 1.08 s).

   Suggested restore clock contract

   Capture only when the current clocksource's watchdog comparison survives a restore: tsc or kvm-clock, with a one-shot tick. Enforce this in the NVX kernel so it holds for every capture path, not only the harness. The recommended fix is a new NVX kernel patch that treats tsc_early_khz as a known frequency:

   --- a/arch/x86/kernel/tsc.c
   +++ b/arch/x86/kernel/tsc.c
   @@ -1455,6 +1455,11 @@ static bool __init determine_cpu_tsc_frequencies(bool early)
    		cpu_khz = x86_platform.calibrate_cpu();
    		if (tsc_early_khz) {
    			tsc_khz = tsc_early_khz;
   +			/*
   +			 * The microVM VMM supplies its exact TSC frequency, and
   +			 * there is no HPET or PM timer to refine it against.
   +			 */
   +			setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
    		} else {
    			tsc_khz = x86_platform.calibrate_tsc();
    			clocksource_tsc.freq_khz = tsc_khz;

   • init_tsc_clocksource(), a device_initcall, then registers tsc at about 0.09 s instead of 1.12 s. Captures are requested from guest userspace, which starts later, so no capture path can see tsc-early.
   • The TSC stays under the watchdog: no tsc=reliable, tsc=nowatchdog, fallback clocksource, or retry. Boots without tsc_early_khz are unchanged.
   • It is what upstream does one second later anyway, and what kvmclock already does on KVM. A more general version for upstream would skip the refinement whenever there is no HPET or PM timer.
   • ci: avoid the MSHV tsc-early restore race and classify TSC warps #251's wait can then become a check that fails on tsc-early without waiting.
   • Snapshots already captured on tsc-early can't be repaired and must be recaptured.

   One case remains for every option that relies on tsc. A guest booted with nohz=off highres=off keeps a periodic tick even on tsc, and would hit the same skew after any restore with roughly 64–500 ms of downtime. A capture-time check of the tick mode catches it (see the guard options).

   Alternatives considered

   Option	Covers	Drawbacks
   Harness waits for tsc (#251, current)	Harness and benchmark captures	Up to about 0.7 s per MSHV or WHP capture; other capture paths are unprotected
   nvx-snapshot waits for, or refuses, tsc-early	Captures through nvx-snapshot, on any kernel	Misses direct writes to port 0x605, such as scratch-paired.sh and OpenVMM's microVM tests (ttrpc, x86_64). Without the kernel fix, it delays every capture before about 1.12 s, including safe ones before the watchdog's first reading, and it must fail rather than capture on timeout
   OpenVMM drops tsc_early_khz and relies on CPUID leaf 0x15	Intel hosts	Linux reads leaf 0x15 only on Intel, so AMD guests would calibrate against the PIT again. Platform snapshots require the token
   OpenVMM delivers every missed periodic tick	All guests	The LAPIC holds one pending interrupt per vector, so ticks must be replayed one at a time, with no bound for long downtime
   OpenVMM adds a continuous reference clock (HPET, PM timer, or paravirtual clock) that restore advances	All guests	New device or interface in the ACPI-free microVM ABI, with boot-time and snapshot-state cost

   Not recommended, because they turn off the watchdog or bypass the restore contract: tsc=reliable or tsc=nowatchdog; exposing TSC_ADJUST (Linux then turns off the watchdog, and a snapshot can't preserve the register separately from IA32_TSC); resetting the watchdog after restore (it would hide a TSC that didn't follow the restore); and not advancing the TSC over the downtime.

   nvx-snapshot refusing tsc-early without waiting is still worth adding alongside the kernel fix. It never triggers when the fix is present, and it turns a kernel that lacks the fix into a capture failure instead of a snapshot that breaks on restore.

   Guard options (third step)

   The guard reads dmesg once, right after resume (restore-processors.sh), but the comparison that spans the snapshot can run up to 0.5 s later.

   Option	Extra wait per restore	Notes
   Read dmesg again after a fixed 1 s, and fail if current_clocksource changed (prototyped)	1 s	Caught every late verdict in the prototype. Adds about 10 s per backend. Every target restores the same snapshot, so the extra time also pushes later targets past the downtime the watchdog still compares (target 2 on azure-mshv-7 went from about 0.3 s to 1.3 s)
   Save jiffies before capture; after restore, wait until it has advanced by 51, then read dmesg	0.5 s minus the downtime: about 0.4 s at target 1 on azure-mshv-7, about 0 on the bare-metal MSHV host and on WHP	Still observes the kernel's own verdict
   Save jiffies and now at from /proc/timer_list before capture; after restore, fail if jiffies trail CLOCK_MONOTONIC by more than the watchdog's 64 ms margin	None	Deterministic at any downtime, so it covers every target. It reimplements the kernel's check and parses a root-only debug file. On KVM the watchdog's reference is kvm-clock instead
   Check only at capture: log current_clocksource and require a one-shot tick (Tick Device: mode: 1 in /proc/timer_list)	None	Nothing is checked after restore

   I suggest the /proc/timer_list comparison plus the capture-time check. Logging the clocksource at capture also completes the second step. Add the jiffies-deadline re-read if the kernel's own verdict is wanted. snapshot-core should get the same capture-time check on MSHV, where it has none today (microvm_tests.py).

   Prototype results

   The prototype had the kernel patch above, capture checks that don't wait, and the fixed 1 s guard re-read. It used the v0.1.0-dev.1cb930f45360 OpenVMM binaries and a kernel built with CI's Docker toolchain, with the release kernel as the baseline.

   A test-only reproduction booted 8 VPs with maxcpus=1, captured at a fixed uptime, and ran an untargeted restore 0.15 s after the capture process exited. It then read dmesg 1.5 s after resume. I ran five captures per capture time.

   Host	Release kernel, captures at 0.8 s and 1.0 s	Patched kernel, same captures	Downtime
   Bare-metal Linux/MSHV (Xeon Silver 4114)	10 of 10 marked unstable, skew 190–230 ms	0 of 10	191–229 ms
   azure-mshv-7 (vmss000006)	10 of 10 marked unstable, skew 160 ms	0 of 10	164–167 ms
   Bare-metal Linux/KVM (Xeon Silver 4114)	0 of 10; tsc already current	0 of 10	169–216 ms
   Bare-metal Windows/WHP (Xeon Silver 4114)	0 of 10, although captured on tsc-early: the downtime was too long for the watchdog to compare	0 of 10	0.75–1.25 s

   • Captures at 1.3 s passed with both kernels, 5 of 5 on each host.
   • With the patch, tsc replaced tsc-early at 0.09 s, /proc/cpuinfo listed tsc_known_freq, and the shell was ready at the same uptime as before (about 0.40 s on MSHV and 0.36 s on KVM).
   • With the release kernel and captures at 0.65 s, the current single-read guard passed 5 of 5 restores on each MSHV host, and Linux then marked the TSC unstable in all of them. The 1 s re-read failed all 5. With the patched kernel, the re-read passed 5 of 5 and the TSC stayed stable.
   • With the patched kernel, restore-processors, restore-tsc-sync, smp-snapshot, and snapshot-core passed 10 of 10 on both MSHV hosts and 5 of 5 on KVM and WHP. Full test-microvm passed 2 of 2 on each host. The Ubuntu smoke tests passed on all four, and the extended Ubuntu tests passed on KVM. Every logged capture and every restore-processor restore was on tsc, and no TSC-instability messages appeared.
   • With the release kernel, the capture check failed restore-processors in 2.3 s with SMP-CLOCKSOURCE-FAIL expected=stable actual=tsc-early.
8. ppenna commented on Sep 30, 2026

   

   Pedro Henrique Penna (ppenna)

   on Sep 30, 2026

   ContributorAuthor

   More actions

   Implementation design: capture only tickless guests

   This design implements the restore clock contract (fourth step) and replaces the kernel patch recommended in the design suggestions. It follows these constraints:

   • OpenVMM can change as needed, and guests follow a strict snapshot protocol.
   • Avoid kernel patches.
   • Add no latency to restore. Capture may wait.
   • Fix the cause for good, outside CI and on Intel and AMD hosts, without timing assumptions.

   It hasn't been prototyped yet.

   Why the fix belongs at capture

   • Restore advances every VP's TSC by the downtime, but Apic::advance_timer queues a single interrupt for all the periods that a periodic LAPIC timer missed. A guest on a periodic tick adds one jiffy per interrupt, so its jiffies fall behind its TSC by the downtime. While tsc-early is current, the watchdog compares exactly those two.
   • On a one-shot tick, Linux recomputes jiffies from the clocksource at the first tick after restore (tick-sched.c), so the TSC and jiffies advance together. Linux switches a CPU to a one-shot tick only when the current clocksource is valid for high-resolution mode (tick-sched.c). On MSHV and WHP, tsc-early never becomes valid, because its only watchdog there is refined-jiffies.
   • Restoring a periodic tick faithfully would mean either replaying the missed ticks, which adds restore latency that grows with the downtime, or not advancing the TSC, which breaks Fix KVM TSC advancement across snapshot downtime in OpenVMM #27 and Update OpenVMM to validate time-adjusted restores #136. Capturing only guests that are already tickless avoids both.

   Contract

   A microVM snapshot may be captured only when:

   1. the current clocksource is tsc or kvm-clock; and
   2. every online CPU runs a one-shot tick.

   The second condition is the one that matters. The first rejects a guest that has already fallen back to refined-jiffies. The clocksource name alone, which #251 checks, isn't enough: with nohz=off highres=off, tsc keeps a periodic tick, and after tsc registers, each CPU stays periodic until its next tick.

   NVX guest: nvx-snapshot waits for the contract

   This is where the capture delay goes. Every NVX capture path already runs nvx-snapshot, except scratch-paired.sh (see below).

   • After the existing pre-capture checks and before any freeze (L444), poll every 10 ms, for up to 5 s, until the contract holds:

     clock_ready() {
         case "$(cat /sys/devices/system/clocksource/clocksource0/current_clocksource)" in
         tsc | kvm-clock) ;;
         *) return 1 ;;
         esac
         # Each tick device is printed as "Tick Device: mode: N", followed by
         # "Broadcast device" or "Per CPU device: N". Mode 1 is one-shot.
         awk '/^Tick Device: mode:/ { mode = $NF; next }
             /^Per CPU device:/ { cpus++; if (mode != 1) periodic++ }
             END { exit !(cpus > 0 && periodic == 0) }' /proc/timer_list
     }

     /proc/timer_list is always built (Makefile) and is readable by root, which nvx-snapshot already needs for /dev/port. It lists only online CPUs. Its first entry is the broadcast device (timer_list.c), which the parser skips. The sketch passes against sample output in that format: all CPUs one-shot, one CPU still periodic, tsc-early, kvm-clock, refined-jiffies, and no per-CPU entries.

   • Log NVX-SNAPSHOT-CLOCK: source=<name> waited_us=<n> to the console, so every log records the capture-time clocksource (second step).

   • On timeout, exit 1 without requesting a capture, like the other pre-capture checks. The VM keeps running.

   • The guest has to do the waiting, not OpenVMM. The requesting vCPU stays blocked in its out until OpenVMM completes the deferred write, so a guest with one online CPU couldn't reach a one-shot tick while OpenVMM held the write.

   • Expected cost: on MSHV and WHP, a capture requested before Linux registers tsc (about 1.12 s after boot) waits until then, up to about 0.7 s after the shell is ready. KVM guests register tsc at boot, so KVM captures and all later captures pay only for the check.

   NVX harness, tests, and documentation

   • Remove ci: avoid the MSHV tsc-early restore race and classify TSC warps #251's stable_clocksource_wait_script and its uses in the capture controller and in WHP snapshot-core. The controller already runs nvx-snapshot. MSHV snapshot-core, which has no check today, gains one.
   • scratch-paired.sh requests capture by writing to port 0x605 directly. Make it run the same check.
   • Add unit tests for clock_ready with /proc/timer_list fixtures: broadcast entry first, mixed per-CPU modes, all one-shot, kvm-clock, and refined-jiffies.
   • Document the contract in the capture boundary and time and entropy sections of the snapshot design.

   OpenVMM: refuse what the VMM can see

   • Add Apic::periodic_timer_armed() next to advance_timer. It's true when the LVT timer is unmasked, in periodic mode, and has a nonzero initial count, which is exactly the state that advance_timer coalesces.
   • Add a partition-unit query, SnapshotRequest::PeriodicLapicTimers, modeled on AdvanceTsc. It sends a per-VP state RPC that reads apic() through the same state access as TscAdvance::apply, and returns the VPs that have an armed periodic timer. At the capture boundary the VPs are already stopped (stop_at_io_boundary clears started), so the query needs no further quiesce.
   • In quiesce_for_snapshot, before quiesce_for_save, return SnapshotQuiesceError::Rejected if the query returns any VP. The controller already releases the boundary and lets the guest continue.
   • Report the outcome to the guest. Today nvx-snapshot exits 0 after a rejected capture, the same as after a restore without a restore packet, and reads of 0x605 return 0xff.
     • Carry the outcome in the completion of the boundary transaction.
     • Reap it in io_read, as io_write already does, and return 0 for no rejection, 1 for a periodic LAPIC timer, and 2 for any other rejection. A restored VM starts at 0.
     • nvx-snapshot reads the port after its write returns. On a nonzero value it thaws its barriers and exits 1. It treats 0xff as a VMM without the status.
     • Update the ABI table.
   • Hide the TSC-deadline timer for WHP microVMs in the versioned CPU contract, as MSHV already does.
     • Linux drives a periodic tick with the device's periodic mode only when the clock event device supports that mode (tick-common.c). A LAPIC in TSC-deadline mode doesn't, so Linux emulates the tick with deadline one-shots, which look tickless to the VMM.
     • With this change, the check is exact on both backends where ci: Linux/MSHV restore-processors marks restored tsc-early unstable via clocksource watchdog #253 can occur. It changes guest-visible CPUID, so it needs a new WHP CPU contract version.
     • KVM needs nothing: kvm-clock is its watchdog, so ci: Linux/MSHV restore-processors marks restored tsc-early unstable via clocksource watchdog #253 can't occur there, and the guest check covers the rest.
   • The check covers the LAPIC, which is every CPU's tick device once Linux has set up its LAPIC timer. The guest check doesn't depend on the device.
   • Make OpenVMM's microVM tests follow the contract. The x86_64 and ttrpc tests write to 0x605 directly.
   • Document the contract and the rejection in the snapshot guide.
   • Optionally, have TscAdvance::apply fail a restore whose saved LAPIC timer is armed and periodic, with a message to recapture, instead of coalescing it. This adds no waiting, but it rejects existing MSHV snapshots taken during tsc-early, which can't be repaired anyway.

   What doesn't change

   Restore and its latency, tsc_early_khz, the TSC advance, the clocksource watchdog, and the NVX kernel all stay as they are. Nothing is vendor-specific. The guard (third step) keeps its single dmesg read. A snapshot that meets the contract can't produce a watchdog comparison of tsc-early that spans the snapshot, and a post-restore wait would add restore latency.

   Delivery

   1. NVX: the nvx-snapshot check, the harness cleanup, scratch-paired.sh, tests, and documentation. This doesn't depend on OpenVMM, and it protects every NVX capture path immediately.
   2. nanvix/openvmm: the capture rejection, the 0x605 status, WHP TSC-deadline hiding, the test updates, and the guide. Then promote it into NVX with a gitlink update.
   3. NVX, with or after the promotion: nvx-snapshot handles the 0x605 status, and the ABI documentation changes.

   Validation

   • Rerun the reproduction, with captures requested at 0.6, 0.8, and 1.0 s and about 150 ms of downtime, on the bare-metal Linux/MSHV host, azure-mshv-7, and a Windows/WHP host with short downtime. Expect each capture to wait until the guest is tickless, no restore to mark the TSC unstable, and waited_us to appear in the logs.
   • Boot with nohz=off highres=off and expect nvx-snapshot to refuse after 5 s. Write to 0x605 directly during tsc-early and expect OpenVMM to reject the capture on MSHV and WHP, the guest to continue, and 0x605 to read 1.
   • Run restore-processors, restore-tsc-sync, smp-snapshot, and snapshot-core 10 times, and full test-microvm twice, on each backend. Include at least one AMD host; none of the earlier runs used one.
   • Check that restore p50s are unchanged, and report the change in capture latency.
   • Add unit tests for periodic_timer_armed, the VP query, the rejection in quiesce_for_snapshot, and the 0x605 status, including in a restored VM.

   Alternatives under these constraints

   Alternative	Why not
   Kernel patch that treats tsc_early_khz as a known frequency (previous recommendation)	It's a kernel patch, and it doesn't cover periodic ticks under nohz=off highres=off. It could later remove the capture wait on top of this design
   OpenVMM appends setcpuid=tsc_known_freq	Linux prints "this is for TESTING ONLY, may break things horribly" and taints the kernel (common.c)
   Drop tsc_early_khz and rely on CPUID leaf 0x15	Intel only (tsc.c). AMD guests would calibrate against the PIT
   kvmclock on MSHV and WHP	Needs no kernel patch (config) and is vendor-neutral, because kvm_get_tsc_khz marks the frequency known. But it's a new paravirtual interface, the MSHV backend installs no MSR intercepts, guests would identify as KVM, and the watchdog would compare the TSC with a clock derived from it. It's the way to remove the capture wait later without a kernel patch
   HPET or ACPI PM timer as the watchdog	Needs ACPI, which is off
   Replay missed ticks on restore	Restore latency that grows with the downtime
   Don't advance the TSC	Breaks #27 and #136
   Expose TSC_ADJUST, use tsc=reliable or tsc=nowatchdog, or reset the watchdog after restore	These hide the symptom by disabling the alarm
   Guest suspends itself before capture	CONFIG_SUSPEND is off, and monotonic time would stop counting the downtime
   OpenVMM holds the capture until the guest is tickless	The requesting vCPU is blocked in its out, so a guest with one online CPU can't make progress
9. Pedro Henrique Penna (ppenna)

   added

   confirmedIssue affects multiple people.Issue affects multiple people.

   and removed

   bugSomething isn't workingSomething isn't working

   on Sep 30, 2026
10. Pedro Henrique Penna (ppenna)

    added 17 commits that reference this issue on Sep 30, 2026

    microvm: enforce the snapshot clock contract

    ...

    d984976

    virt_whp: retain TSC-deadline for microVM snapshots

    ...

    5801894

    microvm: preserve WHP TSC-deadline snapshot boots

    ...

    d541ad2

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    d56d7a0

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    8f7f15c

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    6f46a8d

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    d6ee39a

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    853383c

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    cca021a

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    105775a

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    391f82b

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    6adfb08

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    3e6e473

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    01c5f5e

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    0defef4

    doc: specify NVX time ABI v1

    ...

    9d2c04b

    ci: stop waiting for a stable clocksource before snapshot capture

    ...

    ba7e9f3
11. Pedro Henrique Penna (ppenna)

    closed this as completedin #325on Oct 5, 2026