Adopt bind-once microVM image slots

.github/actions/check-quality/action.yml

@@ -38,6 +38,7 @@ runs:
           guest/common/nvx-init-agent \
           guest/common/nvx-container-enter-azurelinux \
           guest/common/nvx-container-launch-azurelinux \
+          guest/common/nvx-image-slot \
           guest/common/nvx-sandbox-smoke guest/common/nvx-snapshot \
           guest/common/nvx-virtio-restore-probe \
           scripts/setup/setup-linux-mshv.sh \
@@ -54,6 +55,7 @@ runs:
           guest/common/nvx-init-agent \
           guest/common/nvx-container-enter-azurelinux \
           guest/common/nvx-container-launch-azurelinux \
+          guest/common/nvx-image-slot \
           guest/common/nvx-sandbox-smoke guest/common/nvx-snapshot \
           guest/common/nvx-virtio-restore-probe \
           scripts/setup/setup-linux-mshv.sh \

.github/actions/run-benchmark/action.yml

@@ -140,6 +140,31 @@ runs:
             --output-dir "${run_dir}"
         done
 
+    - name: Run Linux image-slot boot cost
+      if: runner.os != 'Windows'
+      shell: bash
+      env:
+        NVX_BENCHMARK_CPUS: ${{ inputs.cpus }}
+        NVX_BENCHMARK_HOST_CPU_RESERVE: ${{ inputs.host-cpu-reserve }}
+        NVX_BENCHMARK_MARKER_TIMEOUT: ${{ inputs.marker-timeout }}
+      run: |
+        set -euo pipefail
+        cpu_args=(--host-cpu-reserve "${NVX_BENCHMARK_HOST_CPU_RESERVE}")
+        if [[ -n "${NVX_BENCHMARK_CPUS}" ]]; then
+          cpu_args+=(--cpus "${NVX_BENCHMARK_CPUS}")
+        fi
+        python3 scripts/nvx.py benchmark \
+          --suite image-slot-boot \
+          --backend "${{ inputs.backend }}" \
+          --platform "${{ inputs.platform }}" \
+          --processors 1 \
+          "${cpu_args[@]}" \
+          --warmups "${{ inputs.warmups }}" \
+          --runs "${{ inputs.runs }}" \
+          --timeout "${NVX_BENCHMARK_MARKER_TIMEOUT}" \
+          --skip-build \
+          --output-dir "data/runs/${{ inputs.platform }}/image-slot-boot"
+
     - name: Run Linux device operation rates
       if: runner.os != 'Windows'
       shell: bash
@@ -324,6 +349,32 @@ runs:
           if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
         }
 
+    - name: Run Windows image-slot boot cost
+      if: runner.os == 'Windows'
+      shell: powershell
+      env:
+        NVX_BENCHMARK_CPUS: ${{ inputs.cpus }}
+        NVX_BENCHMARK_HOST_CPU_RESERVE: ${{ inputs.host-cpu-reserve }}
+        NVX_BENCHMARK_MARKER_TIMEOUT: ${{ inputs.marker-timeout }}
+      run: |
+        $ErrorActionPreference = "Stop"
+        $CpuArgs = @("--host-cpu-reserve", $env:NVX_BENCHMARK_HOST_CPU_RESERVE)
+        if ($env:NVX_BENCHMARK_CPUS) {
+          $CpuArgs += @("--cpus", $env:NVX_BENCHMARK_CPUS)
+        }
+        python scripts\nvx.py benchmark `
+          --suite image-slot-boot `
+          --backend "${{ inputs.backend }}" `
+          --platform "${{ inputs.platform }}" `
+          --processors 1 `
+          --warmups "${{ inputs.warmups }}" `
+          --runs "${{ inputs.runs }}" `
+          --timeout $env:NVX_BENCHMARK_MARKER_TIMEOUT `
+          --skip-build `
+          --output-dir "data/runs/${{ inputs.platform }}/image-slot-boot" `
+          @CpuArgs
+        if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
     - name: Run Windows device operation rates
       if: runner.os == 'Windows'
       shell: powershell

doc/benchmarks.md

@@ -33,7 +33,9 @@ ABI version and processor count are also separate history dimensions. Legacy
 CSV rows are interpreted as ABI v1 with one vCPU; they are never used as an
 ABI-v2 one-vCPU baseline.
 New runs always emit ABI value 2 and remain stored under `microvm-v2` paths so
-they cannot collide with legacy unsuffixed ABI-1 history.
+they cannot collide with legacy unsuffixed ABI-1 history. The image-slot boot
+suite records its configurations with and without image slots together under
+the dedicated `image-slot-boot` path.
 The OpenVMM benchmark coordinator is implemented in `scripts/nvx_tools/benchmark.py` and exposed
 through the supported NVX CLI. Guest payloads live in `scripts/nvx_tools/benchmark_scripts`; the
 coordinator fills the `.sh.in` templates before use.
@@ -261,6 +263,15 @@ python3 scripts/nvx.py performance collect --platform linux-kvm-baremetal --comm
 | Network snapshot | `benchmark --suite network-snapshot` | Compares a network-ready cold boot with snapshot restore and verifies gateway connectivity. |
 | Snapshot lifecycle profile | `benchmark --suite snapshot-profile` | Retains raw capture and restore phase records and summarizes 128/256/512/1024 MiB warm/cold restores. |
 | Virtio device restore profile | `benchmark --suite device-restore-profile` | Verifies active and driver-unbound deferred restore for console, network, and virtio-fs; emits standalone diagnostic JSON and raw logs. |
+| Image-slot boot cost | `benchmark --suite image-slot-boot` | Measures launches without image slots and with image slots at `B = 1` and `B = 4` using matched launches, and reports both slot deltas against the launch without slots. |
+
+Image-slot boot-cost results must use the same host, backend, kernel,
+initramfs, processor count, memory size, warmups, runs, and readiness marker.
+The suite records three series: `no-slots`, `slots-b1` (`B = 1`), and
+`slots-b4` (`B = 4`). Report both slot deltas against `no-slots` rather than
+comparing only the two slot series. Run the matrix on KVM, MSHV, and WHP;
+performance runners retain the raw commands and results under their normal
+`data/runs` output directory.
 
 ## Kernel command lines
 

doc/design/current-limits.md

@@ -18,6 +18,8 @@ The current ABI family intentionally does not provide:
    without reflink support;
 - direct-claimed paired-scratch restore on filesystems without exact-file
    hard-link support;
+- more than four image slots, writable image slots, binding an inactive slot,
+   rebinding or ejecting media, or activating a slot after restore readiness;
 - sandbox-block, network, or control-console construction through the
    management RPC, or its restore of snapshots that contain them;
 - host networking other than the in-process portable endpoint, unrestricted
@@ -47,7 +49,10 @@ transport foundation for that protocol, not completion of it. The shell
 bootstrap does not implement a fleet-safe platform build point, warm-runtime
 handoff, arbitrary OCI policy, or production agent resource guarantees.
 
-Changing a guest-visible address, IRQ, command-line token, feature mask, queue
+Image slots extend ABI 2 in place: a machine that does not declare them keeps
+the original device topology, and development snapshots from before this
+extension are not guaranteed to remain compatible. Otherwise, changing a
+guest-visible address, IRQ, command-line token, feature mask, queue
 shape, time policy, or device behavior requires a new microVM ABI version. A
 backend-specific difference is valid only when it is explicitly part of that
 versioned contract, such as the virtio-net IRQ.

doc/design/integrated-implementation-status.md

@@ -6,7 +6,7 @@ The implementation and remaining proposals separate as follows:
 
 | Area | Current implementation |
 | --- | --- |
-| Base machine | ACPI-free Linux direct boot with Intel MP tables, allowlisted chipset/PMIO, fixed-role sandbox blocks, nine reserved virtio-mmio slots, and 1/2/4/8-vCPU SMP on KVM, MSHV, and WHP, selected explicitly from the command line or the management RPC. Persisted ABI and boot layout remain value 2. |
+| Base machine | ACPI-free Linux direct boot with Intel MP tables, allowlisted chipset/PMIO, fixed-role sandbox blocks, nine reserved virtio-mmio slots, optional four-slot image capacity that replaces the second virtio-fs slot, and 1/2/4/8-vCPU SMP on KVM, MSHV, and WHP, selected explicitly from the command line or the management RPC. Persisted ABI and boot layout remain value 2, with or without image slots. |
 | Snapshot and restore | Guest-requested version-5 publication, same-backend restore, exact saved inventory, private COW RAM, optional fresh RAM expansion, fresh/paired scratch, three sandbox tiers, input gating, single-use resume claims, a single-use restore readiness event, and management exclusion at the capture boundary. The management RPC drives the same capture and restore for blockless machines without a NIC or control console. Older supported manifests remain readable subject to their recorded capabilities. |
 | Resource activation | Opt-in CPU-prefix and 128-MiB-aligned RAM targets within immutable capacity. Explicit MSHV targets materialize only that VP prefix and cannot be saved again; KVM, WHP, and untargeted MSHV retain full VP capacity. |
 | Console | Boot virtio-console with private RX/TX state and reconnect policies; bounded portb/host-relay drain at process exit, with guest exit status propagated to management-RPC waiters. The separate control console has authenticated Linux and Windows endpoints, bounded framing with receive credits, per-instance epochs, broker saved state, and managed workload RPC. |

doc/design/machine-and-device-abi.md

@@ -103,10 +103,11 @@ restore, the same write acknowledges completion of guest repair.
 
 ## Fixed virtio-mmio transport
 
-All nine fixed address slots are reserved, including the dedicated control
-console at `0xd0007000..0xd0007fff` on IRQ 3 (shared status at `0x3001c`) and
-the second virtio-fs slot at `0xd0008000..0xd0008fff` on IRQ 13 (shared status
-at `0x30020`).
+Without image slots, ABI 2 reserves nine fixed address slots, including the
+dedicated control console at `0xd0007000..0xd0007fff` on IRQ 3 (shared status
+at `0x3001c`) and the second virtio-fs slot at `0xd0008000..0xd0008fff` on
+IRQ 13 (shared status at `0x30020`). A slot-declaring machine also reserves
+`0xd0009000..0xd000bfff`, as described below.
 Every microVM cold-booted from the command line or the management RPC
 instantiates the first virtio-fs slot, so it is discoverable before capture even
 without a host attachment; a restore keeps the slot only when the snapshot
@@ -117,8 +118,8 @@ The control slot is instantiated for a control console, either a live
 authenticated local endpoint on Linux or Windows or an explicitly disconnected
 console. Every device uses virtio-mmio, is discovered only through
 profile-owned command-line tokens, and has packed-ring support masked.
-The MP table routes only the 16 ISA interrupts, and no other device uses
-IRQ 13 on KVM, MSHV, or WHP.
+The MP table routes only the 16 ISA interrupts, and no other device of a
+machine without image slots uses IRQ 13 on KVM, MSHV, or WHP.
 
 | Device | Stable identity | MMIO range | IRQ | Availability |
 | --- | --- | ---: | ---: | --- |
@@ -132,6 +133,31 @@ IRQ 13 on KVM, MSHV, or WHP.
 | Control virtio-console | `console:microvm-control0` | `0xd0007000..0xd0007fff` | 3 | Optional authenticated local endpoint |
 | Second virtio-fs | `fs:microvm1` | `0xd0008000..0xd0008fff` | 13 | Optional; requires a HostFs attachment |
 
+Image slots are an optional part of ABI 2. A slot-declaring machine has four
+permanent image-slot transports. Image slot 0 takes the second virtio-fs slot's
+MMIO window and shared-status word, and image slot 1 takes its IRQ 13, so a
+slot-declaring machine attaches at most one virtio-fs share. Omitting image
+slots keeps the original topology. A slot-declaring launch
+always has capacity `S = 4`; `B` is the active cold-boot prefix, and a restore
+may activate a larger prefix `N` only while restore input remains gated.
+Inactive slots are virtio-mmio placeholders with device ID 0. Active slots are
+empty read-only virtio-blk devices with 512-byte logical and 4096-byte physical
+blocks until the host binds media once.
+
+| Device | Stable identity | MMIO range | IRQ | Shared status |
+| --- | --- | ---: | ---: | ---: |
+| Image slot 0 | `blk:image0` | `0xd0008000..0xd0008fff` | 1 | `0x30020` |
+| Image slot 1 | `blk:image1` | `0xd0009000..0xd0009fff` | 13 | `0x30024` |
+| Image slot 2 | `blk:image2` | `0xd000a000..0xd000afff` | 14 | `0x30028` |
+| Image slot 3 | `blk:image3` | `0xd000b000..0xd000bfff` | 15 | `0x3002c` |
+
+The discovery token is `microvm_image_slots=4`. Binding sets capacity and
+raises a virtio configuration-change interrupt; the guest block driver updates
+capacity through its normal config-change path. The host-control protocol
+permits query and bind-once operations only. The same identity is idempotent,
+a different second bind fails, and there is no unbind, eject, rebind, device
+addition, or activation after readiness.
+
 Explicit placement metadata bypasses the standard sequential MMIO allocator.
 The worker validates the complete device count, kind, bus, address, IRQ, and
 feature policy before resolving devices.
@@ -165,7 +191,10 @@ per fixed slot resides in the reserved shared-status page:
 | `custom` block | `0x30014` |
 | `scratch` block | `0x30018` |
 | Control virtio-console | `0x3001c` |
-| Second virtio-fs | `0x30020` |
+| Second virtio-fs, or image slot 0 | `0x30020` |
+| Image slot 1 | `0x30024` |
+| Image slot 2 | `0x30028` |
+| Image slot 3 | `0x3002c` |
 
 OpenVMM publishes config-change and used-buffer bits with a sequentially
 consistent compare-exchange loop. It pulses the device IRQ only when the old

doc/design/remaining-production-work.md

@@ -28,10 +28,11 @@ the required stop signal. Such images may need `CAP_MKNOD` together with a
 cgroup device filter. They are not covered by the current drop-all-capabilities
 bootstrap, and its private mount policy must not be described as that profile.
 
-Azure Files, secret/content volumes, and other optional mounts remain undesigned.
-Guest-side CIFS, a scratch-backed temporary directory, or an explicit small
-content attachment are possible directions, not supported extra device slots
-in the fixed ABI. Managed executions deliver bounded environments through a
+Secret/content volumes and other optional mounts remain undesigned.
+Guest-side network filesystems, a scratch-backed temporary directory, or an
+explicit small content attachment are separate directions. ABI 2 can declare
+four fixed read-only image slots, but they are bind-once block media rather
+than general volume or device hotplug. Managed executions deliver bounded environments through a
 sealed anonymous file inside the guest; snapshot-time launch configuration and
 secret-volume delivery remain distinct proposed features. Scratch cannot
 currently grow online; capacity exhaustion and whole-file copy-up remain

doc/design/sandbox-filesystem-and-agent-architecture.md

@@ -32,9 +32,10 @@ supplied native library and an image-backed edge guest that NVX does not build.
 ## Implemented filesystem bootstrap
 
 The public `nvx sandbox` command accepts one to three role-bearing EROFS lower
-images, a preformatted ext4 scratch image, up to two
-[live host shares](#live-host-shares), an absolute entrypoint, and
-individual argument tokens. It supplies non-secret kernel-command-line
+images (none for a managed launch that declares image slots), a preformatted
+ext4 scratch image, up to two [live host shares](#live-host-shares) (one when
+the launch declares image slots), an absolute entrypoint, and individual
+argument tokens. It supplies non-secret kernel-command-line
 configuration for one-shot runs. Managed execution carries bounded arguments
 and a per-execution environment over the authenticated control channel;
 one-shot environment variables, secrets, and sandbox snapshot orchestration
@@ -84,10 +85,14 @@ virtio-fs tag microvm    (optional share) ---> /run/nvx/rootfs/TARGET
 virtio-fs tag microvm1   (optional share) ---> /run/nvx/rootfs/TARGET
 ```
 
-At least one lower role is required by the bootstrap; distro plus runtime is
-the intended curated-image shape, not a requirement that all three lower
-slots be populated. The initramfs remains the supervisor's root and is not
-another container lower layer.
+At least one lower role is required by the bootstrap unless the launch
+declares image slots; distro plus runtime is the intended curated-image shape,
+not a requirement that all three lower slots be populated. A slot-declaring
+launch without lower roles bind-mounts `/run/nvx/scratch/upper` as
+`/run/nvx/rootfs` instead of assembling an overlay, and skips the image's
+user-database checks: the workload runs with the configured UID and GID and
+home `/`. The initramfs remains the supervisor's root and is not another
+container lower layer.
 
 The container launch helper releases the barrier into private mount, PID, and
 UTS namespaces. The container entry helper makes mounts
@@ -122,9 +127,11 @@ enforces the share's mode and its denied, allowed, and writable paths, and
 accesses host files as the identity that `--mount-owner` selects. Guest
 mount flags are therefore not a security boundary, and the access policy adds
 no guest configuration. Before OpenVMM starts, NVX rejects a third share and
-policy paths outside their share. It also rejects guest targets or host
-directories that equal or contain one another, because one share could
-otherwise hide the other or reach its files under a different policy.
+policy paths outside their share. A launch that declares image slots accepts
+only one share, because image slot 0 takes the second share's MMIO window and
+status word, and image slot 1 takes its IRQ. NVX also rejects guest targets
+or host directories that equal or contain one another, because one share
+could otherwise hide the other or reach its files under a different policy.
 [Machine and device ABI](machine-and-device-abi.md#filesystem) defines the
 device contract.
 

doc/design/snapshot-and-restore.md

@@ -496,6 +496,27 @@ Performance comparisons should separate VP binding and worker construction
 from guest resume-to-readiness. Matching host-side phase costs do not require
 the total restore latencies or tail behavior to match.
 
+## Restore-time image-slot activation
+
+Snapshots of slot-declaring machines record image-slot capacity `S = 4`, the
+cold-boot active prefix `B`, and the empty state of every active slot. Capture
+is rejected before the guest is quiesced if any slot is bound. Snapshots
+without image slots record zero capacity and active count, and their restore
+packet leaves header byte 7 zero.
+
+An explicit `--restore-image-slots N` target is process-local and must satisfy
+`B <= N <= 4`. During gated repair the guest binds the virtio-mmio driver to
+newly active placeholders `B..N-1`, verifies that all slots in `0..N-1` are
+empty read-only block devices, and only then acknowledges the restore gate.
+The target does not modify the snapshot, and no operation activates a slot
+after readiness.
+
+The [restore packet](time-abi.md#restore-packet) carries `N` in header byte 7
+and sets its `IMAGE_SLOT_TARGET` flag; the target is always in `1..4`, and no
+portb status bit advertises it. Every explicit target holds the restore gate,
+so the guest always verifies the active prefix and acknowledges before
+readiness, even when no new slot becomes active.
+
 ## Restore-time memory activation
 
 Restore-time memory activation separates the immutable RAM geometry recorded