engineering: validate `trident stream-image` #477

bfjelds · 2026-01-27T22:32:53Z

🔍 Description

Create tests to validate that stream-image correctly applies a VHD-COSI.

Remove dangerous-options from stream-image
Move build-image.yml out of servicing folder and use to build trident-based (not test-images based) images
Modify existing pipeline code to build go tools for arm64 and amd64
Create testimages.py ImageConfig for rawCosi and direct-streaming installer for arm64 and amd64
Create Makefile arm64 and amd64 targets to create VHD-COSI from rawCosi
Add Makefile generated targets for ISO ImageConfigs in testimages.py
Create storm helper to invoke and validate stream-image
Create pipeline template for arm64 and amd64 that invokes storm helper

tests/images/trident-installer/base/post-install.sh

bfjelds · 2026-01-28T17:04:45Z

/AzurePipelines run [GITHUB]-trident-pr

azure-pipelines · 2026-01-28T17:05:08Z

Azure Pipelines successfully started running 1 pipeline(s).

bfjelds · 2026-01-28T19:28:05Z

/AzurePipelines run [GITHUB]-trident-pr-e2e

azure-pipelines · 2026-01-28T19:28:26Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

This pull request validates the trident stream-image command by creating comprehensive test infrastructure for direct streaming deployment with VHD-COSI images. The PR removes the dangerous-options feature flag from the stream-image command, adds ARM64 architecture support for VMs, and creates end-to-end testing pipelines.

Changes:

Removes the dangerous-options feature flag from trident stream-image command, making it available for general use
Adds ARM64 architecture support including VM templates, libvirt configurations, and pipeline infrastructure for both AMD64 and ARM64
Creates new test infrastructure including storm helpers, test images (trident-rawcosi-testimage and trident-direct-streaming-installer), and pipeline templates for validating direct streaming deployment
Implements raw COSI storage support in Trident with grub configuration and fstab handling improvements

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
tools/virtdeploy/templates/vm_arm64.xml	New ARM64 VM template with QEMU/AAVMF firmware configuration
tools/virtdeploy/helpers/libvirthelper.py	Added platform detection for ARM64 support, device naming adjustments
tools/storm/utils/libvirt.go	Improved screenshot capture with better error handling
tools/storm/helpers/direct_streaming.go	New storm helper for testing direct streaming deployments
tools/storm/helpers/prepare_images.go	Added skip functionality for verity images
tools/pkg/virtdeploy/*.go	Refactored VM XML generation to support both AMD64 and ARM64 architectures
tools/pkg/netlaunch/*.go	Added DirectStreaming configuration and ISO patching support
tools/cmd/virtdeploy/main.go	Added runtime.GOARCH detection for VM architecture
crates/trident/src/cli.rs	Removed dangerous-options feature flag from stream-image
crates/trident/src/subsystems/storage/fstab.rs	Added merge_and_write support for raw COSI storage
crates/trident/src/engine/boot/mod.rs	Added grub.cfg patching for raw COSI with PARTLABEL handling
tests/images/trident-rawcosi-testimage/*	New test image configuration for raw COSI testing
tests/images/trident-installer/*	New direct streaming installer configuration and systemd service
tests/images/testimages.py	Added image configs for direct streaming installer and rawcosi testimage
Makefile	Added targets for building direct streaming test images and ISO/COSI patterns
.pipelines/templates/*	New and updated pipeline templates for ARM64 and AMD64 direct streaming tests

Makefile

.pipelines/templates/stages/direct-streaming/netlaunch-testing.yml

tools/storm/helpers/direct_streaming.go

tools/storm/utils/libvirt.go

tests/images/trident-installer/base/files/trident-direct-streaming.service

tools/storm/helpers/direct_streaming.go

tools/virtdeploy/templates/vm_arm64.xml

.pipelines/templates/stages/common_tasks/find-base-image-version.yml

Makefile

bfjelds · 2026-01-28T19:52:32Z

/AzurePipelines run [GITHUB]-trident-pr-e2e

azure-pipelines · 2026-01-28T19:52:52Z

Azure Pipelines successfully started running 1 pipeline(s).

bfjelds · 2026-01-29T00:49:52Z

/AzurePipelines run [GITHUB]-trident-pr-e2e

azure-pipelines · 2026-01-29T00:50:10Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 11 comments.

Copilot · 2026-01-29T00:56:47Z

tests/images/trident-installer/base/files/trident-direct-streaming.service

+ExecStart=/bin/bash -c "echo Running: trident stream-image $(cat /trident_cdrom/direct-streaming.conf | grep URL | cut -d '=' -f 2) --hash $(cat /trident_cdrom/direct-streaming.conf | grep HASH | cut -d '=' -f 2) --verbosity DEBUG"
+ExecStart=/bin/bash -c "trident stream-image $(cat /trident_cdrom/direct-streaming.conf | grep URL | cut -d '=' -f 2) --hash $(cat /trident_cdrom/direct-streaming.conf | grep HASH | cut -d '=' -f 2) --verbosity DEBUG"


The command line parsing using grep and cut is fragile. If the direct-streaming.conf file contains additional whitespace, comments, or malformed lines, this could fail or extract incorrect values. Consider using a more robust configuration parsing approach, such as sourcing the file if it's in shell variable format, or using a proper configuration file parser.

Copilot · 2026-01-29T00:56:48Z

tools/virtdeploy/templates/vm_arm64.xml

+        {% for cdrom in cdroms %}
+        <disk type="file" device="cdrom">
+            {% if cdrom.source %}
+            <source file="{{ cdrom.source }}" />
+            {% endif %}
+            <driver name="qemu" type="raw" />
+            <target dev="{{ cdrom.dev }}" bus="sata" />
+            <readonly />
+            <address type="drive" controller="1" bus="0" target="0" unit="{{loop.index}}" />


The ARM64 template uses SATA bus for CDROMs (line 42) but virtio bus for disks (line 31). This is inconsistent and may cause compatibility issues. The Go code also hardcodes SATA for CDROMs regardless of architecture. Consider using virtio for CDROMs on ARM64 as well, or verify that mixing SATA CDROMs with virtio disks works correctly on ARM64.

Copilot · 2026-01-29T00:56:48Z

tools/virtdeploy/templates/vm_arm64.xml

+        <loader readonly='yes' secure='no' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.ms.fd</loader>
+        <nvram template='/usr/share/AAVMF/AAVMF_VARS.ms.fd'>/var/lib/libvirt/qemu/nvram/{{ name }}_VARS.fd</nvram>


The loader path uses .ms.fd (Microsoft Secure Boot) but the secure attribute is set to 'no'. This is inconsistent - if secure boot is disabled, the non-secure firmware (AAVMF_CODE.fd without .ms) should be used instead. This matches the pattern used in the Go code (resources.go) where different firmware paths are selected based on the SecureBoot setting.

Copilot · 2026-01-29T00:56:48Z

tools/storm/helpers/direct_streaming.go

+
+	"github.com/microsoft/storm"
+	"github.com/sirupsen/logrus"
+	"github.com/stretchr/testify/assert/yaml"


The import path uses github.com/stretchr/testify/assert/yaml which appears incorrect. The testify library typically exposes YAML unmarshaling through gopkg.in/yaml.v3 or similar packages, not through the assert subpackage. This import should be corrected to use the proper YAML library (e.g., gopkg.in/yaml.v3 or gopkg.in/yaml.v2).

Copilot · 2026-01-29T00:56:48Z

tools/storm/helpers/direct_streaming.go

+		}
+	}()
+
+	time.Sleep(10 * time.Second) // Give netlaunch some time to start


Hardcoded 10-second sleep is fragile. Consider implementing a proper readiness check (e.g., polling the netlaunch port or checking for a specific log message) to ensure the server is actually ready before proceeding.

Copilot · 2026-01-29T00:56:49Z

.pipelines/templates/stages/common_tasks/find-base-image-version.yml

  - script: |
      BASEIMG_VERSION="*.*.*"
      echo "##vso[task.setvariable variable=BASEIMG_VERSION]$BASEIMG_VERSION"
      echo "BASEIMG_VERSION: $BASEIMG_VERSION"


BASEIMG_VERSION is set twice in the preview section (lines 38 and 49-54). The first assignment to "..*" (line 38) will be immediately overwritten by the second assignment. Remove the first assignment for clarity and to avoid confusion.

Copilot · 2026-01-29T00:56:49Z

.pipelines/templates/stages/direct-streaming/netlaunch-testing.yml

+              HASH=$(./scripts/cosi-sha384.py $COSI_FILE)
+              sudo ./bin/storm-trident helper direct-streaming -a \
+                  --log-dir $(ob_outputDirectory) \
+                  --netlaunch-config-file ./tools/vm-netlaunch.yaml \


The test references a file ./tools/vm-netlaunch.yaml that does not appear to exist in the repository. This will cause the test to fail when trying to read the netlaunch config file. Ensure this file is added or update the path to reference an existing configuration file.

Suggested change

--netlaunch-config-file ./tools/vm-netlaunch.yaml \

--netlaunch-config-file ./tools/netlaunch.yaml \

Copilot · 2026-01-29T00:56:49Z

tools/storm/helpers/direct_streaming.go

+	netlaunchContext := context.Background()
+	go func() {
+		logrus.Info("Starting netlaunch...")
+		netlaunchErr := netlaunch.RunNetlaunch(netlaunchContext, netlaunchConfig)
+		logrus.Info("netlaunch stopped.")
+		if netlaunchErr != nil && netlaunchErr != context.Canceled {
+			tc.FailFromError(netlaunchErr)
+		}
+	}()


The context created here is never cancelled, which means the netlaunch goroutine will continue running even after the test completes. This could lead to resource leaks. Consider using context.WithCancel and deferring the cancel call to ensure proper cleanup.

Copilot · 2026-01-29T00:56:50Z

crates/trident/src/engine/boot/mod.rs

+            let regex =
+                regex::Regex::new(r"\broot=((PARTUUID|PARTLABEL)=([a-fA-F0-9\-]{36}))\b").unwrap();
+            let updated = regex
+                .replace_all(
+                    &grub_cfg,
+                    format!("root=PARTLABEL={root_device_label}").as_str(),
+                )


The regex pattern expects a UUID format (36 characters with hyphens), but the replacement uses root_device_label which is described in the comment as a PARTLABEL. Partition labels are not necessarily UUIDs and may not match the 36-character format. This could result in the regex not matching the intended root device, leaving the wrong root parameter in grub.cfg. Consider making the regex more flexible to match non-UUID partition labels, or verify that device_id is always a UUID.

Copilot · 2026-01-29T00:56:50Z

Makefile

+		artifacts/trident-direct-streaming-testimage.cosi \
+		$(DIRECT_STREAMING_HOST_CONFIGURATION)
+	rm -rf $(TMP_NO_HC_VHD_COSI)
+	rm -rf $(TMP_HC)


The AMD64 target attempts to remove TMP_HC variable (line 1167) which is not defined in this target. Only the ARM64 target (line 1141) defines TMP_HC. This will cause a harmless warning but should be removed for correctness.

Suggested change

rm -rf $(TMP_HC)

bfjelds commented Jan 27, 2026

View reviewed changes

tests/images/trident-installer/base/post-install.sh Outdated Show resolved Hide resolved

bfjelds changed the title ~~WIP: engineering: validate trident stream-image~~ engineering: validate trident stream-image Jan 28, 2026

bfjelds marked this pull request as ready for review January 28, 2026 19:27

bfjelds requested a review from a team as a code owner January 28, 2026 19:27

Copilot AI review requested due to automatic review settings January 28, 2026 19:27

Copilot started reviewing on behalf of bfjelds January 28, 2026 19:28 View session

Copilot AI reviewed Jan 28, 2026

View reviewed changes

bfjelds added 17 commits January 28, 2026 22:09

update builder code

cd07fe3

copy trident-installer from test-images

9ccac44

build arm64 base image and vhd cosi

7e72669

fix stages

c32a7b2

else not allowed

f813f89

else not allowed

a1f5b31

add to download-test-images

aa6b378

fix

d3833ff

fix path

21f60ce

fix path

89f95d3

name not needed

1bb7fe4

fix stage dep

7a8f560

try

3c3e243

arm64 trident

41f6320

fix architecture

91bd177

try

5046f12

fix Makefile target

b9f6cb4

bfjelds and others added 24 commits January 28, 2026 22:16

remove skip-reboot

8ef793e

remove out.txt

cd173ab

revert

69efa8b

remove unused code; revert extra changes

71a745b

save

ceeea19

unused dep

845c2b0

not mutable

c4492b2

revert

073a7d6

revert

d35c4ec

revert

76a4288

rename

d6c9b03

update README

f457d0a

ppm is enough not to fail

08a8ed2

spice graphics allows screenshots

a6b57a8

no video

1207ebc

add seabios to get screenshot

8b4dd33

refactor

f4b0284

more refactor

7a2f108

refactor

6603e4a

fix post-isntall.sh

3f84ba4

ai feedback

4bdb02f

update protoc with arch

4cf5582

fix merge, use mux

4d37534

remove debug

5a836ab

Copilot AI review requested due to automatic review settings January 29, 2026 00:49

bfjelds force-pushed the user/bfjelds/direct-streaming-test-fairwater branch from cf9bc2f to 5a836ab Compare January 29, 2026 00:49

Copilot started reviewing on behalf of bfjelds January 29, 2026 00:49 View session

Copilot AI reviewed Jan 29, 2026

View reviewed changes

		ExecStart=/bin/bash -c "echo Running: trident stream-image $(cat /trident_cdrom/direct-streaming.conf \| grep URL \| cut -d '=' -f 2) --hash $(cat /trident_cdrom/direct-streaming.conf \| grep HASH \| cut -d '=' -f 2) --verbosity DEBUG"
		ExecStart=/bin/bash -c "trident stream-image $(cat /trident_cdrom/direct-streaming.conf \| grep URL \| cut -d '=' -f 2) --hash $(cat /trident_cdrom/direct-streaming.conf \| grep HASH \| cut -d '=' -f 2) --verbosity DEBUG"

		<loader readonly='yes' secure='no' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.ms.fd</loader>
		<nvram template='/usr/share/AAVMF/AAVMF_VARS.ms.fd'>/var/lib/libvirt/qemu/nvram/{{ name }}_VARS.fd</nvram>

	--netlaunch-config-file ./tools/vm-netlaunch.yaml \
	--netlaunch-config-file ./tools/netlaunch.yaml \

engineering: validate trident stream-image #477

Are you sure you want to change the base?

engineering: validate trident stream-image #477

Conversation

bfjelds commented Jan 27, 2026

🔍 Description

Uh oh!

Uh oh!

bfjelds commented Jan 28, 2026

Uh oh!

azure-pipelines bot commented Jan 28, 2026

Uh oh!

bfjelds commented Jan 28, 2026

Uh oh!

azure-pipelines bot commented Jan 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bfjelds commented Jan 28, 2026

Uh oh!

azure-pipelines bot commented Jan 28, 2026

Uh oh!

bfjelds commented Jan 29, 2026

Uh oh!

azure-pipelines bot commented Jan 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

engineering: validate `trident stream-image` #477

engineering: validate `trident stream-image` #477