Bug: validator smoke launch of malformed copilot.exe hangs Install.Tests (unsuppressed bad-image hard error; 10-min timeout exceeds 90s blame)

Summary

The release-gate payload validator runs a "startup smoke" that launches the bundled copilot.exe. The Install.Tests theory CopilotWrapperPackagingTests.RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails deliberately overwrites copilot.exe with non-executable text and asks PowerShell to run it, expecting the validator to catch the launch failure and print copilot.exe could not be launched. Usually the launch fails fast, but intermittently the child pwsh blocks synchronously and un-cancellably inside CreateProcess while the OS raises a bad-image ("could not start correctly") hard error for the malformed binary. Because this specific test invocation is given the generic 10-minute process timeout instead of the 30-second validator timeout, the stall lasts far longer than VSTest's 90-second --blame-hang timeout, so the hang collector aborts the whole test host. The run reports all assertions passed (e.g. 7,489/7,489 fast, 7,721/7,721 full) and then test host crashed, turning a green suite into a gate failure.

Two independent defects combine to produce the host crash, plus a test-design weakness that makes the failure branch inherently nondeterministic.

Root Cause

1. Bad-image hard error at grandchild launch is not suppressed (the stall)

packaging/validate/Assert-CopilotRuntimePayload.ps1 performs the smoke launch:

# Assert-CopilotRuntimePayload.ps1:100-108
$PSNativeCommandUseErrorActionPreference = $false
try
{
    $startup = & $copilotExe 2>&1        # line 103 — launches the payload copilot.exe
    $startupExitCode = $LASTEXITCODE
}
catch
{
    throw "Bundled copilot.exe could not be launched: $($_.Exception.Message) (issue #1376)."   # line 108
}

When copilot.exe is not a valid PE image, & $copilotExe reaches
NativeCommandProcessor.InitNativeProcess → Process.StartWithCreateProcess → Kernel32.CreateProcess. If neither the launching process nor its job has established critical-error suppression, Windows raises a critical-error hard-error (bad image format / STATUS_INVALID_IMAGE_FORMAT) for the malformed image. On an interactive session that hard error is presented as a modal message box, and CreateProcess blocks the pipeline thread until it is dismissed — which never happens under an automated run. The child pwsh dump captured exactly this state:

Interop.Kernel32.CreateProcess
System.Diagnostics.Process.StartWithCreateProcess
System.Management.Automation.NativeCommandProcessor.InitNativeProcess

The contained runner does own a job, but it only suppresses the wrong class of error. Phantom.Workspaces.Data.Core/ProcessRunner.cs ConfigureJob sets:

// ProcessRunner.cs:922-924
information.BasicLimitInformation.LimitFlags =
    JOB_OBJECT_LIMIT_DIE_ON_UNHANDLED_EXCEPTION | JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;

JOB_OBJECT_LIMIT_DIE_ON_UNHANDLED_EXCEPTION applies only SEM_NOGPFAULTERRORBOX (GP-fault dialog suppression) to job members. A bad-image launch is not a GP fault; its hard-error message box is governed by the separate SEM_FAILCRITICALERRORS flag, which is never set for the validator pwsh or its children. This is the same modal-hang class diagnosed in #1514, but through the one error-mode flag that #1514's job/probe suppression does not cover.

2. This invocation uses the 10-minute timeout, exceeding the 90-second blame-hang timeout (why it becomes a host crash instead of a bounded failure)

The GitHub-step helper routes the pwsh launch through the default InvokeAsync, which uses LongRunningProcessTimeout:

// CopilotWrapperPackagingTests.cs:418-442
private static async Task<MxcRepositoryTestSupport.ProcessResult> InvokeValidatorAsGitHubStepAsync(
    string payloadDirectory, bool useNativeErrorPreference)
{
    ...
    return await MxcRepositoryTestSupport.InvokeAsync(   // line 437 — no InvocationOptions
        "pwsh", "-NoProfile", "-NonInteractive", "-File", scriptPath);
}

// MxcIntegrationTests.cs:1240-1241, 1383-1387
private static readonly TimeSpan LongRunningProcessTimeout = TimeSpan.FromMinutes(10);
private static readonly TimeSpan ValidatorProcessTimeout   = TimeSpan.FromSeconds(30);
...
internal static Task<ProcessResult> InvokeAsync(string fileName, params string[] arguments) =>
    InvokeAsync(fileName, new InvocationOptions { Timeout = LongRunningProcessTimeout }, arguments);

The fast/full gates run with a 90-second per-test blame-hang timeout:

# scripts/run-tests.ps1:7, 62-64
[string] $PerTestHangTimeout = '90s',
...
$dotnetArgs += @('--blame-hang', '--blame-hang-timeout', $PerTestHangTimeout)

ProcessRunner.RunWindowsContainedAsync can recover from the stall: on timeout it flips timedOut, calls TerminateJobObject to kill the whole pwsh tree, drains the pipes, and throws a bounded TimeoutException (ProcessRunner.cs:399-470). But that recovery is scheduled for 600 seconds, while VSTest's blame collector fires at 90 seconds. The hang collector therefore terminates the test host (capturing testhost + pwsh dumps, outcome Failed with zero test failures) long before the runner's own timeout can kill the stuck pwsh. The testhost dump shows it still awaiting both pipe readers, which cannot complete while pwsh holds the pipe:

ProcessRunner.ReadLinesAsync
ProcessRunner.RunWindowsContainedAsync
MxcRepositoryTestSupport.InvokeAsync
CopilotWrapperPackagingTests.InvokeValidatorAsGitHubStepAsync
RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails

3. The failure branch is exercised through a nondeterministic OS launch (why even a bounded fix is not enough on its own)

RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails (CopilotWrapperPackagingTests.cs:145-165) writes "not a Windows executable" into copilot.exe and relies on the OS refusing to launch it to reach the catch branch. Whether that refusal is fast-and-clean or a blocking hard error depends on machine state outside the test's control (interactive session hard-error UI, Defender real-time scan of the freshly written binary, Program Compatibility Assistant). So even after the stall is bounded, asserting copilot.exe could not be launched this way can still time out instead of asserting the intended message.

Proven vs. uncertain

Proven (dumps + source): child pwsh blocked in Kernel32.CreateProcess; testhost blocked in the two pipe readers; this invocation uses the 10-minute timeout; blame-hang timeout is 90s; ConfigureJob sets only GP-fault (not critical-error) suppression. The 10-min-vs-90s mismatch fully explains why the stall surfaces as a host crash rather than a bounded failure.
Environmental / not fully pinned: the exact OS actor that makes CreateProcess block for a given occurrence (critical-error hard-error box vs. Defender scan vs. PCA). All are variants of "the OS launch of arbitrary malformed bytes is not guaranteed to fail fast," which the fix removes by not depending on that path.

Current status vs. #1585–#1590 / #1593

Observed at HEAD d5913892 (fix Phantom.Workspaces.exe has no embedded PE icon: missing ApplicationIcon plus corrupt brain.ico asset #1585) during the Phantom.Workspaces.exe has no embedded PE icon: missing ApplicationIcon plus corrupt brain.ico asset #1585–Git workspace scan silently narrows discovery to the wrong subtree when a fixed-drive root is trimmed to a drive-relative path #1589 check-in batch. The NativeLaunchFailureFails test and the validator try/catch were introduced earlier by commit f04c8524 ("Fix Copilot validator native exit state") and predate that batch.
Current HEAD is fb6091c8 (merge of fix Bug: Tests seed supposedly-valid persisted entities through raw/fake DALs, bypassing schema validation (encodes unstorable fixtures) #1590). Neither Bug: Tests seed supposedly-valid persisted entities through raw/fake DALs, bypassing schema validation (encodes unstorable fixtures) #1590 (508cbc04, persisted-seed validation) nor Design and persist a canonical routing/reachability data model on user-computer-profile (local inbound registry + reverse-hub + direct HTTP routes) #1593 (routing/reachability data model, unrelated) touched CopilotWrapperPackagingTests, MxcIntegrationTests, ProcessRunner, or Assert-CopilotRuntimePayload.ps1. The report is not stale — InvokeValidatorAsGitHubStepAsync still uses the 10-minute timeout and ConfigureJob still omits critical-error suppression at current HEAD.

Occurrence evidence

Reproduced three times at HEAD d5913892 on the same machine state (dumps archived to a local directory and then removed from the worktree; the paths are local and are omitted here):

Run	Assertions	Outcome	Last in-flight test
Fast gate #1	7,489 / 7,489 passed (187/187 Install)	`Failed` — test host crashed	`RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails`
Fast gate #2	187/187 Install passed	`Failed` — host aborted	same
Full/slow gate	7,721 / 7,721 passed	`Failed` — host aborted	same

Each run archived matched testhost + pwsh collector dumps; dumpasync showed the runner awaiting ReadLinesAsync/RunWindowsContainedAsync and the child pwsh blocked in NativeCommandProcessor.InitNativeProcess → Process.StartWithCreateProcess → Kernel32.CreateProcess. The repeat across fast and full gates makes it reliably reproducible in that machine state rather than a one-off.

Affected Files

Path	Relevance
`packaging/validate/Assert-CopilotRuntimePayload.ps1` (smoke launch line 103; catch line 106-108)	Launches the payload `copilot.exe` with no critical-error suppression; source of the blocking `CreateProcess`.
`Phantom.Workspaces.Install.Tests/CopilotWrapperPackagingTests.cs` (test 145-165; helper 418-442, `InvokeAsync` at 437)	The hanging test and the helper that hands `pwsh` the 10-minute timeout instead of the validator timeout.
`Phantom.Workspaces.Install.Tests/MxcIntegrationTests.cs` (timeouts 1240-1241; `InvokeAsync` 1383-1387; `InvokePowerShellAsync` 1599-1608)	Defines `LongRunningProcessTimeout` (10 min) and the unused-here `ValidatorProcessTimeout` (30 s); the overload plumbing that the helper should use.
`Phantom.Workspaces.Data.Core/ProcessRunner.cs` (`ConfigureJob` 920-945, flags 922-924; timeout/kill 399-470)	Job suppresses only GP faults, not bad-image critical errors; its timeout kill is correct but scheduled after the blame timeout.
`scripts/run-tests.ps1` (7, 62-64)	Sets the 90-second `--blame-hang-timeout` that the 10-minute invocation exceeds.
`Phantom.Workspaces.Test.WindowsProcessProbe/`	Existing modal-safe probe (from #1514) that already sets `SEM_FAILCRITICALERRORS

Design / Fix

Apply both a bounding fix and a determinism fix; add the defense-in-depth suppression so production is safe too.

A. Bound the invocation to the validator timeout (required)

Give the GitHub-step validator launch the 30-second ValidatorProcessTimeout instead of the 10-minute default, using the existing InvokeAsync(fileName, InvocationOptions, args) overload (mirroring InvokePowerShellAsync, MxcIntegrationTests.cs:1599-1608):

// CopilotWrapperPackagingTests.cs — InvokeValidatorAsGitHubStepAsync
return await MxcRepositoryTestSupport.InvokeAsync(
    "pwsh",
    new MxcRepositoryTestSupport.InvocationOptions { Timeout = /* ValidatorProcessTimeout */ TimeSpan.FromSeconds(30) },
    "-NoProfile", "-NonInteractive", "-File", scriptPath);

(Expose ValidatorProcessTimeout to the packaging test, or reuse InvokePowerShellAsync.) This alone converts a host crash into a deterministic, killable TimeoutException at 30 s (< 90 s blame), and the existing observer plumbing proves job/tree termination and both pipe drains.

B. Make the failure branch deterministic (required)

Stop depending on the OS launch of arbitrary malformed bytes. Preferred: have the validator establish critical-error suppression on its own dedicated pwsh process before the smoke, so a bad-image launch returns the error to PowerShell (hitting the catch) instead of blocking on a hard-error box. Setting the error mode on the isolated validator pwsh is safe (unlike the parallel xUnit host, which #1514 correctly refuses to mutate):

# Assert-CopilotRuntimePayload.ps1 — before the smoke launch
Add-Type -Namespace Native -Name ErrorMode -MemberDefinition @'
[System.Runtime.InteropServices.DllImport("kernel32.dll")]
public static extern uint SetErrorMode(uint uMode);
'@
# SEM_FAILCRITICALERRORS (0x1) | SEM_NOGPFAULTERRORBOX (0x2)
[void][Native.ErrorMode]::SetErrorMode(0x0001 -bor 0x0002)

With SEM_FAILCRITICALERRORS set on the launching process, CreateProcess of the malformed image returns STATUS_INVALID_IMAGE_FORMAT to pwsh, & $copilotExe throws, and the catch emits copilot.exe could not be launched deterministically and fast — no modal, no dependence on Defender/PCA timing. This also hardens the real release gate against a corrupt shipped copilot.exe.

C. Defense-in-depth in the contained runner (recommended)

So any job-hosted grandchild launch of a bad image fails fast rather than blocking, route the deterministic launch-failure test through the existing modal-safe WindowsProcessProbe/harness (#1514), which already owns a permanent SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX error mode, and/or add a targeted repeated-stability assertion that the validator child timeout kills the full tree and drains/cancels stdout+stderr.

Considered / Background (not chosen alone)

Timeout-only fix (A without B): bounds the crash but can still fail the test with a TimeoutException instead of the intended assertion whenever the OS launch stalls; insufficient for a green, meaningful test.
SetErrorMode on the xUnit test host: rejected per Windows child-process tests can hang on unsuppressed STATUS_DLL_INIT_FAILED dialog #1514 — process-global, inherited, and races parallel tests. B deliberately mutates only the disposable validator pwsh.
Widening the runner job to add a critical-error flag: job limit flags do not expose SEM_FAILCRITICALERRORS; that suppression must come from process error mode (probe/validator), which is why C uses the probe rather than a new job flag.

Expected Tests

Test Name	Class	What It Verifies
`RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails`	`CopilotWrapperPackagingTests`	With critical-error suppression established, the malformed `copilot.exe` launch fails fast and the validator emits `copilot.exe could not be launched` with a non-zero exit — no hang.
`RuntimePayload_GitHubFinalCommand_NativeLaunchFailure_CompletesRepeatedlyWithoutHang`	`CopilotWrapperPackagingTests`	Repeated runs of the launch-failure scenario each complete within the validator timeout, proving the flake is removed.
`InvokeValidatorAsGitHubStep_UsesValidatorTimeout_NotLongRunningTimeout`	`CopilotWrapperPackagingTests`	The GitHub-step validator invocation is bounded by the 30-second validator timeout, keeping any stall below the 90-second blame-hang timeout.
`InvokeAsync_ValidatorChildStall_KillsProcessTreeAndDrainsStreams`	`MxcIntegrationTests`	A forced validator-child stall trips the timeout, `TerminateJobObject` removes the whole tree (zero active job members), and both redirected streams are drained/cancelled.
`ConfigureJob_BadImageGrandchildLaunch_DoesNotBlockOnHardError`	`ProcessRunnerModalSafetyTests`	A job-hosted child that launches a malformed image returns the launch error instead of blocking on an un-suppressed critical-error hard error.

Dependencies / Related Issues

Related (closed): Windows child-process tests can hang on unsuppressed STATUS_DLL_INIT_FAILED dialog #1514 — Windows child-process modal hangs; established the SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX / job-flag analysis and the WindowsProcessProbe harness reused by fix C.
Related (closed): Bug: Install.Tests testhost hang after MxcRuntimePayload_RequiredNativeUnit_IsPresent completes #1494 — the previous Install.Tests test-host hang (MxcRuntimePayload_RequiredNativeUnit_IsPresent), which hardened the contained-runner pipe/resource lifecycle this issue builds on.
Origin: the smoke and catch branch come from Installed release omits Copilot runtime and does not discover copilot.exe from PATH #1376 (bundle + validate the Copilot CLI runtime) and were made GitHub-step-accurate by commit f04c8524.
No open duplicate exists; Bug: validator smoke launch of malformed copilot.exe hangs Install.Tests (unsuppressed bad-image hard error; 10-min timeout exceeds 90s blame) #1591 is the sole open tracker for this hang.

Implementation Plan

Route InvokeValidatorAsGitHubStepAsync through the 30-second ValidatorProcessTimeout (fix A); add the "uses validator timeout" and "completes repeatedly without hang" tests.
Establish SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX on the validator pwsh before the smoke launch (fix B); update RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails to assert the deterministic could not be launched outcome.
Add the contained-runner defense-in-depth coverage via the WindowsProcessProbe harness (fix C) and the InvokeAsync stall/tree-kill/drain test.
Re-run the fast and full gates repeatedly on the reproducing machine state to confirm no testhost/pwsh blame dumps are produced.

Design

Root cause

The GitHub-step helper starts the validator pwsh with the generic ten-minute integration timeout. Inside that process, the validator directly invokes arbitrary copilot.exe bytes without establishing SEM_FAILCRITICALERRORS. A malformed PE can therefore hold the PowerShell pipeline synchronously in CreateProcess behind a Windows bad-image hard-error UI. VSTest's 90-second inactivity collector kills the testhost long before the contained runner's ten-minute tree timeout. The test also depends on OS behavior for arbitrary text, so its intended diagnostic is not deterministic.

Approach

Add an in-process PE preflight to the validator: require a regular, readable PE executable image before native launch and map malformed/non-executable input to the existing clear Bundled copilot.exe could not be launched diagnostic. This makes the malformed fixture deterministic without weakening provenance or payload checks; a valid payload is still actually launched and must produce the expected server-mode response.
Around the real smoke launch only, use Windows GetThreadErrorMode/SetThreadErrorMode to add SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX, preserve the caller's existing flags, and restore the exact previous thread mode in finally. Thread-scoped suppression avoids process-global races and also protects valid-looking images that fail later in the loader.
Route the generated GitHub-step script through the same 30-second validator invocation options as other payload validators. Keep that inner bound explicitly below the 90-second test inactivity contract; do not add heartbeats or mask hangs.
Retain the contained runner's kill-on-close job and stream-drain behavior. Extend the existing modal-safe Windows probe with a deterministic malformed-image grandchild scenario so ProcessRunner cleanup and bad-image failure behavior are exercised without ever exposing the parallel testhost to hard-error UI.

Tests first

Repeat malformed copilot.exe validation concurrently and require every attempt to fail with the intended launch diagnostic, no timeout, direct job assignment, and zero active job processes after cleanup.
Assert the central validator timeout is 30 seconds and below the 90-second outer inactivity contract.
Add ProcessRunnerModalSafetyTests coverage for a bad-image grandchild launched only inside the dedicated modal-safe probe.
Preserve the real payload accepted-exit smoke and unexpected-valid-executable diagnostics. The existing event-driven stalled-validator test continues to prove timeout termination of validator + descendant, stream diagnostics, handle release, and artifact cleanup.

Alternatives considered

Timeout-only was rejected because it merely turns the flake into a 30-second timeout. SetErrorMode in xUnit was rejected because it is process-global and races parallel tests. A heartbeat was rejected because it would hide a real CreateProcess hang.

Files

packaging/validate/Assert-CopilotRuntimePayload.ps1
Phantom.Workspaces.Install.Tests/CopilotWrapperPackagingTests.cs
Phantom.Workspaces.Install.Tests/MxcIntegrationTests.cs
Phantom.Workspaces.Testing/Processes/WindowsChildProcessTestHarness.cs
Phantom.Workspaces.Test.WindowsProcessProbe/Program.cs
Phantom.Workspaces.Data.Core.Tests/ProcessRunnerModalSafetyTests.cs

Implementation notes

Commit: 7a255d72d782d6f108b55b64f75f7b25ed99f6c4

What changed

The payload validator now preflights copilot.exe with PEReader and requires a Windows executable image before native launch. Malformed text and PE files without ExecutableImage fail immediately with the existing clear copilot.exe could not be launched diagnostic.
Real payloads are still launched and must reach the expected server-mode argument validation; provenance/payload checks are unchanged.
The actual smoke launch adds SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX using GetThreadErrorMode/SetThreadErrorMode, preserves existing flags, and restores the exact prior thread mode in finally. No process-global error mode is mutated.
GitHub-step validation now uses the central 30-second validator timeout, explicitly below the 90-second inactivity collector, while retaining ProcessRunner kill-tree and stream-drain containment.
The dedicated modal-safe Windows probe now creates a malformed image and proves ProcessRunner returns error 193/216 without a hard-error hang and releases all resources that were acquired. Repeated concurrent validator tests prove non-zero deterministic diagnostics, direct job containment, and zero active job members after cleanup. A valid but non-executable PE fixture covers preflight rejection.

Deviations from proposed design

None. No heartbeat was added; real stalls remain visible and bounded.

Scope changes

The existing event-driven InvokePowerShellAsync_StalledValidator_TimesOutAndCleansProcessTreeAndArtifacts test already proves timeout termination of the validator/descendant tree, stream drain diagnostics, handle release, and artifact cleanup, so it was retained and included in the targeted class run rather than duplicated.

Validation

dotnet build --no-incremental: succeeded, zero error lines.
scripts\run-tests.ps1 -Mode fast -TestNames CopilotWrapperPackagingTests,ProcessRunnerModalSafetyTests,MxcIntegrationTests: 31 executed, 31 passed, 0 failed.

Files changed

packaging/validate/Assert-CopilotRuntimePayload.ps1
Phantom.Workspaces.Install.Tests/CopilotWrapperPackagingTests.cs
Phantom.Workspaces.Install.Tests/MxcIntegrationTests.cs
Phantom.Workspaces.Data.Core.Tests/ProcessRunnerModalSafetyTests.cs
Phantom.Workspaces.Test.WindowsProcessProbe/Program.cs
Phantom.Workspaces.Testing/Processes/WindowsChildProcessTestHarness.cs

Batch stability evidence

The combined touched-test union was run 10 consecutive times with scripts\run-tests-stability.ps1 -NoBuild:

Tests: CopilotLaunchPolicyEnvelopeTests,CopilotWrapperPackagingTests,ProcessRunnerModalSafetyTests,MxcIntegrationTests
Per run: 51 executed, 51 passed, 0 failed
Combined: 510 executions, 510 passed
No .dmp or dump-named artifacts were produced.
Final worktree has no staged or unstaged changes.

Verification result

Status: ✅ Verified
Issue: #1591 — validator smoke launch of malformed copilot.exe hangs Install.Tests
Fix commit: 7a255d72d782d6f108b55b64f75f7b25ed99f6c4

Commits on branch batch/1592-and-1-more (base github/features fb6091c8):

7a255d72 Fix #1591: fail malformed Copilot smoke quickly
27ba2ce8 Fix #1592: serialize policy launch-root security

This verification ran in an isolated, read-only pass: the issue body/comments were read as claims only, and the verdict is derived from independently inspecting the full diff, the resulting source, and independently re-run tests.

Contract assessment

Malformed/bad-image smoke fails promptly, noninteractively, bounded below the 90 s blame timeout, clear diagnostic — ✅ Assert-CopilotRuntimePayload.ps1 now runs an in-process PEReader preflight and requires the ExecutableImage COFF characteristic before native launch; malformed/non-executable input maps to the existing Bundled copilot.exe could not be launched: invalid Windows executable image … diagnostic. The GitHub-step launch is bounded by the central 30 s ValidatorProcessTimeout (CreateValidatorInvocationOptions), asserted < 90 s by InvokeValidatorAsGitHubStep_UsesValidatorTimeout_NotLongRunningTimeout.
Hard-error UI suppression correctly scoped, concurrency-safe, restored on all paths; real valid smoke stays meaningful — ✅ The real smoke launch wraps only itself in thread-scoped GetThreadErrorMode/SetThreadErrorMode with SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX, OR-ed onto the existing flags and restored to the exact prior value in finally (both success and failure paths). No process-global SetErrorMode is mutated, so parallel xUnit hosts cannot race (the Windows child-process tests can hang on unsuppressed STATUS_DLL_INIT_FAILED dialog #1514 rejection). The preflight does not short-circuit valid images: RuntimePayload_GitHubFinalCommand_AcceptedNativeExitIsClean and …_UnexpectedNativeExitFails still launch a real executable (whoami.exe) and assert the server-mode outcome.
Unexpected exit/output/launch error still fail closed; process-tree cleanup; no weakened provenance/hash/payload validation — ✅ The if ($startupExitCode -ne 1 -or output -notmatch 'SDK server mode requires --server or --headless') { throw } check is unchanged (…_UnexpectedNativeExitFails proves it). Provenance/fingerprint/hash/tamper checks are untouched by the diff and still pass (…_MissingPreparedFingerprint/_WrongPreparedFingerprint/_TamperedPreparedPath/_TamperedPreparedHash/_PreparedPropertiesWithoutTestOptIn/_ProductionBuildRefusesPreparedOverride). Contained-runner KillOnClose=KillTree + TerminateJobObject cleanup is retained.
Tests exercise real child pwsh/native launch, fail on the base, no sleeps/polling — ✅
- RuntimePayload_GitHubFinalCommand_NonExecutablePeFailsBeforeLaunch clears the PE ExecutableImage bit of a real payload and asserts non-zero exit + both copilot.exe could not be launched and invalid Windows executable image (a diagnostic that exists only in the fix → fails on base).
- RuntimePayload_GitHubFinalCommand_NativeLaunchFailureFails (the originally-hanging test) now completes deterministically via the preflight and 30 s bound.
- RuntimePayload_GitHubFinalCommand_NativeLaunchFailure_CompletesRepeatedlyWithoutHang runs 4 concurrent validator pwsh processes and asserts direct job containment + ActiveJobProcessesAfterCleanup == 0.
- ProcessRunner_BadImageGrandchildLaunch_DoesNotBlockOnHardError performs a real CreateProcess of a malformed binary through the modal-safe probe and asserts no hang, Win32 193/216, CreateProcess failure stage, and full handle/job/output cleanup.
- The retained event-driven MxcRuntimePayloadTests.InvokePowerShellAsync_StalledValidator_TimesOutAndCleansProcessTreeAndArtifacts proves timeout → whole-tree kill → stream drain → handle release → artifact cleanup using a ManualTimeoutTimeProvider + EventWaitHandle/TaskCompletionSource (no sleeps/polling). This substitutes for the design's proposed InvokeAsync_ValidatorChildStall_… and genuinely covers that behaviour.

The ProcessRunnerFailureResult refactor in the probe correctly treats process/thread handles as released when CreateProcess never succeeded, and leaves the CreateProcess-succeeded failure scenarios (Configure/Assign/Resume) computing cleanup exactly as before. No disabled tests, TODOs, or timing dependence. No cross-commit interaction with #1592 (disjoint files; #1591 does not modify ProcessRunner.cs product code).

Outstanding gaps

None — all criteria satisfied.

needs-slow-tests

Not required. The diff touches neither the Git repository layer nor the Repository/FileSystem abstraction layers (Repository|FileSystem|\.git matches nothing). The relevant classes run in the fast suite.

Tests

Independently re-run via scripts\run-tests.ps1 only (fast mode), union across both batched issues:
CopilotLaunchPolicyEnvelopeTests, CopilotWrapperPackagingTests, ProcessRunnerModalSafetyTests, MxcRuntimePayloadTests

Single run: 65 executed / 65 passed / 0 failed (Install.Tests 30, Data.Core.Tests 15, Llm.Core.Tests 20).
10-run stability (run-tests-stability.ps1 -NoBuild): 10/10 runs passed, 650 executions / 650 passed.
Build succeeded (no error/Build FAILED); no .dmp/hang-sequence artifacts produced; worktree clean afterward.
All #1591 tests above (packaging, bad-image grandchild, stalled-validator) executed and passed in every run.

Note: the earlier batch stability comment listed MxcIntegrationTests as a target, but no test's fully-qualified name contains MxcIntegrationTests (the file's classes are MxcRuntimePayloadTests, ReleasePackagingTests, etc.), so that name matched 0 tests and the retained StalledValidator coverage was not actually exercised there. This verification re-targeted the correct class MxcRuntimePayloadTests (14 tests, incl. StalledValidator), which passed on all 10 runs. This is a stability-command naming observation only — the test/coverage exists and passes, so it is not an outstanding gap.

Fix #1591: fail malformed Copilot smoke quickly

Checked in

Fix commit: 7a255d72d782d6f108b55b64f75f7b25ed99f6c4
Branch fast-forwarded: features -> 7a255d72d782d6f108b55b64f75f7b25ed99f6c4

Check-in gates: build passed; changed-test stability 10/10 (650/650); fast gate 7,511/7,511; no dumps; submodules clean.

Bug: validator smoke launch of malformed copilot.exe hangs Install.Tests (unsuppressed bad-image hard error; 10-min timeout exceeds 90s blame) #1591

Description

Summary

Root Cause

1. Bad-image hard error at grandchild launch is not suppressed (the stall)

2. This invocation uses the 10-minute timeout, exceeding the 90-second blame-hang timeout (why it becomes a host crash instead of a bounded failure)

3. The failure branch is exercised through a nondeterministic OS launch (why even a bounded fix is not enough on its own)

Proven vs. uncertain

Current status vs. #1585–#1590 / #1593

Occurrence evidence

Affected Files

Design / Fix

A. Bound the invocation to the validator timeout (required)

B. Make the failure branch deterministic (required)

C. Defense-in-depth in the contained runner (recommended)

Considered / Background (not chosen alone)

Expected Tests

Dependencies / Related Issues

Implementation Plan

Activity

JoshuaRowePhantom commented on Sep 21, 2026

Design

Root cause

Approach

Tests first

Alternatives considered

Files

JoshuaRowePhantom commented on Sep 21, 2026

Implementation notes

What changed

Deviations from proposed design

Scope changes

Validation

Files changed

JoshuaRowePhantom commented on Sep 21, 2026

Batch stability evidence

JoshuaRowePhantom commented on Sep 21, 2026

Verification result

Contract assessment

Outstanding gaps

needs-slow-tests

Tests

JoshuaRowePhantom commented on Sep 21, 2026

Checked in

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions