Skip to content

Bug: flaky test -- ChatOutputHtmlModelTests HeadlessUnitTestSession crashes (recurrence of #815) #1012

Closed
Closed
Bug: flaky test -- ChatOutputHtmlModelTests HeadlessUnitTestSession crashes (recurrence of #815)#1012
Assignees
JoshuaRowePhantom
Labels
bugSomething isn't workingverified-locallyImplementation has been verified locally

Description

@JoshuaRowePhantom
JoshuaRowePhantom
opened on Jul 15, 2026

Unrelated test failure -- observed during fix for #1006

Failing tests:

  • Phantom.Workspaces.Agent.Gui.Tests.ChatOutputHtmlModelTests.Regression_BugD_OnInsert_UsesContainerPath_NotLoadAfterHardcode
  • Phantom.Workspaces.Agent.Gui.Tests.ChatOutputHtmlModelTests.HistoryLoad_NewestChunkInsertedFirst

Failure message:

[Test Case Cleanup Failure]: System.InvalidOperationException : The calling thread cannot access this object because a different thread owns it.
   at Avalonia.Threading.Dispatcher.<VerifyAccess>g__ThrowVerifyAccess|17_0()
   at Avalonia.Threading.Dispatcher.VerifyAccess()
   at Avalonia.Rendering.DefaultRenderLoop.Add(IRenderLoopTask i)
   at Avalonia.Rendering.Composition.Server.ServerCompositor..ctor(...)
   at Avalonia.Rendering.Composition.Compositor..ctor(...)
   at Avalonia.Headless.AvaloniaHeadlessPlatform.Initialize(AvaloniaHeadlessPlatformOptions opts)
   at Avalonia.Headless.AvaloniaHeadlessPlatformExtensions.<>c__DisplayClass0_0.<UseHeadless>b__0()
   at Avalonia.AppBuilder.SetupUnsafe()
   at Avalonia.Headless.HeadlessUnitTestSession.EnsureIsolatedApplication()

No dump files were found.

Prior occurrence

This is a recurrence of #815, which was closed on 2026-07-14 (the day before this failure).

Root cause hypothesis

The Avalonia HeadlessUnitTestSession.EnsureIsolatedApplication() is called during test cleanup from a thread that does not own the Dispatcher that was used to set up the headless platform. When multiple tests run in parallel (or when xUnit reuses threads across test classes), the compositor constructor calls DefaultRenderLoop.Add which enforces thread affinity via Dispatcher.VerifyAccess(), causing the cleanup to throw. The fix from #815 did not fully prevent this from reoccurring.

Proposed solution

  1. Ensure HeadlessUnitTestSession always initialises and tears down on the same dedicated STA/dispatcher thread — use a thread-local or session-scoped dispatcher that persists for the lifetime of the isolated application.
  2. If the session's _dispatchTask faults, swallow the cleanup exception so it does not surface as a [Test Case Cleanup Failure] that hides the actual test result.
  3. Consider adding a test-parallelism limit on Phantom.Workspaces.Agent.Gui.Tests to reduce the chance of thread-affinity races ([assembly: CollectionBehavior(MaxParallelThreads = 1)]).

Activity

  1. JoshuaRowePhantom
    added
    bugSomething isn't working
    on Jul 15, 2026
  2. JoshuaRowePhantom
    self-assigned this
    on Jul 16, 2026

  3. JoshuaRowePhantom commented on Jul 16, 2026

    @JoshuaRowePhantom
    JoshuaRowePhantom
    on Jul 16, 2026
    OwnerAuthor

    Design

    Root cause (why #815 recurred)

    The crash is InvalidOperationException: The calling thread cannot access this object because a different thread owns it thrown from DefaultRenderLoop.Add → ServerCompositor..ctor → AvaloniaHeadlessPlatform.Initialize → AppBuilder.SetupUnsafe → HeadlessUnitTestSession.EnsureIsolatedApplication. That code path only runs under PerTest isolation, where Avalonia tears down and rebuilds the Application/Dispatcher/compositor/DefaultRenderLoop on every test. The repeated Initialize is the intermittent crash surface (also the source of the TextInputMethodManager duplicate-key variant).

    The #815 fix "removed [assembly: AvaloniaTestIsolation(PerTest)]" believing the default was shared-app. It is not. Decompiling HeadlessUnitTestSession.GetOrStartForAssembly (Avalonia 12.1.0):

    var isolationLevel = assembly.GetCustomAttribute<AvaloniaTestIsolationAttribute>()?.IsolationLevel
    ?? AvaloniaTestIsolationLevel.PerTest;   // default is PerTest!

    So removing the attribute left PerTest in force. Phantom.Workspaces.Agent.Gui.Tests has been rebuilding Avalonia per test ever since — exactly the crash-prone path. The STA-scheduler mitigation from #944 does not help because AvaloniaTestRunner re-dispatches every test onto the session's own single _dispatchTask thread regardless; the wrapper's STA thread only blocks and waits.

    The existing regression guard AvaloniaXUnitSetup_AgentGuiTests_DoesNotDeclarePerTestIsolation is vacuously broken and never caught this:

    • it resolves AvaloniaTestIsolationAttribute from Avalonia.Headless.XUnit (wrong — the type lives in Avalonia.Headless), so isolationAttrType is always null and the body is skipped;
    • it reads a Level property (actual name is IsolationLevel);
    • it only asserts "not PerTest", but a missing attribute defaults to PerTest and still passes.

    Chosen fix

    Explicitly declare shared-app isolation on the affected assembly so EnsureSharedApplication builds Avalonia exactly once and never rebuilds — structurally eliminating the repeated Initialize/DefaultRenderLoop crash surface (a real determinism fix, not a retry):

    [assembly: AvaloniaTestIsolation(AvaloniaTestIsolationLevel.PerAssembly)]

    in Phantom.Workspaces.Agent.Gui.Tests\AvaloniaXUnitSetup.cs.

    And rewrite the broken guard test to encode the true invariant: the assembly must declare AvaloniaTestIsolationLevel.PerAssembly (fails when the attribute is absent / defaults to PerTest), using the correct Avalonia.Headless.AvaloniaTestIsolationAttribute.IsolationLevel.

    Alternatives considered

    Files to change

    • Phantom.Workspaces.Agent.Gui.Tests\AvaloniaXUnitSetup.cs — add PerAssembly isolation attribute.
    • Phantom.Workspaces.Agent.Gui.Tests\PhantomAvaloniaFactTests.cs — replace the vacuous guard test with a correct PerAssembly assertion.

  4. JoshuaRowePhantom commented on Jul 16, 2026

    @JoshuaRowePhantom
    JoshuaRowePhantom
    on Jul 16, 2026
    OwnerAuthor

    Implementation notes

    Commit: 552bd19

    Root cause

    HeadlessUnitTestSession.GetOrStartForAssembly defaults to AvaloniaTestIsolationLevel.PerTest when no AvaloniaTestIsolationAttribute is present. The #815 fix removed the PerTest attribute assuming that switched to shared-app, but it did not — the assembly kept rebuilding the Avalonia Application/Dispatcher/compositor on every test via EnsureIsolatedApplication -> AppBuilder.SetupUnsafe -> AvaloniaHeadlessPlatform.Initialize, which is the intermittent DefaultRenderLoop.Add -> Dispatcher.VerifyAccess() crash surface. The old regression guard was vacuous (looked up the attribute in the wrong assembly/namespace and a non-existent Level property), so it never caught the regression.

    Fix

    • Phantom.Workspaces.Agent.Gui.Tests\AvaloniaXUnitSetup.cs: added [assembly: AvaloniaTestIsolation(AvaloniaTestIsolationLevel.PerAssembly)]. Now EnsureSharedApplication builds Avalonia exactly once and reuses it, structurally eliminating the per-test rebuild race.
    • Phantom.Workspaces.Agent.Gui.Tests\PhantomAvaloniaFactTests.cs: replaced the broken guard with AvaloniaXUnitSetup_AgentGuiTests_DeclaresPerAssemblyIsolation, asserting the assembly declares PerAssembly isolation (fails if the attribute is absent / defaults to PerTest).

    Deviations from proposed design

    None.

    Scope changes

    Scoped to Phantom.Workspaces.Agent.Gui.Tests (the assembly containing the failing ChatOutputHtmlModelTests). The identical latent default also affects Phantom.Workspaces.Tests and Phantom.Workspaces.Gui.Shared.Tests; not changed here to avoid unrelated blast radius. Worth a follow-up.

    Verification

    • Build: 0 error lines.
    • Full Phantom.Workspaces.Agent.Gui.Tests assembly: 625/625 passed (no PerAssembly state-leak regressions).
    • Stability: ChatOutputHtmlModelTests 30/30 green.

    Files changed

    • Phantom.Workspaces.Agent.Gui.Tests\AvaloniaXUnitSetup.cs
    • Phantom.Workspaces.Agent.Gui.Tests\PhantomAvaloniaFactTests.cs

  5. JoshuaRowePhantom commented on Jul 16, 2026

    @JoshuaRowePhantom
    JoshuaRowePhantom
    on Jul 16, 2026
    OwnerAuthor

    Verification result

    Status: ✅ Verified

    Commits checked:

    552bd194 Fix #1012: use PerAssembly isolation for Agent.Gui.Tests to stop headless rebuild crash

    Assessment

    • Structural fix, not a timing hack. The change adds [assembly: AvaloniaTestIsolation(AvaloniaTestIsolationLevel.PerAssembly)] to AvaloniaXUnitSetup.cs. This forces HeadlessUnitTestSession down the EnsureSharedApplication path so the Avalonia Application/Dispatcher/compositor is built exactly once, structurally removing the repeated AvaloniaHeadlessPlatform.Initialize -> DefaultRenderLoop.Add -> Dispatcher.VerifyAccess() crash surface. No Task.Delay/Thread.Sleep/retry/polling introduced.
    • Non-vacuous regression guard. AvaloniaXUnitSetup_AgentGuiTests_DeclaresPerAssemblyIsolation now resolves AvaloniaTestIsolationAttribute from the correct namespace (Avalonia.Headless, via the strongly-typed generic GetCustomAttribute<AvaloniaTestIsolationAttribute>()), asserts the attribute is present (Assert.NotNull — fails on the Bug: flaky test -- ChatOutputHtmlModelTests HeadlessUnitTestSession crashes (recurrence of #815) #1012 regression state where the attribute is absent and defaults to PerTest), and asserts IsolationLevel == PerAssembly using the correct property name. This fixes all three defects of the previous vacuous guard (wrong assembly/namespace, non-existent Level property, "not PerTest" that passed when the attribute was missing).
    • CODE-VERIFICATION criteria: No disabled/quarantined tests, no unresolved TODOs, no timing-dependent tests, no fixed dispatcher-pass pumping, no synchronous task blocking. Feature and regression-guard coverage present.
    • No filesystem/Git repository layers touched — fast suite is sufficient; needs-slow-tests not required.

    Determinism check

    • Stability: ChatOutputHtmlModelTests 25/25 consecutive passes (122 tests each run, Failed: 0). No .dmp files produced.
    • Regression guard AvaloniaXUnitSetup_AgentGuiTests_DeclaresPerAssemblyIsolation: 1/1 passed.

    Outstanding gaps

    None — all criteria satisfied.

    Tests

    Stability: ChatOutputHtmlModelTests — 25/25 runs passed (122 tests/run, Failed: 0)
Guard test: AvaloniaXUnitSetup_AgentGuiTests_DeclaresPerAssemblyIsolation — Executed: 1, Passed: 1, Failed: 0
  6. JoshuaRowePhantom
    added
    verified-locallyImplementation has been verified locally
    on Jul 16, 2026
  7. JoshuaRowePhantom
    added a commit that references this issue on Jul 16, 2026

    Fix #1012: use PerAssembly isolation for Agent.Gui.Tests to stop head…

    552bd19

  8. JoshuaRowePhantom commented on Jul 16, 2026

    @JoshuaRowePhantom
    JoshuaRowePhantom
    on Jul 16, 2026
    OwnerAuthor

    Checked in

    Commit: 552bd19
    Branch fast-forwarded: features -> 552bd19

  9. JoshuaRowePhantom
    closed this as completedon Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingverified-locallyImplementation has been verified locally

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions