Skip to content

Flaky test: SkillLibraryViewTests scroller-style expectation depends on suite order #29

Description

@sarthakagrawal927

Flaky test

SkillLibraryViewTests.rendersLibraryAtSupportedWidthsWithoutLiveData() (Tests/ContextDaddyTests/SkillLibraryViewTests.swift:119):

Expectation failed: NSScroller.preferredScrollerStyle == (scrollbarPreference == "Always" ? .legacy : .overlay)

This is the one-off failure seen on main at c6bd95b that passed on rerun.

Evidence (local Mac, 2026-10-09, swift test --skip DesignSnapshotTests, same command as CI)

  • Full suite: failed 5 of 5 runs, always this single expectation (154 tests otherwise pass).
  • swift test --filter rendersLibraryAtSupportedWidthsWithoutLiveData: passed 3 of 3.
  • Skipping PluginCountQualificationTests, IssueOneAcceptanceTests and UsageAllowanceViewTests: passed.
  • --no-parallel still failed, so this is not only a race.

Cause

The test's own comment says AppKit caches the scroller style per process. The test sets AppleShowScrollBars and then asserts the style, but any suite that builds an NSHostingView or NSScrollView first (those three suites) fixes the style, so the assertion depends on which suite runs first and on the machine's pointing-device default. A clean CI runner mostly lands on legacy, so it usually passes.

Suggested fix (not applied, since it is a test-design decision)

Set AppleShowScrollBars before any AppKit view is created in the process, for example from a shared test bootstrap, or drop the process-global assertion and verify the layout against the style actually in effect. The environment variable CONTEXTDADDY_TEST_SCROLLBARS already exists for per-style runs.

Activity

  1. sarthakagrawal927 commented on Oct 9, 2026

    @sarthakagrawal927
    MemberAuthor

    Fixed by #30 (squash 06ac896). The test now lays out against the scroller style AppKit resolved, checks the hosted scroll view uses that style and that legacy scrollers reserve their width, and keeps the strict requested-style check for explicit CONTEXTDADDY_TEST_SCROLLBARS runs. Local: 5 of 5 full swift test --skip DesignSnapshotTests runs pass (26 + 154 tests); PR CI (test, shared_candidate, site) passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions