Flaky Windows CI: bookmark button test intermittently fails

[richardkmichael]

Problem

The "bookmark button" test is flaky on Windows CI. It fails on first attempt but passes on retry (2nd or 3rd attempt).

Symptoms

• Test: extension.spec.js → "Basics" → "bookmark button"
• Locator: [data-extension-bookmark]
• Error: element(s) not found after 10s timeout
• Behavior: Content script logs show button was added, but it disappears before test assertion

CI Evidence

• Run 20773484593: Windows passed on 3rd attempt (2 retries)
• Test results show extension-Basics-bookmark-button-chromium-retry1 artifacts

Analysis

The content script runs and logs "Bookmark button added", but the button is removed before Playwright can find it. This suggests:

1. React hydration timing: GitHub's React app re-renders the header area after our content script injects the button, removing it
2. Playwright waitFor timing: The test may not be waiting correctly for the button to stabilize
3. Windows-specific timing: Windows CI runners may have different performance characteristics

Diagnostic Logging Added

Commit 4eee383 added timing logs to help diagnose:

• [timestamp ms] init() readyState
• [timestamp ms] Creating button
• [timestamp ms] Bookmark button added to DOM
• [timestamp ms] Button was REMOVED from DOM (if removed)

Potential Fixes

1. Improve test waitFor: Use more robust waiting strategy

   // Instead of just toBeVisible, wait for stable state
   await expect(page.locator('[data-extension-bookmark]')).toBeVisible();
   await page.waitForTimeout(100); // Let React settle
   await expect(page.locator('[data-extension-bookmark]')).toBeVisible();

2. Re-insert button on removal: Make MutationObserver re-insert button if React removes it (masks the problem but works)

3. Wait for React hydration: Detect when GitHub's React app has finished hydrating before inserting

Environment

• Windows Server 2022 (windows-latest)
• Playwright 1.57.0
• Chromium headless

Labels

• bug
• flaky-test
• windows