Skip to content

Fix weekly test-windows/test-macos action#413

Closed
yamcodes wants to merge 1 commit intomainfrom
consistently-use-playwright-image
Closed

Fix weekly test-windows/test-macos action#413
yamcodes wants to merge 1 commit intomainfrom
consistently-use-playwright-image

Conversation

@yamcodes
Copy link
Owner

@yamcodes yamcodes commented Nov 22, 2025

  • Changed the test runners for Windows and macOS to run on Ubuntu.
  • Introduced a Playwright container for consistent browser testing.
  • Added environment variable setup for Turbo SCM in pull request workflows.
  • Removed unnecessary caching steps and Playwright browser installation commands.

Summary by CodeRabbit

  • Chores
    • Updated continuous integration testing infrastructure to improve consistency and efficiency of test execution across different environments. Tests now run within a standardized containerized setup to ensure reliable and reproducible results.

✏️ Tip: You can customize this high-level summary in your review settings.

- Changed the test runners for Windows and macOS to run on Ubuntu.
- Introduced a Playwright container for consistent browser testing.
- Added environment variable setup for Turbo SCM in pull request workflows.
- Removed unnecessary caching steps and Playwright browser installation commands.
@changeset-bot
Copy link

changeset-bot bot commented Nov 22, 2025

⚠️ No Changeset found

Latest commit: fb541d1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel
Copy link

vercel bot commented Nov 22, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
arkenv Ready Ready Preview Comment Nov 22, 2025 5:57am

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 22, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

The test workflow is restructured to run on ubuntu-latest with a Playwright container image instead of native Windows and macOS runners. Manual setup steps for pnpm, Node.js, and Playwright browsers are removed. A dependency on a resolve-playwright-version job is introduced to populate the container image tag, and PR environment variable propagation is added.

Changes

Cohort / File(s) Change Summary
GitHub Actions Workflow Restructuring
.github/workflows/test.yml
Replaced test-windows and test-macos jobs with containerized ubuntu-latest jobs using mcr.microsoft.com/playwright container image. Removed pnpm setup, Node.js setup, and browser installation steps. Added resolve-playwright-version job dependency, container configuration with --user 1001, and Turbo SCM base/head environment variable setup for PRs. Core test/build/e2e steps preserved.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Verify the Playwright container image version resolution mechanism works correctly
  • Confirm that removing manual setup steps doesn't break the containerized environment (all dependencies must be pre-installed in the container)
  • Check job dependency ordering and ensure resolve-playwright-version output is correctly consumed

Possibly related PRs

Suggested labels

github_actions, infra

Poem

🐰 Tests in containers, no runners to keep,
Ubuntu with Playwright, running so deep,
Version resolved in a job up above,
Workflows now streamlined—a gift from this dove! 🎁

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Title check ❓ Inconclusive The PR title 'Fix weekly test-windows/test-macos action' is vague and does not clearly convey the main change. The actual change involves reworking Windows/macOS test runners to use a containerized Playwright approach on Ubuntu, which is a significant architectural shift beyond a simple 'fix'. Clarify the title to better reflect the main change, such as 'Use Playwright container for Windows/macOS test workflows' or 'Migrate test-windows/test-macos to containerized Playwright environment'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch consistently-use-playwright-image

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions bot added the github actions Pull requests that update GitHub Actions code label Nov 22, 2025
@yamcodes yamcodes added infra Infrastructure related issue or pull request and removed github actions Pull requests that update GitHub Actions code labels Nov 22, 2025
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 128 to 131
if: github.event_name == 'schedule' || contains(join(github.event.pull_request.labels.*.name, ','), 'windows')
runs-on: windows-latest
needs: resolve-playwright-version
runs-on: ubuntu-latest
strategy:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Windows/macOS jobs no longer exercise target OSes

The test-windows job now declares runs-on: ubuntu-latest and runs inside the Playwright Linux container (same pattern at lines 164‑167 for test-macos), so even when the windows/macos labels are applied or on the scheduled run, we only exercise Linux. That removes the Windows and macOS coverage this workflow used to provide (there are no other runs-on: windows/macos jobs in the repo), so platform-specific regressions will slip through while the jobs still appear to have validated those platforms.

Useful? React with 👍 / 👎.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
.github/workflows/test.yml (1)

102-106: Consider extracting duplicated Turbo SCM setup into a reusable workflow or pattern.

The "Set Turbo base/head for PRs" step is duplicated identically across test-e2e, test-windows, and test-macos jobs. This violates DRY and creates maintenance burden if the setup logic needs to change.

Options:

  1. Extract into a reusable workflow (.github/workflows/turbo-setup.yml)
  2. Create a composite action for Turbo SCM setup
  3. Document this as a standard pattern for all jobs that need Turbo PR context

Also applies to: 142-146, 178-182

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 485da8d and fb541d1.

📒 Files selected for processing (1)
  • .github/workflows/test.yml (2 hunks)
🔇 Additional comments (3)
.github/workflows/test.yml (3)

127-137: Clarify intent: job names imply OS-specific testing but now run in a container.

test-windows and test-macos jobs no longer run on native Windows/macOS runners; they now run on ubuntu-latest within a Playwright container. This is a significant semantic change:

  • If the goal is to test on actual Windows/macOS OS, this breaks that requirement. You'll only be testing the Ubuntu-based container behavior.
  • If the goal is consistent Playwright browser testing, the naming should be updated to reflect that these are now containerized tests, not OS-specific tests (e.g., test-browser-windows or test-playwright-windows).

Verify that removing native OS runners doesn't hide OS-specific issues (e.g., path handling, platform-specific test logic, or browser behavior differences).


95-98: Approve: Consistent Playwright container setup across jobs.

The container configuration is consistently applied to all Playwright-dependent jobs (test-e2e, test-windows, test-macos) with matching image tags and user permissions. This is a solid approach for standardizing the test environment.

However, verify that the --user 1001 container option doesn't cause unexpected permission issues:

  • Does the container image already have this user configured?
  • Are artifact upload permissions (lines 121-125, 156-160, 192-196) compatible with user 1001?

You can verify with a quick test workflow run to confirm artifact uploads succeed.

Also applies to: 135-137, 171-173


67-86: The Playwright version extraction is working correctly and requires no changes.

The verification confirms the awk command successfully extracts version 1.56.1 from pnpm-workspace.yaml and produces a valid semantic version. The parsing is functioning as intended with the current repository structure, and the extracted version is ready for use in container image tags.

@yamcodes yamcodes changed the title Consistently use Playwright image across job Fix weekly test-windows/test-macos action Nov 22, 2025
@yamcodes yamcodes closed this Nov 22, 2025
@yamcodes yamcodes deleted the consistently-use-playwright-image branch November 23, 2025 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra Infrastructure related issue or pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant