chore(ci): auto-update Firefox versions and run firefox:beta in CI#33885
Draft
mschile wants to merge 8 commits into
Draft
chore(ci): auto-update Firefox versions and run firefox:beta in CI#33885mschile wants to merge 8 commits into
mschile wants to merge 8 commits into
Conversation
Extends the daily Update Browser Versions GitHub Action to also track Firefox stable and beta from Mozilla's product-details JSON, mirroring the existing Chrome automation, and wires a new driver-integration-tests-firefox-beta job into CI so we catch upcoming Firefox regressions before stable ships.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b141f46. Configure here.
…on downgrades Cypress's launcher only recognizes Firefox channels `stable`, `dev`, and `nightly`. A Mozilla "Beta" build (e.g. 152.0b1) is detected as the `dev` channel because its version-string regex matches `\S+b\S*`, so `--browser firefox:beta` would fail with browser-not-found. Rename the new anchor, job, and outputs from `*-beta` to `*-dev` to match the identifier Cypress actually accepts. Also replace the `!==` check for the Firefox dev version with a proper "newer only" comparator that parses both stable (`145.0.2`) and dev/beta (`152.0b1`) version strings, so the daily auto-update workflow can never open a downgrade PR if upstream regresses or we manually pin ahead.
…elease gate) The first run against Mozilla's Beta channel surfaced real upstream behavior changes — cookie header ordering in cookie_behavior.cy.ts and selectionchange event firing in focus_blur.cy.js — that are the exact kind of early-warning signal this job exists to catch. We want to keep seeing those failures, but they shouldn't block routine PR merges or production releases. Remove driver-integration-tests-firefox-dev from the requires lists of all-jobs-passed, verify-accessibility-results, and ready-to-release. The job still runs and posts a visible GitHub status; reviewers can inspect failures without being gated on a Mozilla Beta build.
cypress
 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Project |
cypress
|
| Branch Review |
mschile/peaceful-booth-b39218
|
| Run status |
|
| Run duration | 19m 09s |
| Commit |
|
| Committer | Matt Schile |
| View all properties for this run ↗︎ | |
| Test results | |
|---|---|
 |
4
|
 |
11
|
 |
1243
|
 |
0
|
 |
32997
|
| View all changes introduced in this branch ↗︎ | |
UI Coverage
64%
|
|
|---|---|
|
|
27
|
|
|
48
|
Accessibility
98.98%
|
|
|---|---|
|
|
0 critical
3 serious
1 moderate
0 minor
|
|
|
18
|
Tests for review
| Test | Artifacts | |||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| intercept blur methods correctly > focus |
| |||||||||||||||||||||||||||||||
| Test | Artifacts | |
|---|---|---|
| issue 28527 > fails and then retries and verifies about:blank is not displayed |
Test Replay
Screenshots
|
|
| Test | Artifacts | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ... > stops waiting when an xhr request is canceled |
| |||||||||||||||||||
| Test | Artifacts | |
|---|---|---|
| issue 28527 > fails and then retries and verifies about:blank is not displayed |
Screenshots
|
|
| Test | Artifacts | |
|---|---|---|
| ... > stops waiting when an fetch request is canceled |
Test Replay
|
|
| Test | Artifacts | |
|---|---|---|
| issue 28527 > fails and then retries and verifies about:blank is not displayed |
Test Replay
Screenshots
|
|
The first 5 flaky specs are shown, see all 11 specs in Cypress Cloud.
…ooth-b39218 # Conflicts: # .circleci/src/pipeline/@pipeline.yml
![cypress-bot[bot]](https://avatars.githubusercontent.com/in/24384?s=60&v=4){kind=small}
AtofStryker
reviewed
May 22, 2026
Collaborator
AtofStryker
left a comment
AtofStryker
left a comment
There was a problem hiding this comment.
So when Firefox releases the next version, we should see a PR created to bump the version, correct?
cypress-bot
Bot
dismissed
their stale review
Cursor Bugbot risk assessment is no longer Low Risk. Auto-approval dismissed; manual review required.
…m dev to beta Cypress's launcher previously only recognized Firefox channels `stable`, `dev`, and `nightly`. Mozilla's Beta releases (e.g. `152.0b1`) happen to share the `\S+b\S*` version-string format with Firefox Developer Edition, so Beta builds were getting detected as `firefox:dev`. That worked, but the `dev` label was confusing because Mozilla has a literal Developer Edition product distinct from Beta. Add a new `firefox:beta` entry to packages/launcher/lib/known-browsers (same version regex and `firefox` binary fallback as `dev`; distinct `Firefox Beta` displayName and PPA-style `firefox-beta` binary). Also add the channel to the Darwin and Windows path mappings. The `dev` channel still works for actual Developer Edition installs, so this is purely additive. Update everything around the canary CI job to use the new name: - `firefox-beta-version` anchor (was `firefox-dev-version`) - `driver-integration-tests-firefox-beta` (was `…-firefox-dev`) - `firefox:beta` browser arg (was `firefox:dev`) - `latest_firefox_beta_version` workflow outputs (was `…_dev_version`) - `Update Firefox (beta) to X` PR description (was `(dev)`) Aligns naming with the existing `chrome:beta` convention while staying backwards-compatible with the existing `firefox:dev` channel.
Renamed for clarity per review feedback — the function already handled both Chrome-style dotted-numeric versions and Firefox's <main>b<N> beta format (strings without a b segment compare as if b were 0), so the Firefox-specific name was misleading.
Mozilla's stability ordering is Release → Beta → Developer Edition → Nightly — Developer Edition is built on the Beta channel with extra experimental features enabled, so it's strictly less stable than Beta. Reorder the firefox entries in known-browsers, darwin, and windows to match. For known-browsers this also flips auto-detection identity for ambiguous binaries (Mozilla's Beta installer reuses the firefox binary that the \\\S+b\\\S* regex catches in both channels): an auto-detected beta-format build now reports as firefox:beta rather than firefox:dev, which matches Mozilla's current channel naming. Explicit --browser firefox:dev still works for actual Developer Edition installs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Additional details
firefox-stable-versionin.circleci/src/pipeline/@pipeline.ymlwas being bumped by hand (unlike Chrome stable/beta and Chrome for Testing, which are auto-bumped daily by the Update Browser Versions GitHub Action). As a result Firefox drifted behind upstream, and we had no CI signal against Firefox beta at all.This PR:
firefox_versions.json— readingLATEST_FIREFOX_VERSIONandLATEST_FIREFOX_RELEASED_DEVEL_VERSION. The same workflow now opens a single PR when any of Chrome stable/beta, Chrome for Testing, Firefox stable, or Firefox dev changes.firefox-dev-versionanchor in.circleci/src/pipeline/@pipeline.yml(pinned to today's Mozilla Beta,152.0b1). Cypress's launcher identifies these builds as thefirefox:devchannel — Mozilla's "Beta" version-string format (152.0b1) matches Cypress's\S+b\S*regex — so the anchor, job, and--browserarg all usedevto match the channel Cypress actually accepts.driver-integration-tests-firefox-devjob intopull-request.ymland@main.yml. Unlikedriver-integration-tests-chrome-beta, this job is advisory-only — it runs and posts a visible GitHub status, but is intentionally not in theall-jobs-passed/verify-accessibility-results/ready-to-releaseaggregatorrequiresblocks. The first run against Mozilla Beta surfaced real upstream behavior changes (cookie header ordering incookie_behavior.cy.ts,selectionchangeevent firing infocus_blur.cy.js) — exactly the kind of early-warning signal we want — but those should not gate routine PR merges or production releases.compareFirefoxVersionshelper that orders both stable (145.0.2) and dev (152.0b1) version strings, so the daily workflow never opens a PR that lowers the pinned beta if Mozilla regresses or we manually pin ahead.A
system-tests-firefox-devwas intentionally not added in this first cut to keep CI minutes down — driver integration coverage gives the early-warning signal without doubling the system-test runtime. Easy to add later if we want it.Note
Medium Risk
Medium risk because it changes CI automation and adds a new Firefox Beta (
firefox:dev) test job plus new external version fetching logic, which could affect CI stability and maintenance PR generation.Overview
Extends the daily Update Browser Versions GitHub Action to also track and update pinned Firefox stable and Mozilla Beta (Cypress
firefox:dev) versions in.circleci/src/pipeline/@pipeline.yml, including branch naming/PR messaging updates and unit test coverage.Updates CircleCI to define a new
firefox-dev-versionanchor and rundriver-integration-tests-firefox-devagainst that pin; the job is wired into PR and main workflows but is explicitly excluded from merge/release gating aggregators as an advisory canary.Reviewed by Cursor Bugbot for commit d4a5219. Bugbot is set up for automated code reviews on this repo. Configure here.
Steps to test
workflow_dispatch. With the values currently pinned in this PR (Firefox stable145.0.2, Firefox dev152.0b1), the workflow should detect that Firefox stable is behind upstream and open a follow-up PR bumping it. Re-running on a clean state should be a no-op.driver-integration-tests-firefox-devjob appears inpull-requestand installs Firefox152.0b1successfully. The job is advisory — a red status on it should not fail theall-jobs-passedaggregator.yarn test-scripts scripts/unit/github-actions/update-browser-version-spec.js— 30 tests pass, including new Firefox cases (stable bump, dev bump, mixed updates, no-downgrade for both Firefox channels).bash scripts/pack-ci.sh --all --validate— packs and validates cleanly.How has the user experience changed?
No change. Internal CI / release-automation tooling only.
PR Tasks
cypress-documentation?type definitions?