Skip to content

CI: add Playwright e2e job for dashboard smoke + optional screenshot artifact #215

Closed

Description

@flyingrobots

PR #176 added Playwright e2e coverage (e2e/session-dashboard.spec.ts) and an optional docs screenshot regen path gated by ECHO_CAPTURE_DASHBOARD_SCREENSHOT=1.

Right now .github/workflows/ci.yml runs Rust fmt/clippy/tests/guards only; it does not run Playwright, so:

  • e2e regressions won't be caught in PRs
  • screenshot drift is only caught manually

Proposal

Add a Playwright job to CI, with a cautious default to avoid flakiness / long runtimes:

Option A (recommended): PR smoke + artifacts

  • Run pnpm exec playwright test e2e/session-dashboard.spec.ts on PRs.
  • Upload Playwright report + screenshot attachments as artifacts.
  • Keep ECHO_CAPTURE_DASHBOARD_SCREENSHOT off in CI (don't mutate repo), but still attach the PNG to the CI artifacts so humans can compare.

Option B: manual / nightly screenshot regen

  • Add a workflow_dispatch job that runs the same test with ECHO_CAPTURE_DASHBOARD_SCREENSHOT=1 and uploads the produced docs/assets/wvp/session-dashboard.png as an artifact (or opens a PR via bot if desired).

Implementation notes

  • Node is already in-repo (package.json, pnpm-lock.yaml), so CI job should use pnpm with cache.
  • Install browsers: pnpm exec playwright install --with-deps on ubuntu runners (or install on macOS).
  • Keep timeouts reasonable and avoid parallel test runs if flake observed.

Acceptance criteria

  • New CI workflow job runs Playwright e2e (at least e2e/session-dashboard.spec.ts) on PRs.
  • Job uploads artifacts (Playwright report and attached screenshot).
  • Job is reliable (no persistent flakes) or can be label-gated if needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttoolingTooling/CI/CLI

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions