-
Notifications
You must be signed in to change notification settings - Fork 21
chore(tests): visual regression tests for post-header
#6826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces comprehensive visual regression testing for the post-header component using Playwright. The implementation adds test infrastructure, helper utilities, and HTML fixtures for testing 7 header variants (portal, microsite, jobs, and onepager - each with logged-in/out states) across 3 breakpoints (desktop, tablet, mobile) and 3 browsers (Chromium, Firefox, WebKit).
Key changes:
- Adds Playwright test framework with configuration for visual regression testing
- Creates reusable test helpers for interacting with header components (megadropdowns, language menus, user menus, burger menus)
- Implements consolidated test suite that efficiently tests shared features once while testing variant-specific features separately
- Sets up GitHub Actions workflow with snapshot caching and optimization
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
packages/components/playwright.config.ts |
Playwright configuration with test directories, screenshot settings, and multi-browser setup |
packages/components/visual-tests/post-header.consolidated.spec.ts |
Main test suite implementing comprehensive visual regression tests for all header variants |
packages/components/visual-tests/helpers/header-test-helpers.ts |
Reusable helper functions for interacting with header components (hover, focus, open/close actions) |
packages/components/visual-tests/helpers/header-variants.config.ts |
Configuration defining feature sets for each header variant to enable selective testing |
packages/components/visual-tests/components/*.html |
HTML test fixtures for 7 header variants (onepager, microsite logged-in/out, jobs logged-in/out, portal logged-in/out) |
packages/components/package.json |
Adds test:visual scripts and Playwright/http-server dependencies |
packages/components/tsconfig.json |
Includes Playwright types and visual-tests directory in compilation |
packages/components/tsconfig.eslint.json |
Extends ESLint configuration to include visual test files |
packages/components/stencil.config.ts |
Excludes visual-tests from Jest coverage |
packages/components/.gitignore |
Ignores test results and generated screenshot artifacts |
package.json |
Adds convenience scripts at root level for running visual tests |
.github/workflows/visual-tests.yaml |
GitHub Actions workflow for automated visual regression testing with caching |
pnpm-lock.yaml |
Updates lock file with Playwright and http-server dependencies |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
packages/components/visual-tests/components/post-header-jobs-loggedout.html
Outdated
Show resolved
Hide resolved
| - name: Optimize PNG screenshots | ||
| if: | | ||
| (github.ref == 'refs/heads/main' && steps.snapshot-cache.outputs.cache-hit != 'true') || | ||
| github.event.inputs.update-snapshots == 'true' | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y pngquant | ||
| find packages/components/visual-tests/__screenshots__ -name "*.png" -exec pngquant --force --ext .png --quality 80-95 {} \; | ||
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The --quality 80-95 setting for pngquant is quite aggressive and may degrade image quality significantly, potentially causing false positives in visual regression tests due to compression artifacts. Playwright's screenshot comparison is pixel-perfect by nature, and lossy compression can introduce differences between baseline and test screenshots even when the UI hasn't changed. Consider either:
- Using lossless PNG optimization tools like
optipngoroxipnginstead - Using a higher quality range like
--quality 95-100to minimize compression artifacts - Skipping PNG optimization entirely for visual regression test baselines
| - name: Optimize PNG screenshots | |
| if: | | |
| (github.ref == 'refs/heads/main' && steps.snapshot-cache.outputs.cache-hit != 'true') || | |
| github.event.inputs.update-snapshots == 'true' | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y pngquant | |
| find packages/components/visual-tests/__screenshots__ -name "*.png" -exec pngquant --force --ext .png --quality 80-95 {} \; |
| <script src="../../dist/post-components/post-components.esm.js" type="module"></script> | ||
| </head> | ||
| <body> | ||
| <!-- One Pager Variant --> |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment "" is misleading in multiple files. This comment appears in files for portal, microsite, and jobs variants, not just the onepager variant. Either remove the comment or update it to match the actual variant being tested.
| <!-- One Pager Variant --> |
packages/components/visual-tests/components/post-header-jobs-loggedin.html
Outdated
Show resolved
Hide resolved
| <meta charset="UTF-8" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> | ||
| <meta name="design-system-settings" data-post-icon-base="/www/assets/icons" /> | ||
| <title>Post Header Portal Logout Component Visual Regression Test</title> |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The title tag is inconsistent with the other HTML files. Other variants use "Post Header Component Visual Regression Test" while this one says "Post Header Portal Logout Component Visual Regression Test". Consider using a consistent title format across all variants, such as "Post Header [variant] Visual Regression Test".
| <title>Post Header Portal Logout Component Visual Regression Test</title> | |
| <title>Post Header Portal Logged Out Visual Regression Test</title> |
|
|
||
| expect: { | ||
| toHaveScreenshot: { | ||
| threshold: 0.2, |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A screenshot comparison threshold of 0.2 (20%) is very high for visual regression testing. This means screenshots can differ by up to 20% and still pass the test, which could allow significant visual regressions to go undetected. Industry standard thresholds are typically 0.01-0.05 (1-5%). Consider lowering this threshold to 0.02-0.05 to catch meaningful visual regressions while allowing for minor anti-aliasing differences across browsers.
| threshold: 0.2, | |
| threshold: 0.02, |
| uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 | ||
| with: | ||
| path: packages/components/visual-tests/__screenshots__ | ||
| key: visual-snapshots-main-${{ hashFiles('packages/components/src/**/*.{tsx,ts,scss}') }} |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The cache key on line 50 doesn't include the visual test files themselves. If you modify the HTML test fixtures, helper functions, or test spec files without changing component source code, the cache will still be used, potentially causing stale baselines. Consider including the visual test files in the hash:
key: visual-snapshots-main-${{ hashFiles('packages/components/src/**/*.{tsx,ts,scss}', 'packages/components/visual-tests/**/*.{html,ts}') }}| key: visual-snapshots-main-${{ hashFiles('packages/components/src/**/*.{tsx,ts,scss}') }} | |
| key: visual-snapshots-main-${{ hashFiles('packages/components/src/**/*.{tsx,ts,scss}', 'packages/components/visual-tests/**/*.{html,ts}') }} |
packages/components/.gitignore
Outdated
| +**/visual-tests/__screenshots__/ | ||
|
|
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 39 appears to be redundant with lines 36-38. Lines 36-38 already ignore *-actual.png, *-diff.png, and *-expected.png files in __screenshots__ directories. Line 39 attempts to ignore the entire __screenshots__/ directory tree, but the patterns on lines 36-38 are more specific. Consider whether you want to ignore only the generated comparison images (lines 36-38) or the entire __screenshots__/ directory (line 39). If you want to commit baseline snapshots to git, keep lines 36-38 and remove line 39. If you don't want to commit any screenshots, keep only line 39.
| +**/visual-tests/__screenshots__/ |
| const menu = document.querySelector<any>('post-language-menu'); | ||
| const button = (menu?.shadowRoot as any)?.querySelector('button'); |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using any type circumvents TypeScript's type safety. Consider using proper types or unknown instead. For example:
const menu = document.querySelector('post-language-menu');
const button = menu?.shadowRoot?.querySelector('button');This maintains type safety without needing to cast to any.
…oggedout.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…oggedin.html Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…wisspost/design-system into visual-regression-tests-header
|




📄 Description
Please include a summary of the changes made in this PR.
🚀 Demo
If applicable, please add a screenshot or video to illustrate the changes.
🔮 Design review
📝 Checklist