Improve Playwright PR comment format #7728
Draft
+303
−26
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit improves the Playwright test results comments on PRs to be more concise and actionable:
1. Enhanced extract-playwright-counts.ts:
- Added interfaces for test results, locations, and attachments
- Implemented extractFailingTests() to recursively extract failing test details
- Now extracts test names, file paths, line numbers, errors, and trace paths
- Returns failingTests array in the JSON output
2. Updated pr-playwright-deploy-and-comment.sh:
- Made summary more concise (single line with counts)
- Added "Failed Tests" section showing each failing test with:
* Direct link to test source code on GitHub
* Browser configuration where it failed
* Direct link to Playwright trace viewer
- Moved browser-specific reports into a collapsible <details> section
- Reduced overall verbosity while keeping important info upfront
The new format makes it much easier for developers to:
- Quickly see which tests failed
- Jump directly to the failing test code
- Access the Playwright trace viewer (which few people knew existed)
Implements: https://www.notion.so/Implement-Improve-Playwright-PR-comment-format-2d16d73d36508129979ad74391bee39d
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
🎭 Playwright Test Results497 ✅ • 0 ✅ • 3 📊 Test Reports by Browser
|
🎨 Storybook Build Status✅ Build completed successfully! ⏰ Completed at: 12/22/2025, 08:56:08 PM UTC 🔗 Links🎉 Your Storybook is ready for review! |
Bundle Size ReportSummary
Category Glance Per-category breakdownApp Entry Points — 3.19 MB (baseline 3.19 MB) • ⚪ 0 BMain entry bundles and manifests
Graph Workspace — 996 kB (baseline 996 kB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration
Views & Navigation — 6.54 kB (baseline 6.54 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Panels & Settings — 295 kB (baseline 295 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
UI Components — 196 kB (baseline 196 kB) • ⚪ 0 BReusable component library chunks
Data & Services — 12.5 kB (baseline 12.5 kB) • ⚪ 0 BStores, services, APIs, and repositories
Utilities & Hooks — 1.41 kB (baseline 1.41 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Vendor & Third-Party — 9.1 MB (baseline 9.1 MB) • ⚪ 0 BExternal libraries and shared vendor chunks
Other — 3.44 MB (baseline 3.44 MB) • ⚪ 0 BBundles that do not match a named category
|
Add categorization of test failures by type (screenshot assertions, expectation failures, timeouts, and other) to help developers quickly understand what types of issues are occurring. Changes: - Add categorizeFailureType() function to detect failure types from error messages - Track failure type counts in TestCounts interface - Display "Failure Breakdown" section in PR comments when tests fail - Show counts for: screenshot, expectation, timeout, and other failures 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Replace sed with bash parameter expansion for /index.html removal (SC2001) - Remove unused trace_link variable (SC2034)
- Replace console.log with process.stdout.write for JSON output - Remove unused reportDir parameter from extractFailingTests function Fixes eslint(no-console) and oxc(only-used-in-recursion) warnings
🔧 Auto-fixes AppliedThis PR has been automatically updated to fix linting and formatting issues.
Changes made:
|
🔧 Auto-fixes AppliedThis PR has been automatically updated to fix linting and formatting issues.
Changes made:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves the Playwright test results comments on PRs to make them more concise and actionable for developers.
Changes
1. Enhanced Test Data Extraction (
extract-playwright-counts.ts)extractFailingTests()to recursively extract failing test details from the report structure2. Improved PR Comment Format (
pr-playwright-deploy-and-comment.sh)<details>section to reduce clutterExample Format
When all tests pass:
When tests fail:
Benefits
Testing
This PR should be tested by:
Related
🤖 Generated with Claude Code
┆Issue is synchronized with this Notion page by Unito