Skip to content

Commit 6d78642

Browse files
mydeaclaude
andcommitted
fix(ci): Improve flaky test issue deduplication
Strip more matrix/variant noise from job and test names so the same flaky test dedupes to a single issue instead of one per variant. Job names now also collapse (Node xx) and (n/m) shard groups, fold all Playwright bundle/esm build configs into one bucket, and drop E2E "-node-xx" app-name suffixes. Test names now also strip the [chromium] browser prefix, Playwright line:column locations (which drift on every file edit and reopened issues), the trailing [esm]/[cjs] module-format suffix, and the bare "esm/cjs >" describe segment so the suffix and describe-block forms converge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 4b05f69 commit 6d78642

1 file changed

Lines changed: 24 additions & 11 deletions

File tree

scripts/report-ci-failures.mjs

Lines changed: 24 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -19,31 +19,44 @@ import { readFileSync } from 'node:fs';
1919

2020
/**
2121
* Collapse matrix variants of a job name so the same test failing across the matrix dedupes to a
22-
* single issue instead of one per node/TS version. We strip only version-like parenthetical groups
23-
* — a bare number (e.g. node version) or a `TS x.y` bracket — leaving other parentheticals (e.g.
24-
* `(nextjs-app, 20)`) intact:
22+
* single issue instead of one per variant. We strip version-like parenthetical groups — a bare
23+
* number (node version), a `TS x.y` bracket, a `Node xx` bracket, or a `n/m` shard — and fold the
24+
* Playwright bundle/esm build configs and E2E `-node-xx` app suffixes into a single bucket, leaving
25+
* other parentheticals (e.g. `(nextjs-app, 20)`) intact:
2526
*
26-
* "Node (22) Integration Tests" -> "Node Integration Tests"
27-
* "Node (24) Integration Tests" -> "Node Integration Tests"
28-
* "Node (24) (TS 3.8) Integration Tests" -> "Node Integration Tests"
27+
* "Node (22) Integration Tests" -> "Node Integration Tests"
28+
* "Node (24) (TS 3.8) Integration Tests" -> "Node Integration Tests"
29+
* "aws-serverless-layer (Node 22) Test" -> "aws-serverless-layer Test"
30+
* "Playwright bundle_tracing_replay Tests" -> "Playwright Tests"
31+
* "Playwright esm (1/4) Tests" -> "Playwright Tests"
32+
* "E2E react-router-7-framework-node-20-18 Test" -> "E2E react-router-7-framework Test"
2933
*/
3034
function normalizeJobName(name) {
3135
return name
32-
.replace(/\(\s*(?:\d+|TS\s+[\d.]+)\s*\)/gi, ' ')
36+
.replace(/\(\s*(?:\d+|TS\s+[\d.]+|Node\s+\d+|\d+\/\d+)\s*\)/gi, ' ')
37+
.replace(/Playwright\s+(?:bundle\w*|esm|cjs)\s+Tests/gi, 'Playwright Tests')
38+
.replace(/-node-\d+(?:-\d+)*/gi, '')
3339
.replace(/\s+/g, ' ')
3440
.trim();
3541
}
3642

3743
/**
38-
* Collapse esm/cjs variants of a test name so the same test failing in both module formats dedupes
39-
* to a single issue instead of one per variant:
44+
* Collapse variants of a test name so the same test failing under different module formats, browser
45+
* projects, or (for Playwright) at a different source line dedupes to a single issue. We strip:
4046
*
41-
* "... > esm/cjs > esm > should send messages" -> "... > esm/cjs > should send messages"
42-
* "... > esm/cjs > cjs > should send messages" -> "... > esm/cjs > should send messages"
47+
* - Playwright browser prefix: "[chromium] › suites/... " -> "suites/... "
48+
* - Playwright file line/column: "test.ts:33:11 › ..." -> "test.ts › ..." (drifts on edits)
49+
* - old esm/cjs describe block: "... > esm/cjs > esm > x" -> "... > x"
50+
* - bare esm/cjs describe block: "... > esm/cjs > x" -> "... > x"
51+
* - trailing module suffix: "... should send [esm]" -> "... should send"
4352
*/
4453
function normalizeTestName(name) {
4554
return name
55+
.replace(/^\[(?:chromium|firefox|webkit)\]\s*\s*/i, '')
56+
.replace(/(\.[cm]?[jt]sx?):\d+:\d+/gi, '$1')
4657
.replace(/esm\/cjs\s*>\s*(?:esm|cjs)\b/gi, 'esm/cjs')
58+
.replace(/\besm\/cjs\s*>\s*/gi, '')
59+
.replace(/\s*\[(?:esm|cjs)\]/gi, '')
4760
.replace(/\s+/g, ' ')
4861
.trim();
4962
}

0 commit comments

Comments
 (0)