Repository navigation
fix(cli): group sampled check findings by element and rule - #5174
Conversation
somanshreddy
left a comment
There was a problem hiding this comment.
Reviewed at 809aca4b (full PR).
- Grouping.
groupSampledFindingskeys on[sourceFile, selector]pluscode, merges the sample times, and keeps the higher severity. For contrast it keeps the worst ratio (checkPipeline.ts:1343-1347).contrastFailureHeldnow keys on the element instead ofselector|text, so an element whose text changes between samples (a counter, say) still counts as held, and its real failures stay errors. Layout issues only group undercollapse-static, which is on by default. - Selectors.
uniqueSelectorFor(layout-audit.browser.js:94) falls back to annth-of-typepath when the preferred selector matches more than one element. That stops two look-alike elements from merging into one row. - Tests.
check.test.ts,layout-audit.browser.test.tsandlayoutAudit.test.tspass locally, 276/276. Mutants:- caught: grouping key ignores
code; contrast keeps the first sample instead of the worst; held key reverted toselector|text. - survived: severity never escalates within a group. For contrast that mutant is equivalent, because
heldis per element, so every sample has the same severity. For layout codes whose severity can vary by sample (panel_out_of_canvaswarning/info) it isn't pinned.
- caught: grouping key ignores
Nits, none blocking:
textOverflowIssuesandclippedTextIssuecalluniqueSelectorForbefore they know there's an issue. That runs aquerySelectorAllfor every text element on every sample, and recurses up the tree when the selector isn't unique. On a large DOM, computing it only once a finding exists would avoid that cost.- With
--no-collapse-static, caption-zone and frame findings used to be deduplicated to their first sample. They now print one row per sample. That's probably intended for the opt-out, but it's worth a line in the changelog. - An
nth-of-typeselector can change between samples if the timeline adds or removes siblings, which would split one element's rows.
CI: no required check is red. Build and the Windows render were still pending when I approved.
— Somu
somanshreddy
left a comment
There was a problem hiding this comment.
Reviewed at a9d71ccf: the fix commit over my approved 809aca4b (1 runtime line plus a test).
collapseStaticLayoutIssues no longer overwrites time with firstSeen (layoutAudit.ts:266). A collapsed issue now keeps the time of the sample its rect/bbox came from, and groupSampledFindings carries that whole finding through, so the time and position always describe the same sample. Range labels still come from firstSeen and lastSeen (layoutAudit.ts:150), so the printed span doesn't change.
The three CLI suites pass locally (277/277). Putting the time: firstSeen line back makes the new test fail, so it's pinned. My earlier nits still stand and aren't blocking. No required check was red when I approved.
— Somu
Edit accuracy: accurate 2059 (base branch 2059), smooth 1528 of thoseThe gate passes. Quarantined, measured but not gated (0) |
A held low-contrast element previously appeared once for each failed sample. Check now reports one finding per source file, element selector, and rule, with the sorted occurrence times in JSON and terminal output. Real low contrast still fails the check; its worst sample supplies the color and screenshot anchor. Layout findings retain their existing persistence grading, and the explicit no-collapse option retains individual layout observations.
Findings in separate composition files stay separate even when their selectors match. Sample-level contrast checked/passed counts remain measurements, while finding counts represent grouped problems.
Validation: four reporter regressions fail on main, and the same-class connector identity regression also fails before its fix. The check suite passes 76 tests in three consecutive serial runs; browser audit tests pass 162 tests in three runs, and layout utility tests pass 39 tests. CLI typecheck and commit checks pass. Actual source CLI browser runs on a synthetic broken fixture produce one overlap and one contrast error, with five contrast times listed. The intentional-layout-marker control still reports the same real contrast error; JSON and terminal output both show it once. Both commands retain exit 1, and the disjoint-scene control stays clean with exit 0. These controls were repeated at the final head.
Layout findings use the existing unique-selector owner so separate same-class elements remain separate. A real-browser two-connector fixture reports two distinct findings, each listing its repeated times and retaining its own geometry.
Frame and caption observations now reach the same reporter before consolidation, so stationary failures retain all sampled times and separate elements with identical text stay separate. A stationary-frame regression fails before this fix.
Grouped snapshot evidence keeps its representative time paired with its original geometry; firstSeen records the earliest occurrence separately. A sparse-before-dense regression reproduces the old mismatched crop and verifies the corrected crop request.