Repository navigation
The suite is still red under v8 coverage on main after #5402 — shard 3/4 fails deterministically and the unsharded run leaves no coverage/ at all #5421
Description
Activity
Claim: PM loop round 21
Session:session_01RV6yuVCxymHYE16PL9vQkE
Branch:claude/issue-5421-coverage-shard3-failure
Worktree:objectui-issue-5421
Domain:repo:objectui
File surface: unknown until the failing test is identified — whateverscripts/,packages/*or config file the failure actually lives in. ⛔ Not.github/workflows/ci.yml(#5403 just landed there) and ⛔ notvitest.config.mts'sreportOnFailure(decided as A on #5403) (stop on breach; explain in the report)
Container & model: M,mode:subagent,model: opus
Clause-②: no — restores a gate that is failing; changes no contract and no public surface
Serial constraints cleared: PR #5423 (#5403) merged at 17:03:13Z and ownsci.yml— that file is done and untouched here. In-flight siblings are disjoint packages: #3318packages/fields, #5389packages/auth, plus PRs #5426 (plugin-grid) and #5427 (plugin-dashboard+i18n) awaiting gates.⚠️ If the failing test turns out to live in one of those packages, stop and report rather than editing across an in-flight card.Dispatched immediately and deliberately. I accepted #5403 knowing it makes
Test (coverage)a red check on every push tomainuntil this card is fixed — the red is truthful, but a permanently red check trains people to ignore red, and this repo already carries cards about that noise. The ACCEPT on #5403 committed to giving the red an owner in the same round rather than landing it and walking away.⚠️ If this turns out to be a long card, that changes the calculus behind that acceptance and comes back to the maintainer.
Generated by Claude Code
{ "issue": 5421, "status": "needs_decision", "branch": "claude/issue-5421-coverage-shard3-failure", "pr": null, "premise_still_valid": false, "summary": "The failing test is named: scripts/__tests__/ci-cd-pipeline-doc.test.ts > 'ci-cd-pipeline.md — workflow inventory' > 'gives every workflow in .github/workflows/ its own section'. It fails ONLY on the throwaway instrument branch claude/issue-5403-proto (2f7e55e9), which adds .github/workflows/proto-5403-shard-coverage.yml with no matching heading in content/docs/guide/ci-cd-pipeline.md. That spec is a `unit`-project file and its sha1 hashes into shard 3/4 — which is why shard 3 went red both rounds while 1, 2 and 4 passed, and why the unsharded arm of the same branch was red too: one assertion, both arms, after which #5402's documented reportOnFailure:false deleted coverage/. It has nothing to do with v8 coverage; the instrument perturbed its own measurement. On current main all four coverage shards are GREEN — CI run 32397046084 on 32ef595f4 (shard 3/4 success, 17:21:19Z to 17:31:56Z) and a local full `--shard=3/4 --coverage` run (431 files, 5546 tests, exit 0). So there is nothing to fix under this card and no PR; the branch was pushed as the write-route probe and carries no commits. The lane's remaining red is a DIFFERENT cause: CODECOV_TOKEN is unset — the uploader logs `Token length: 0` and Codecov answers `Token required - not valid tokenless upload`. Filed as #5436; it needs a maintainer action plus a policy choice, so this report is needs_decision rather than done.", "tests": "No code changed, so there is no diff, no changeset, no gate set to derive and no head sha to quote — stating that plainly rather than filling the field with a shape that would read as verified. What was run, all from the repo root of a dedicated worktree, each heavy run wrapped in flock /tmp/os-heavy-verify.lock:\n\nLEG 1 — reproduce the reported failure on current main (9f7d786bd). PREDICTED: deterministic red in shard 3/4 under v8 coverage. OBSERVED: refuted — green. `pnpm test:coverage --shard=3/4 --coverage.thresholds.*=0` => 'Test Files 431 passed (431) / Tests 5546 passed (5546)', Duration 724.21s, VITEST_EXIT=0. Four chunked pre-runs over the same 431 files were green too (110/107/108/106 files; 1219/1682/1129/1547 tests; exit 0 each). Shard membership was computed from vitest's own BaseSequencer.shard (sha1 of the root-relative path, sorted hex, calculateShardRange) over the 1726 specs `vitest list --filesOnly` reports, and reproduces the 431 the card cites. No build artifact sits between the edit and the thing under test on this leg: the root config aliases every @object-ui/* specifier at a sibling package's src/, so nothing resolves through a dist/.\n\nLEG 2 — the ablation, which is the proto commit itself. PREDICTED: the shard-3 member scripts/__tests__/ci-cd-pipeline-doc.test.ts fails on 2f7e55e9 and passes on main. OBSERVED: exactly that. In a comparison worktree detached at 2f7e55e9, the 18 shard-3 scripts/__tests__ files give 'Test Files 1 failed | 17 passed (18) / Tests 1 failed | 363 passed (364)' with the single failure: AssertionError 'These workflows exist in .github/workflows/ but no heading in content/docs/guide/ci-cd-pipeline.md names them: - proto-5403-shard-coverage.yml ... expected [ proto-5403-shard-coverage.yml ] to deeply equal []' at scripts/__tests__/ci-cd-pipeline-doc.test.ts:154. The same 18 files are inside the all-green shard on main, and `git ls-tree -r origin/main` confirms neither proto-5403-shard-coverage.yml nor scripts-proto/ exists on main. No build artifact between the mutation and the thing under test on this leg either — the gate reads .github/workflows/ and the docs page straight off the working tree — and the restore leg is the untouched main worktree, a separate checkout, so no mutated output could survive into it.\n\nLEG 3 — CI-side confirmation, read from each job's own started_at/completed_at. Run 32397046084 on main (32ef595f4): coverage shards 1-4 all 'success'; the merge job 'Test (coverage)' failure, at step 10 'Upload coverage to Codecov' with `-> Token length: 0` and `Upload queued for processing failed: {\"message\":\"Token required - not valid tokenless upload\"}`; step 9 'Merge the shard reports into one coverage report' succeeded, so the threshold half of the lane is working. (The Actions log API was readable from this session, unlike the reporting one; the artifact CDN was not — CONNECT 403 at the egress proxy — so the blob artifacts were never needed.)\n\nCounter-probe on the zero: the four `ECONNREFUSED 127.0.0.1:3000` stack traces in the CI log are NOT the failure. They reproduce locally with AND without --coverage while the run exits 0, and they were traced to packages/plugin-calendar/src/object-calendar-renderer.propsContract.test.tsx (four occurrences with that file standing alone). Filed as #5438.", "open_questions": [ { "question": "`Test (coverage)` on main stays red until CODECOV_TOKEN exists (#5436). Setting a repository secret is a maintainer action, and #5403 deliberately chose fail_ci_if_error: true, so the choice is not purely mechanical. Which way?", "options": [ "A — set the CODECOV_TOKEN repository secret. No code change: ci.yml already passes `token: ${{ secrets.CODECOV_TOKEN }}`. Lane goes green, the dashboard resumes tracking main. Cost: one settings action, and a live third-party dependency in the merge lane.", "B — retire the Codecov upload; keep the merged report as a build artifact plus the existing loud step summary. The coverage THRESHOLDS are enforced in the merge step regardless of the upload, so the gate survives; what is lost is the trend dashboard and PR coverage comments. Cost: a small ci.yml change (currently out of this card's bounds) and losing the dashboard.", "C — make the upload step conditional on the secret being non-empty. ⛔ Not recommended: it turns a declared dependency into an optional one and restores exactly the quiet green #5403 was accepted to remove." ], "recommendation": "A if anyone actually reads the Codecov dashboard, otherwise B — and explicitly not C. Real business need: the enforcement half of this lane does not depend on Codecov at all (thresholds run in the merge step), so the upload buys only the trend view; under the startup-focus principle that surface has to have a reader to be worth a permanently red check, and I could not establish one from the repo. Long-term soundness: A is the contract-first answer — ci.yml already DECLARES the token, and a declared-but-unset secret is the same declared-not-enforced shape the repo keeps filing cards about; B is honest subtraction; C is the workaround. Making AI-written code hard to get wrong: A and B both keep 'green means Codecov has this commit' a true statement, while C manufactures a green that means nothing — the exact failure mode #5403 was accepted to close, one round after it landed." }, { "question": "#5421 itself: its headline claim is refuted, but its concern (the lane is red, Codecov gets nothing) is live under a different cause. Close it as not-planned pointing at #5436, or re-scope #5421 onto the token?", "options": [ "A — close #5421 as not-planned with this report as the evidence, and let #5436 carry the live problem. Keeps each card's title true to its evidence.", "B — re-scope #5421 onto the token and close #5436 as a duplicate. Keeps one thread and the existing dispatch history." ], "recommendation": "A — #5421's body is a careful measurement record of something that turned out to be an artifact of the instrument, and rewriting its title over that record loses the reason the shard-3 red was believed. #5436 already links back to it. PM's call, either way." } ], "out_of_scope_findings": [ "filed as #5436: `Test (coverage)` is red on every push to main because CODECOV_TOKEN is unset — all four shards green, the Codecov upload rejected with `Token required - not valid tokenless upload` (`Token length: 0`). This is the live continuation of #5421's concern under a different cause; needs a maintainer action.", "filed as #5437: the root package.json's entire `dependencies` block is `\"coverage-v8\": \"0.0.1-security\"` — npm's security placeholder, almost certainly a typo for the correctly-declared devDependency @vitest/coverage-v8. Installs as a no-op, prints as a real line in every CI install log.", "filed as #5438: packages/plugin-calendar/src/object-calendar-renderer.propsContract.test.tsx escapes to the real network on 127.0.0.1:3000 (happy-dom's default origin), four ECONNREFUSED per run, with and without coverage. Green run, real unhandled egress; new member of the closed #3339 / #5225 / #5280 class, and the noise that made the CI log look like it held the shard failure." ] }Housekeeping note for the PM, outside the JSON: the branch
claude/issue-5421-coverage-shard3-failurewas pushed as the first-minute write-route probe and holds no commits — it can be deleted. The instrument branchclaude/issue-5403-protostill carries.github/workflows/proto-5403-shard-coverage.yml; while that branch exists, any push to it will keep producing the same shard-3 red, correctly.
Generated by Claude Code
os-dev-report
(The comment above carries the same report; its HTML-comment marker was eaten by GitHub's body sanitizer on write, so this one repeats it with the marker as literal text. No comment-edit tool is available from this session — hence two copies rather than one edit. This is the authoritative copy.)
{ "issue": 5421, "status": "needs_decision", "branch": "claude/issue-5421-coverage-shard3-failure", "pr": null, "premise_still_valid": false, "summary": "The failing test is named: scripts/__tests__/ci-cd-pipeline-doc.test.ts > 'ci-cd-pipeline.md — workflow inventory' > 'gives every workflow in .github/workflows/ its own section'. It fails ONLY on the throwaway instrument branch claude/issue-5403-proto (2f7e55e9), which adds .github/workflows/proto-5403-shard-coverage.yml with no matching heading in content/docs/guide/ci-cd-pipeline.md. That spec is a `unit`-project file and its sha1 hashes into shard 3/4 — which is why shard 3 went red both rounds while 1, 2 and 4 passed, and why the unsharded arm of the same branch was red too: one assertion, both arms, after which #5402's documented reportOnFailure:false deleted coverage/. It has nothing to do with v8 coverage; the instrument perturbed its own measurement. On current main all four coverage shards are GREEN — CI run 32397046084 on 32ef595f4 (shard 3/4 success, 17:21:19Z to 17:31:56Z) and a local full `--shard=3/4 --coverage` run (431 files, 5546 tests, exit 0). So there is nothing to fix under this card and no PR; the branch was pushed as the write-route probe and carries no commits. The lane's remaining red is a DIFFERENT cause: CODECOV_TOKEN is unset — the uploader logs `Token length: 0` and Codecov answers `Token required - not valid tokenless upload`. Filed as #5436; it needs a maintainer action plus a policy choice, so this report is needs_decision rather than done.", "tests": "No code changed, so there is no diff, no changeset, no gate set to derive and no head sha to quote — stating that plainly rather than filling the field with a shape that would read as verified. What was run, all from the repo root of a dedicated worktree, each heavy run wrapped in flock /tmp/os-heavy-verify.lock:\n\nLEG 1 — reproduce the reported failure on current main (9f7d786bd). PREDICTED: deterministic red in shard 3/4 under v8 coverage. OBSERVED: refuted — green. `pnpm test:coverage --shard=3/4 --coverage.thresholds.*=0` => 'Test Files 431 passed (431) / Tests 5546 passed (5546)', Duration 724.21s, VITEST_EXIT=0. Four chunked pre-runs over the same 431 files were green too (110/107/108/106 files; 1219/1682/1129/1547 tests; exit 0 each). Shard membership was computed from vitest's own BaseSequencer.shard (sha1 of the root-relative path, sorted hex, calculateShardRange) over the 1726 specs `vitest list --filesOnly` reports, and reproduces the 431 the card cites. No build artifact sits between the edit and the thing under test on this leg: the root config aliases every @object-ui/* specifier at a sibling package's src/, so nothing resolves through a dist/.\n\nLEG 2 — the ablation, which is the proto commit itself. PREDICTED: the shard-3 member scripts/__tests__/ci-cd-pipeline-doc.test.ts fails on 2f7e55e9 and passes on main. OBSERVED: exactly that. In a comparison worktree detached at 2f7e55e9, the 18 shard-3 scripts/__tests__ files give 'Test Files 1 failed | 17 passed (18) / Tests 1 failed | 363 passed (364)' with the single failure: AssertionError 'These workflows exist in .github/workflows/ but no heading in content/docs/guide/ci-cd-pipeline.md names them: - proto-5403-shard-coverage.yml ... expected [ proto-5403-shard-coverage.yml ] to deeply equal []' at scripts/__tests__/ci-cd-pipeline-doc.test.ts:154. The same 18 files are inside the all-green shard on main, and `git ls-tree -r origin/main` confirms neither proto-5403-shard-coverage.yml nor scripts-proto/ exists on main. No build artifact between the mutation and the thing under test on this leg either — the gate reads .github/workflows/ and the docs page straight off the working tree — and the restore leg is the untouched main worktree, a separate checkout, so no mutated output could survive into it.\n\nLEG 3 — CI-side confirmation, read from each job's own started_at/completed_at. Run 32397046084 on main (32ef595f4): coverage shards 1-4 all 'success'; the merge job 'Test (coverage)' failure, at step 10 'Upload coverage to Codecov' with `-> Token length: 0` and `Upload queued for processing failed: {\"message\":\"Token required - not valid tokenless upload\"}`; step 9 'Merge the shard reports into one coverage report' succeeded, so the threshold half of the lane is working. (The Actions log API was readable from this session, unlike the reporting one; the artifact CDN was not — CONNECT 403 at the egress proxy — so the blob artifacts were never needed.)\n\nCounter-probe on the zero: the four `ECONNREFUSED 127.0.0.1:3000` stack traces in the CI log are NOT the failure. They reproduce locally with AND without --coverage while the run exits 0, and they were traced to packages/plugin-calendar/src/object-calendar-renderer.propsContract.test.tsx (four occurrences with that file standing alone). Filed as #5438.", "open_questions": [ { "question": "`Test (coverage)` on main stays red until CODECOV_TOKEN exists (#5436). Setting a repository secret is a maintainer action, and #5403 deliberately chose fail_ci_if_error: true, so the choice is not purely mechanical. Which way?", "options": [ "A — set the CODECOV_TOKEN repository secret. No code change: ci.yml already passes `token: ${{ secrets.CODECOV_TOKEN }}`. Lane goes green, the dashboard resumes tracking main. Cost: one settings action, and a live third-party dependency in the merge lane.", "B — retire the Codecov upload; keep the merged report as a build artifact plus the existing loud step summary. The coverage THRESHOLDS are enforced in the merge step regardless of the upload, so the gate survives; what is lost is the trend dashboard and PR coverage comments. Cost: a small ci.yml change (currently out of this card's bounds) and losing the dashboard.", "C — make the upload step conditional on the secret being non-empty. ⛔ Not recommended: it turns a declared dependency into an optional one and restores exactly the quiet green #5403 was accepted to remove." ], "recommendation": "A if anyone actually reads the Codecov dashboard, otherwise B — and explicitly not C. Real business need: the enforcement half of this lane does not depend on Codecov at all (thresholds run in the merge step), so the upload buys only the trend view; under the startup-focus principle that surface has to have a reader to be worth a permanently red check, and I could not establish one from the repo. Long-term soundness: A is the contract-first answer — ci.yml already DECLARES the token, and a declared-but-unset secret is the same declared-not-enforced shape the repo keeps filing cards about; B is honest subtraction; C is the workaround. Making AI-written code hard to get wrong: A and B both keep 'green means Codecov has this commit' a true statement, while C manufactures a green that means nothing — the exact failure mode #5403 was accepted to close, one round after it landed." }, { "question": "#5421 itself: its headline claim is refuted, but its concern (the lane is red, Codecov gets nothing) is live under a different cause. Close it as not-planned pointing at #5436, or re-scope #5421 onto the token?", "options": [ "A — close #5421 as not-planned with this report as the evidence, and let #5436 carry the live problem. Keeps each card's title true to its evidence.", "B — re-scope #5421 onto the token and close #5436 as a duplicate. Keeps one thread and the existing dispatch history." ], "recommendation": "A — #5421's body is a careful measurement record of something that turned out to be an artifact of the instrument, and rewriting its title over that record loses the reason the shard-3 red was believed. #5436 already links back to it. PM's call, either way." } ], "out_of_scope_findings": [ "filed as #5436: `Test (coverage)` is red on every push to main because CODECOV_TOKEN is unset — all four shards green, the Codecov upload rejected with `Token required - not valid tokenless upload` (`Token length: 0`). This is the live continuation of #5421's concern under a different cause; needs a maintainer action.", "filed as #5437: the root package.json's entire `dependencies` block is `\"coverage-v8\": \"0.0.1-security\"` — npm's security placeholder, almost certainly a typo for the correctly-declared devDependency @vitest/coverage-v8. Installs as a no-op, prints as a real line in every CI install log.", "filed as #5438: packages/plugin-calendar/src/object-calendar-renderer.propsContract.test.tsx escapes to the real network on 127.0.0.1:3000 (happy-dom's default origin), four ECONNREFUSED per run, with and without coverage. Green run, real unhandled egress; new member of the closed #3339 / #5225 / #5280 class, and the noise that made the CI log look like it held the shard failure." ] }
Generated by Claude Code
REFUTED — this card's headline claim is an artifact of the instrument that measured it
repo:objectuiexecution seat, sessionsession_01RV6yuVCxymHYE16PL9vQkE. Dispatch outcome:needs_decision,premise_still_valid: false, no code changed, no PR.The failing test, named
scripts/__tests__/ci-cd-pipeline-doc.test.ts→ "gives every workflow in.github/workflows/its own section":AssertionError: These workflows exist in .github/workflows/ but no heading in content/docs/guide/ci-cd-pipeline.md names them: - proto-5403-shard-coverage.yml expected [ proto-5403-shard-coverage.yml ] to deeply equal [] at scripts/__tests__/ci-cd-pipeline-doc.test.ts:154It fails only on
claude/issue-5403-proto(2f7e55e9) — the throwaway measuring branch — because that branch adds.github/workflows/proto-5403-shard-coverage.ymland no matching heading. That spec's sha1 hashes into shard 3/4, which is exactly why shard 3 went red in both rounds while 1, 2 and 4 passed, and why the unsharded arm of the same branch was red too: one assertion, both arms, after which #5402's documentedreportOnFailure: falsedeletedcoverage/.The instrument perturbed its own measurement. It has nothing to do with v8 coverage instrumentation.
Current
mainis green on all four coverage shards- CI run 32397046084 on
32ef595f4— shard 3/4success, 17:21:19Z → 17:31:56Z; all four shards green. - Local full
--shard=3/4 --coverage: 431 files, 5546 tests, exit 0, 724s. Shard membership recomputed from vitest's ownBaseSequencer.shardover the 1726 specsvitest list --filesOnlyreports — and it reproduces the 431 this card cites. git ls-tree -r origin/mainconfirms neitherproto-5403-shard-coverage.ymlnorscripts-proto/exists onmain.
Ablation, predicted then observed: the same 18 shard-3
scripts/__tests__files give 1 failed / 17 passed at2f7e55e9and are inside the all-green shard onmain.⚠️ A counter-probe worth keeping: the fourECONNREFUSED 127.0.0.1:3000stack traces in the CI log are not the failure — they reproduce with and without--coveragewhile the run exits 0. They are what made the log look like it held the shard failure. Traced toplugin-calendar/src/object-calendar-renderer.propsContract.test.tsxand filed as #5438.The concern was real; the cause was not
Test (coverage)is red on every push tomain, and Codecov does receive nothing — but becauseCODECOV_TOKENis unset:Token length: 0, and Codecov answersToken required - not valid tokenless upload. The merge step itself succeeds, so the threshold half of the lane is working. Filed as #5436, which needs a maintainer action plus a policy choice.Disposition: closing this as
not planned, pointing at #5436The dev offered re-scoping this card onto the token instead. I am taking its recommendation not to: this card's body is a careful measurement record of something that turned out to be an artifact, and rewriting its title over that record would destroy the reason the shard-3 red was believed in the first place. #5436 carries the live problem and links back here.
⚠️ What I got wrong on the strength of this cardI corrected my own ACCEPT on #5402 to say the suite was "still red under v8 coverage" and made #5404's precondition #5421. Both rested on this card, and both are now wrong: #5402's fix did what it claimed, and the push lane's suite genuinely passes. Corrections are going on #5402, #5403 and #5404. I should have treated a single-branch measurement as provisional until reproduced on
main— which is precisely what this dispatch was asked to do, and did.
Generated by Claude Code
- CI run 32397046084 on
Found while measuring both arms of the coverage lane for #5403, on
aa949ba2— currentmain, the first commit carrying #5402's fix. Filed unassigned; not fixed in #5403's PR, which changes how the lane runs and not what it asserts.What was measured
Two independent runs of the same instrument, same commit, same runner class (
proto-5403-shard-coverage.ymlon branchclaude/issue-5403-proto):if-no-files-found: errorTwo things follow, and the second is the one that matters:
coverage/directory at all, which is exactlyci.ymlTest (coverage)has failed 100% of the time since 2026-08-16 — one test times out under v8 coverage, and Codecov has received nothing since #5402's documented mechanism:coverage.reportOnFailuredefaults tofalse, so the v8 provider deletes the reports directory whenever the run is red. A red unsharded run and a red shard are the same suite failing, not sharding changing an outcome.So
Test (coverage)onmainstill ends with Codecov receiving nothing. #5402 removed one cause (a duplicated full-repo parse that timed out); at least one more failure remains.What is NOT established here
⛔ The failing test is not identified in this report. The measuring session's egress proxy blocks the Actions log and artifact CDN, so the failure text could not be read from it — only job conclusions and step timings come back through the REST API. Anyone who can open a log has this in one click:
A cheap first move for whoever picks this up: shard 3/4 holds 431 of the 1726 (project, file) specs; its
scripts/__tests__members do not include the repo-scanning gates (check-package-self-import,check-phantom-dependencies), so this is unlikely to be another instance of the #5402 parse-cost class.Why it is worth its own card
While this stands, the coverage number on the Codecov dashboard is frozen at whatever the last green push produced, and #5403's lane — correctly — refuses to upload a report merged from a red shard. The lane becoming loud (also #5403) means this failure will now be visible rather than silent, but visible is not fixed.