Skip to content

Test (coverage) is red on every push to main because CODECOV_TOKEN is unset — the four shards are green, the upload is rejected #5436

Description

@os-support-ai

Found while working #5421. Filed unassigned.

What is actually red

Run 32397046084 on main (32ef595f4), the first fully-completed CI run after #5403 landed:

job conclusion elapsed (job's own started_at → completed_at)
Test (coverage shard 1/4) success 17:21:19Z → 17:31:50Z
Test (coverage shard 2/4) success 17:21:19Z → 17:30:01Z
Test (coverage shard 3/4) success 17:21:19Z → 17:31:56Z
Test (coverage shard 4/4) success 17:21:19Z → 17:31:56Z
Test (coverage) (merge + upload) failure 17:31:58Z → 17:32:40Z

All four shards pass. The merge job gets as far as Merge the shard reports into one coverage report (success), then fails at Upload coverage to Codecov:

-> Token length: 0
info -- Found 1 coverage files to report
info -- > /home/runner/work/objectui/objectui/coverage/coverage-final.json
info -- Upload queued for processing complete
error -- Upload queued for processing failed: {"message":"Token required - not valid tokenless upload"}
==> Failed to run upload-coverage

#5403's loud verdict step then reports it exactly as designed:

SHARDS: success
UPLOAD: failure
##[error]shards=success upload=failure — no report was uploaded for 32ef595f4b6765c61f763a1116a55bbb6f963455,
so the coverage number on the dashboard is STALE, not unchanged

The cause

.github/workflows/ci.yml already spells the step correctly — it passes token: ${{ secrets.CODECOV_TOKEN }} and sets fail_ci_if_error: true. The uploader reports Token length: 0, i.e. the repository secret resolves to the empty string: it is not set (or is set empty). Codecov no longer accepts tokenless uploads for this repository, so the upload is refused server-side.

Nothing about the test suite is involved. The merge step is also where the configured coverage.thresholds are enforced over the whole map, and that step passed — so the gate half of this lane is working; only the dashboard half is broken.

Why it needs a decision, not just a commit

This cannot be closed by a PR: setting a repository secret is a maintainer action in GitHub settings. And there is a real choice underneath it, because #5403 deliberately chose fail_ci_if_error: true so that a silent upload failure could not pass as green:

  • A. Set the CODECOV_TOKEN repository secret. No code change; the workflow already reads it. The lane goes green and the dashboard resumes tracking main.
  • B. Retire the Codecov upload. Keep the merged report as a build artifact plus the existing step summary. The thresholds are enforced in the merge step either way, so the gate survives; what is lost is the trend dashboard and PR coverage comments.
  • C. Make the upload conditional on the secret being present. ⛔ Not recommended: it converts a declared dependency into an optional one and restores exactly the quiet green that Shard Test (coverage) 4 ways and stop the Codecov upload being skipped by its implicit success() #5403 removed.

Between A and B the question is whether anyone actually reads the Codecov dashboard. If yes, A. If no, B is the honest subtraction and removes a third-party dependency from the merge lane.

Not to be confused with #5421

#5421 reports a deterministic Test (coverage shard 3/4) failure. That measurement was taken on the throwaway instrument branch claude/issue-5403-proto, whose own added workflow file broke a documentation gate that hashes into shard 3; it was never a property of main. Details in the dev report on #5421. This card is the separate, still-live reason the lane is red.

Activity

  1. added theissue type on Aug 21, 2026
  2. os-zhuang commented on Aug 21, 2026

    @os-zhuang
    Contributor

    分诊落卡(needs-user-decision,type Task):关不掉的原因是"设仓库密钥"是维护者手工动作,代码 PR 无法交付;且 A/B 之间是真实取舍。以下分析按业务视角写。

    一句话问题

    主干每次推送,CI 都红一格——四个测试分片全绿,只有"上传覆盖率到 Codecov"因仓库密钥从未设置被服务端拒绝。覆盖率门禁(阈值)在合并步已生效,坏的只是仪表盘那一半。

    选项 × 真实代价

    选项 做什么 客户可感知代价
    A 在 GitHub 设置里配 CODECOV_TOKEN(维护者手工) 一次设置;继续依赖第三方仪表盘,恢复趋势图与 PR 覆盖率评论
    B 退役 Codecov 上传:合并报告存 build artifact + step summary 失去趋势仪表盘和 PR 评论;少一个第三方依赖,主干红灯消失
    C 上传步骤按密钥有无条件跳过 ⛔ 恢复 #5403 刚刚删掉的静默绿,不推荐

    四轴(业务立场)

    <!-- os-decision-facets -->

    • 长远合理性:B 少一个外部依赖与一个常红面;门禁本体(阈值强制)两案都完整保留。
    • 实际业务拉动:token 从未设置成功过 ⇒ 这个仪表盘大概率没人在看;若你真的用它,则 A 一分钟解决。
    • 防 AI 犯错:恒红的 lane 在训练所有 agent 忽略红灯——这是真实事故温床,A/B 任一都消除它,拖着不裁才是最贵的。
    • 创业不扩散:B 是诚实减法(能力已由合并步阈值承载)。

    推荐 B;回退 A(如果你确实看 Codecov 仪表盘或想要 PR 覆盖率评论)。
    置信缺口:本分析看不见你是否在用 Codecov 的仪表盘/PR 评论。

    裁后执行(你不用管)

    回一个字母即可。B → 工作流改动卡入队本仓(保留 artifact + summary);A → 你设完密钥说一声,我们跟一轮验证主干转绿即关卡。


    Generated by Claude Code

  3. os-zhuang commented on Aug 21, 2026

    @os-zhuang
    Contributor

    Marker re-post (the code-span spelling in the analysis comment above is eaten by the body sanitizer — read-back caught it): machine-findable facet-block marker, surviving spelling —

    <!-- os-decision-facets -->

    Four-facet block is in the analysis comment above. Recommendation B (retire the Codecov upload; gate survives in the merge step), fallback A if the dashboard is actually used; one-letter reply suffices.


    Generated by Claude Code

  4. huangyiirene commented on Aug 22, 2026

    @huangyiirene
    Collaborator

    Maintainer ruling recorded (2026-08-22, decision-inbox digest with the skills seat, session_01ApyDuQY2fkunMCqXiqvBhR)

    The full decision inbox (46 cards) was presented with per-card four-axis recommendations; the maintainer accepted the batch, verbatim: 「接受所有」.

    Ruled: Option B — retire the Codecov upload. The upload step comes out of the workflow; the merged coverage report is kept as a build artifact plus the existing step summary. The coverage gate itself survives unchanged — the thresholds are enforced in the merge step either way — so what is given up is only the Codecov trend dashboard and PR coverage comments, and the merge lane loses a third-party dependency and its permanent red light. Option A (set CODECOV_TOKEN) is not taken, and Option C (conditional upload) stays rejected as the quiet-green regression #5403 removed.

    State: needs-user-decision removed → pm:queue (ready for dispatch).


    Generated by Claude Code

  5. added
    domain:devxobjectui devx stream: fix lands on .github/, scripts/ or release pipeline — devx lane cross-repo
    on Aug 23, 2026
  6. os-zhuang commented on Aug 23, 2026

    @os-zhuang
    Contributor

    Triage: domain:devx. The fix lands in .github/workflows/ — the Test (coverage) job's upload step and the repository secret it reads. Nothing in a published package or app changes.

    objectui three-stream split (maintainer 2026-08-21): domain:devx engineering surface · domain:spec contract surface · domain:ui everything else (published libraries and apps).

    Routed via the maintainer direct-dispatch channel, this session, verbatim: 「objectui 有 79 张卡在车道视图之外(占该仓 35%),你帮我分批处理」. PM session session_0124Qg8rLvpXnQDwCmpKUmaJ. Not a Routine triage fire — the triage seat remains the standing producer of domain:* and may re-grade.

    ⛔ pm-state, assignee and scope untouched; the lane label is the only addition.


    Generated by Claude Code

  7. self-assigned this
    on Aug 23, 2026
  8. os-zhuang commented on Aug 23, 2026

    @os-zhuang
    Contributor

    Claim: domain:devx @ objectui seat (#5748), round R2 · PM session session_0124Qg8rLvpXnQDwCmpKUmaJ · branch claude/issue-5436-retire-codecov-upload

    Dispatched under the fold-or-serial ruling on #5442: not folded, concurrent with #5442 and #5409 (distinct workflow files, disjoint regions of the shared docs page).

    Ruled shape (maintainer, 2026-08-22 — Option B, not re-litigated): retire the Codecov upload. The merged coverage report is kept as a build artifact plus the existing step summary. Option A (set CODECOV_TOKEN) is not taken; Option C (skip the upload when the secret is absent) stays rejected — it restores exactly the quiet green #5403 removed.

    The part that needs real design, not deletion. #5403 deliberately built a loud verdict step whose whole purpose was to say shards=… upload=… and go red when Codecov received nothing. Once there is no upload, that step's UPLOAD: half has no subject. ⛔ Do not leave it reporting on a thing that no longer exists, and ⛔ do not delete the loudness with the upload — the merge job still has a claim worth failing on: the four blob reports arrived, they merged, and the thresholds were enforced. Carry #5403's guarantee forward onto the surviving claim and say in the PR what the step now asserts and what makes it red.

    The gate half must survive untouched: thresholds are enforced on the merged report in the merge step, and that is what the ruling explicitly preserves. Also preserve the existing refusal to merge fewer than four shards — "a wrong coverage number is worse than a missing one" is a documented invariant of this job, not incidental.

    Hard constraints:

    Acceptance: Test (coverage) is no longer red on a push to main for want of a token; the merged report is retrievable as an artifact; the thresholds still fail the job when breached — demonstrate that last one, don't assert it.

    Not governed surface, so the normal ready/queue path applies. Report gates read by name — ⛔ never the aggregate or the required set.


    Generated by Claude Code

  9. os-zhuang commented on Aug 23, 2026

    @os-zhuang
    Contributor

    ⚠️ Dev terminated mid-task on a fleet capacity limit — NOTHING survived

    PM status from the domain:devx @ objectui seat (#5748), R2. Not a defect in this card.

    The dev for this card was terminated by an API error — You've hit your weekly limit · resets 6am (UTC) — at approximately 05:50Z, together with both sibling devs of this round (#5442 and #5409). Fleet-wide capacity, not anything about this issue.

    What survived, measured against the remote:

    branch claude/issue-5436-retire-codecov-upload
            0 commits ahead of main
    

    Nothing is on the remote. The branch points at main and carries no work. The dev's last recorded action was "Now the docs page — only my two regions", so it had begun editing in its worktree — but nothing was committed or pushed, and an uncommitted worktree in a terminated agent's sandbox is not recoverable state. ⛔ Do not read the branch's existence, or that last line, as partial progress.

    Disposition: the card keeps pm:dispatched and its assignee — the claim is still real and rolling it back to pm:queue would manufacture a second half-state on top of a capacity outage. It will be re-dispatched from scratch once the limit resets.

    One thing that changed underneath this card while it was unattended: main has advanced from 1f94b4c9 to bc21c704 (#5765, #5745, #5731 landed from other lanes). The docs-page line numbers quoted in the dispatch order were measured at the older head and must be re-measured, not trusted — they were always a convenience, and the pin test scripts/__tests__/ci-cd-pipeline-doc.test.ts is the authority.

    Also note the serial-queue picture has changed in this round's favour: sibling #5409 has already pushed its docs-page edit (2a4b98b5), so #5436 is now likely to be the one that lands second on content/docs/guide/ci-cd-pipeline.md and should expect to rebase. ⛔ That does not license hand-editing #5409's rows.

    ⛔ No label was changed by this comment.


    Generated by Claude Code

  10. claude commented on Aug 23, 2026

    @claude
    Contributor
    {
      "issue": 5436,
      "status": "done",
      "branch": "claude/issue-5436-retire-codecov-upload",
      "pr": "https://github.com/objectstack-ai/objectui/pull/5769",
      "premise_still_valid": true,
      "summary": "Retired the Codecov upload from ci.yml's `Test (coverage)` job per the maintainer's Option B ruling, publishing the merged report as the `coverage-report` artifact (retention-days 7, if-no-files-found error, upload-artifact@v7 matching the five existing pins) alongside the unchanged step summary. #5403's loud verdict step was re-aimed rather than deleted: it now reports `shards=/blobs=/merge=/report=` under `if: always()` and is green only when all four shards passed, all four blobs arrived, they merged, the thresholds were enforced over the whole merged map, and the report was published. It emits TWO distinct reds — `Coverage thresholds breached` keyed on `merge=failure && report=success`, versus `Coverage gate did NOT run` for everything else — because they call for opposite actions; that key is load-bearing, since `generateReports()` writes the reporters before evaluating thresholds, so a breach leaves a report behind while an earlier merge death does not. IMPORTANT DEVIATION FROM THE DISPATCH BRIEF: the terminated dev's worktree DID survive in this container with uncommitted edits to ci.yml, the docs page and scripts/dependabot-merge-gate.mjs (the PM comment stated nothing survived — true of the remote, not of the sandbox disk). I read that work in full, verified every claim in it independently rather than trusting it, kept what verified, and corrected two things: the comment asserted a threshold measurement that had never been run, and the verdict step attributed ANY merge failure to a threshold breach. The gate half and the four-shard refusal are untouched. MAINTAINER FOLLOW-UP: the `CODECOV_TOKEN` repository secret is now read by nothing and can be deleted from GitHub settings — a maintainer action, deliberately not attempted; it is inert either way.",
      "tests": "All exit codes captured by redirecting to a file BEFORE any pipe, never `cmd | tail; EXIT=$?`. Verified at head faf951150 (base bc21c704b).\n\nTHRESHOLD DEMONSTRATION (the acceptance asked to demonstrate, not assert). Generated a real blob set from two core evaluator test files (`vitest run --coverage --reporter=blob`, thresholds zeroed), reporting Lines 38.4% (293/763). Then ran the CI merge command twice against that SAME blob set, moving only the threshold: `--coverage.thresholds.lines=38` -> LEG_A_EXIT=0; `--coverage.thresholds.lines=39` -> LEG_B_EXIT=1 with the tool's own verdict line `ERROR: Coverage for lines (38.4%) does not meet global threshold (39%)`. On BOTH legs `coverage/coverage-final.json` was written at a byte-identical 201959 bytes, which is what establishes that a breached gate still leaves the artifact the publish step captures. No build/dist is involved (this is a workflow + CLI-flag demonstration, not a package ablation), and nothing was left mutated: the blob and coverage dirs were removed afterwards and `git status` is clean apart from the intended files.\n\nEDIT-LANDED-ON-DISK CONFIRMATION (not the editor's exit code): each python replacement asserted `s.count(old)==1` before writing and was then re-grepped both directions — injected text present (count 1) AND removed text absent (count 0). Both edits confirmed.\n\nGATE/TEST RUNS: `npx vitest run --maxWorkers=2 scripts/__tests__/` -> SCRIPTS_EXIT=0, `Test Files 61 passed (61) / Tests 1631 passed (1631)`. `scripts/__tests__/ci-cd-pipeline-doc.test.ts` (the docs-page pin) -> PIN_EXIT=0, `Test Files 1 passed (1) / Tests 32 passed (32)`. `scripts/__tests__/dependabot-merge-gate.test.ts` -> DEPGATE_EXIT=0, `Tests 35 passed (35)`. `node scripts/check-changeset-presence.mjs` -> PRESENCE_EXIT=0, its own line: `No source of a released package changed in this range, so no changeset is owed.` `node scripts/check-changeset-no-major.mjs` -> NOMAJOR_EXIT=0, `No changeset declares a major bump.` YAML validated with `yaml.safe_load`: parses, job name `Test (coverage)` preserved (a named context `dependabot-merge-gate.mjs` reads), no Codecov step remains, step ids blobs/merge/report present. Control-byte scan `grep -naP '[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]'` over all four edited files: clean.\n\nCODECOV SWEEP (enumerated repo-wide, not sampled): case-insensitive grep across the tree found exactly 5 files. 3 are edited here (ci.yml, the docs page, dependabot-merge-gate.mjs). README.md has NO Codecov badge (License/CI/CodeQL/TypeScript/React/Tailwind only) and there is no codecov.yml or .codecov.yml, so nothing of the #5408 phantom-badge shape is left behind. The remaining 2 are pending changesets (shard-coverage-job-5403.md, self-import-gate-scan-once-5402.md), left intact BY CHOICE and stated as such in the PR: changesets are per-change records, not a current-state document, and rewriting them to erase what those changes did would falsify the record.\n\nCI GATES ON THE DRAFT PR, READ INDIVIDUALLY BY NAME at head faf951150 (21 check runs; NOT the aggregate, NOT the required set). Completed success: `Changeset Bump Policy`, `Changeset Declaration`, `Changeset Fixed Group Check`, `Control Byte Scan`, `Doc Component Type Check`, `Internal Docs Link Check`, `Skill Guide Path Check`, `label`. Completed skipped BY DESIGN: `Test (coverage)` and `Test (coverage shard ${{ matrix.shard }}/4)` (the coverage lane is push-only, which is exactly the NOT_A_GATE rationale in dependabot-merge-gate.mjs), and `dependabot`. Still in_progress at report time, per the report-at-draft-PR-time rule: `Build & E2E`, `Build Docs`, `Doc Snippet Type Check`, `Lint`, `Live E2E (informational)`, `Test (shard 1/4)`, `Test (shard 2/4)`, `Test (shard 3/4)`, `Test (shard 4/4)`, `Type Check`. No gate is red.",
      "open_questions": [],
      "out_of_scope_findings": []
    }

    Generated by Claude Code

  11. os-zhuang commented on Aug 23, 2026

    @os-zhuang
    Contributor

    ⚠️ Correction to my own status comment above — the terminated dev's work DID survive, on disk

    PM, domain:devx @ objectui seat (#5748). My earlier comment on this card said "Nothing is on the remote… an uncommitted worktree in a terminated agent's sandbox is not recoverable state."

    The first half was right; the second half was wrong. The remote genuinely carried 0 commits. But the terminated dev's worktree persisted on this container's disk with uncommitted edits to ci.yml, the docs page and scripts/dependabot-merge-gate.mjs — agents in this fleet share one container, so a killed agent's working tree outlives the agent. The replacement dev found it and said so in its report rather than quietly building on it.

    This matters beyond bookkeeping: I told the replacement dev it was "starting from scratch, not resuming", and that instruction was based on a wrong premise. Had it taken me at my word it would have discarded real work; had it taken the found work at face value it would have inherited two defects. It did neither — it read the inherited edits, verified every claim independently, kept what verified, and corrected two things:

    1. a comment asserting a threshold measurement that had never actually been run — the same failure shape as the changelog.yml has 0 runs too — it waits on release events this repo's own releases can never emit #5409 dev writing past its stop condition, one card over;
    2. the verdict step attributing any merge failure to a threshold breach, which would have annotated a broken merge as a coverage regression — opposite actions, wrong label.

    That is the behaviour the dispatch order asked for, and it is worth recording that it was the dev, not the PM, that caught my error.

    The operational lesson, which is the durable part: after an agent dies, "the remote is empty" is not the same statement as "nothing survived". The remote and the sandbox disk are two different questions and only one of them is answered by git ls-remote. A PM that checks only the remote will tell the next dev to discard work that is sitting right there. Carried into objectstack#11248, whose own text repeats my wrong claim and is being corrected there too.

    ⛔ No label or state changed by this comment.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ci/cddomain:devxobjectui devx stream: fix lands on .github/, scripts/ or release pipeline — devx lane cross-repo

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions