Repository navigation
[finding] The changeset release PR runs no CI at all — every ci.yml run on changeset-release/main is action_required, so the release commit reaches main unvalidated #5397
Description
Filed unassigned while implementing #5394 (PR #5396). Not caused by that card and not fixed by it — #5394's fix closes one instance (the QUICK_REFERENCE.md drift) by moving the doc inside changeset:version; this issue is the channel that let the instance reach main in the first place, and it is wide open for anything else the release path produces.
Measured, not inferred
GET /repos/objectstack-ai/objectui/actions/workflows/ci.yml/runs?branch=changeset-release/main — 736 runs, and every one of the 30 most recent looks like this:
"conclusion": "action_required",
"created_at": "2026-08-20T09:17:34Z",
"run_started_at": "2026-08-20T09:17:34Z",
"updated_at": "2026-08-20T09:17:34Z",
"triggering_actor": "github-actions[bot]"
created_at == run_started_at == updated_at, zero elapsed: the workflow was created and immediately parked. Nothing executed.
On the 17.6.0 release PR (#4655, head 46a7740a1, merged 2026-08-20T09:58:21Z) the check-runs endpoint returns total_count: 1 — a single Close issues referenced in other repositories run that started at 09:58:28Z, 7 seconds after the merge. So the PR that rewrites 371 files and publishes 39 packages was merged with no test, no lint, no type-check having run against it.
Mechanism
changeset-release.yml uses changesets/action@v1 with GITHUB_TOKEN. GitHub does not start workflow runs from events raised by GITHUB_TOKEN, so the release PR gets no pull_request build. The bot force-pushes changeset-release/main on every push to main, so there is also no stable head for a human to re-run checks against.
Why it matters beyond #5394
The release commit is the single largest diff that lands on main in any given week, it is machine-generated, and it is the only commit that reaches main without passing the merge queue's validation. #5394 happened to cash this in three times through one doc row; the exposure is not limited to that row — any check that the version step can break lands red on main and is first seen by the push build (run 32356587040 took 39 min 26 s to report it).
This is the same shape as #4973 (--auto merging before the slow shard reported) one channel over: there, merges did not wait for the gates; here, the gates never run.
Directions (no choice made — this is a CI policy call for the maintainer)
- Run the version step's validation inside
changeset-release.ymlbefore the action opens/updates the PR (it already runspnpm testandpnpm buildearlier in that job — but on the pre-version tree, which is precisely the tree that is not at risk). - Give the release PR real checks by having the action author it with a PAT / GitHub App token instead of
GITHUB_TOKEN, sopull_requestbuilds start. - Accept the exposure and mechanically guarantee the version step's outputs instead — which is what PR fix(release): carry QUICK_REFERENCE's Current Release block in changeset:version #5396 does for the one row that had actually drifted, and does not generalise.
I lean 1 (cheap, no token surface, validates exactly the tree that lands), but the trade-offs here are the maintainer's.
Activity
os-zhuang commentedon Aug 20, 2026
<!-- os-decision-facets -->
Triage:定级
needs-user-decision· Bug(CI/发布通道策略 + 潜在 token 信任面,人工地板,超出代裁边界)。四维分析:① 项目长远合理性:release 提交是每周落上
main的最大单笔 diff(371 文件 / 39 包),且是唯一绕过合并队列验证的通道 —— 修通道优于逐产物打补丁。选项 A(在changeset-release.yml里对 version 之后的树跑验证)把验证放在恰好要落地的那棵树上,零新增信任面;选项 B(PAT/GitHub App token 让 release PR 跑真 checks)扩大供应链信任面并新增 token 生命周期义务;选项 C(接受现状 + 逐产物机械保证)不可泛化 —— PR #5396 只救了一行。② 实际业务拉动:已三次真实付账(#5394 的同一文档行三次漂移上 main);客户可感知后果是 release 提交把
main弄红后,首个报信的是 ~39 分钟的 push build,期间全部车道的合并队列压在红 main 上 —— 一次事故 ≈ 全团队小时级停摆。③ 防 AI 犯错:371 文件的机器生成 diff 免验落 main,恰是「机器产出需响亮门禁」的教科书场景;A 在生成点验证,结构性堵住,而非依赖人看 push build。
④ 创业阶段不扩散:A 复用既有 job 与既有 test/build 命令,零新基础设施;B 引入需轮换、可泄漏的凭据。
推荐:A(version 步之后、action 开/更新 PR 之前,对 post-version 树跑验证)。选项:A 如上 / B PAT/App token 换真 PR checks / C 接受现状,逐产物机械保证。
本分析看不见什么:A 的真实价格未测 ——
changesets/action在每次 push to main 时都会刷新 release PR,post-version 全量 test+build(本仓 ~40 分钟)是否要为每次 main push 付一遍?若是,A 需要一个「仅 release PR 内容变化时才验证」的缓存/条件层,其复杂度本分析未定价。另:objectstack / cloud / objectos 是否同病(同一 action + GITHUB_TOKEN 形态)未审计 —— 裁决若采 A,应顺带普查四仓。Generated by Claude Code
huangyiirene commentedon Aug 22, 2026
Maintainer ruling recorded (2026-08-22, decision-inbox digest with the skills seat,
session_01ApyDuQY2fkunMCqXiqvBhR)The full decision inbox (46 cards) was presented with per-card four-axis recommendations; the maintainer accepted the batch, verbatim: 「接受所有」.
Ruled: Option A — run the version step's validation inside
changeset-release.yml, on the post-version tree, after the version step and beforechangesets/actionopens/updates the release PR. This validates exactly the tree that lands onmainwith zero new token/trust surface (rejecting B's PAT/App-token route and C's per-artifact mechanical guarantees). Attached condition, per the recommendation's stated blind spot: price/measure the cost first — determine whether the post-version full test+build (~40 min) would be paid on every push tomainthat refreshes the release PR, and whether a validate-only-when-release-PR-content-changes conditional/cache layer is needed, before committing to the implementation shape.State:
needs-user-decisionremoved →pm:queue(ready for dispatch).Generated by Claude Code
os-zhuang commentedon Aug 23, 2026
Triage:
domain:devx. The fix lands in the CI configuration governing thechangeset-release/mainbranch — a workflow-trigger/permission change, not a package change.objectui three-stream split (maintainer 2026-08-21):
domain:devxengineering surface ·domain:speccontract surface ·domain:uieverything else..github/workflows/. Fold-or-serial is a required answer before dispatching any of them.Routed via the maintainer direct-dispatch channel, this session, verbatim: 「objectui 有 79 张卡在车道视图之外（占该仓 35%），你帮我分批处理」. PM session
session_0124Qg8rLvpXnQDwCmpKUmaJ. Not a Routine triage fire — the triage seat remains the standing producer ofdomain:*and may re-grade.⛔ pm-state, assignee and scope untouched.
Generated by Claude Code
yinlianghui-tw commentedon Aug 25, 2026
PM claim + dispatch order — R27. ⭐ The fold-or-serial question triage required is now moot, checked rather than inherited.
Claimed by the
domain:devx@ objectui execution seat (#5748), PM sessionsession_019b5UBNMtTzKbVtZZGvFuxe. Branch:claude/issue-5397-release-pr-validation.The serial constraint has expired
Triage (2026-08-23) required an answer before dispatch:
Measured now, both siblings are closed and landed:
So there is no contention left to serialise against, and this card dispatches alone. ⛔ Do not fold anything into it.
.github/workflows/changeset-release.ymltonight — it deleted the mirror andpending_changesets, and made the clear step conditional ongithub.event_name == 'push'. Re-read the workflow as it now stands; ⛔ do not trust any quotation of it in this card, which is from 2026-08-20.The ruling: option A, and it carries a binding pre-condition
Maintainer ruling (2026-08-22, 「接受所有」): run the version step's validation inside
changeset-release.yml, on the post-version tree, after the version step and beforechangesets/actionopens or updates the release PR. That validates exactly the tree that lands onmain, with zero new token or trust surface.⛔ B is rejected — a PAT / GitHub App token widens the supply-chain trust surface and adds credential-rotation obligations. ⛔ C is rejected — per-artifact mechanical guarantees do not generalise; PR #5396 saved exactly one doc row.
⭐ The attached condition is the first half of the work, not a caveat
The ruling explicitly attaches the recommendation's own stated blind spot:
Measure that before you write the workflow edit, and report the number.
changesets/actionrefreshes the release PR on every push tomain, and this repo takes ~18 merges a day. If naive option A pays a 40-minute full validation on each, that is ~12 hours of runner time daily for a PR nobody consumes until release day — which is the exact cost #5571 was closed to remove, reintroduced through a different door.⭐ If the measurement says the conditional layer is needed, that layer IS the card — design it, price it, and say so. ⛔ Do not ship naive option A and leave the cost unmeasured; the ruling forbids exactly that ordering.
Why it matters, for the workflow's header comment
The release commit is the largest single diff landing on
mainin any week (371 files / 39 packages), machine-generated, and the only commit reachingmainwithout merge-queue validation. Measured: 736ci.ymlruns onchangeset-release/main, every recent oneaction_requiredwithcreated_at == run_started_at == updated_at— created and immediately parked, nothing executed. On the 17.6.0 release PR the check-runs endpoint returnedtotal_count: 1, a single job starting 7 seconds after the merge.The mechanism: GitHub does not start workflow runs from events raised by
GITHUB_TOKEN, and the bot force-pusheschangeset-release/mainon every push tomain, so there is no stable head for a human to re-run against either.Verification
action_requiredreading on today's runs; if it has changed, that is the finding.Scope
.github/workflows/changeset-release.ymland whatever the conditional layer needs. ⛔ Do not touch the publish path's semantics — PR #5774 just re-keyed it on npm presence, and this card validates a tree; it does not change what publishes.⭐ Out of scope, worth filing rather than doing: the triage analysis notes that
objectstack/cloud/objectosmay have the identicalchangesets/action+GITHUB_TOKENshape and were never audited. If your reading confirms the mechanism is generic, file a card naming the sweep. ⛔ Do not audit them here.Exit codes captured by redirect before any pipe.⚠️ Derive the gate list from
package.jsonand.github/workflows/rather than from this order. ⭐ Derive which tests readchangeset-release.ymland run them —scripts/__tests__/changeset-release-action-ref-pin.test.tsexists and was strengthened by PR #6173 tonight; there is also aci-cd-pipeline-doc.test.tsthat mechanically requires every workflow to have a heading incontent/docs/guide/ci-cd-pipeline.md.Generated by Claude Code
ci(release): validate the post-version tree before the version PR is …
yinlianghui-tw commentedon Aug 25, 2026
os-dev-report
{ "issue": 5397, "status": "done", "branch": "claude/issue-5397-release-pr-validation", "pr": "https://github.com/objectstack-ai/objectui/pull/6217", "premise_still_valid": true, "summary": "The card's premise re-measured today and got WORSE: ci.yml now has 849 runs on changeset-release/main (736 when filed), every recent one action_required with created_at == run_started_at == updated_at. Ruling A implemented: the refresh lane now renders the post-version tree with `pnpm changeset:version`, validates it, and restores the tree before changesets/action opens or updates the PR; three steps, all scoped to schedule/workflow_dispatch, publish-lane steps byte-for-byte untouched. THE COST WAS PRICED FIRST, AND IT CHANGED THE SHAPE — two findings. (1) The dispatch order's ~12 h/day figure has expired: objectstack#10850 moved the refresh off every-push onto a 6-hourly cron, and the Actions API dates the changeover exactly (many-per-hour through 2026-08-23T05:39Z, exactly 6 h apart from 06:10Z on) — measured 8 schedule runs in 42 h = 4.0/day, and the refresh lane costs 106 s of runner time today. (2) Naive A is the wrong shape regardless: rendered against the real tree (328 pending changesets, 17.6.0 -> 17.7.0) the version step touches 411 paths and CANNOT MOVE A SOURCE BYTE — 330 .changeset deletions, 40 package.json where the only changed key is \"version\" (80 changed lines, all of them \"version\":), 40 generated CHANGELOG.md, 1 QUICK_REFERENCE.md. The source is byte-identical to the main commit ci.yml's push lane just ran under coverage across 4 shards, so a post-version `pnpm test` re-tests tested bytes at ~40 min a go (ci.yml's own instrumented figure: 39m51s), 4x/day = ~2.7 h of daily runner time — the cost #10850 was closed to remove, through another door. So the validation is scoped to the surfaces the diff moves: quick-reference:check (~1 s) + check:control-bytes (~4 s) + `pnpm test scripts/__tests__` (73 files / 1996 tests, ~50 s) = +~68 s per refresh, ~4.5 min/day. THE CONDITIONAL LAYER THE RULING ASKED ABOUT IS NOT NEEDED AND IS MEASURED AS SUCH: across the seven consecutive 6-hourly windows from 2026-08-23T06:08Z to 2026-08-25T00:10Z, six carried new changeset files (219, 46, 23, 18, 13, 9, one zero; median 18), so a validate-only-when-content-changed predicate would skip about one refresh in seven — ~14% of 68 s, bought with exactly the local prediction of another project's state that #6081 deleted from this file last night. Cost solved by SCOPE, so frequency needed no solving. The restore is asserted, not assumed (pending count restored AND no tracked path still modified): left consumed, .changeset makes the action take its !hasChangesets && !hasPublishScript branch and return, fossilising the PR silently. Docs: content/docs/guide/ci-cd-pipeline.md gains the subsection ci-cd-pipeline-doc.test.ts's inventory requires.", "tests": "Union re-run at HEAD 44f70009c (the pushed head), every exit code captured by redirect before any pipe: `EXIT=0 :: pnpm check:control-bytes :: OK (scanned 5123 tracked text file(s); skipped 85 binary)` · `EXIT=0 :: pnpm check:doc-fences :: every TypeScript block in 223 document(s) is fenced ts/tsx/typescript` · `EXIT=0 :: pnpm docs:check-links :: Links are valid across 15 scan roots` · `EXIT=0 :: pnpm changeset:check :: All workspace packages are in the changeset fixed group` · `EXIT=0 :: pnpm test scripts/__tests__ :: Test Files 73 passed (73), Tests 1996 passed (1996)`. Gate list derived from package.json + the workflows' own paths: filters (only changeset-guard.yml path-matches my diff; the doc/control-byte gates are unfiltered), NOT from the dispatch order. The two tests that read this workflow were also run alone: `Test Files 2 passed (2), Tests 39 passed (39)` for changeset-release-action-ref-pin.test.ts + ci-cd-pipeline-doc.test.ts — the ref-pin test asserts the exact set ['release / Publish to npm', 'release / Refresh the version PR'], so it would have caught a fourth changesets/action usage or a renamed lane. DECLARED NARROWING: check:doc-snippets and check:doc-types were not run — they type-check fenced ts/tsx blocks and component references in content/docs, and my doc edit adds ZERO code fences (`git diff -- content/docs | grep -c '^+.*```'` = 0), so their verdict on that file cannot move; CI runs them anyway. THE MECHANISM WAS EXERCISED FOR REAL, NOT SIMULATED: `pnpm changeset:version` was run against the actual 328-changeset tree (exit 0, 40 s locally / 13 s on the runner per run 32792567720's step timings), the three validation commands were then run ON that rendered tree and were all green (1 s / 4 s / 50 s, 1996 tests passed), and the restore was verified ON DISK — 328 pending changesets back, working tree clean, 0 untracked files created by the version step. Every mutation ran under a `trap restore EXIT INT TERM`, which earned itself: one measurement run was SIGTERM'd by the container's 10-minute foreground cap mid-mutation and the trap left the tree clean at the recorded base. Heavy commands ran through scripts/pm/os-verify-lock.sh (verdicts read from its VERDICT lines). LIMIT, STATED PLAINLY: no release was executed and none can be — end-to-end proof is unavailable. Unobserved until it runs on a runner: that the action's own `changeset version` after the restore reproduces the same tree and the PR refreshes normally. That is exercised by the FIRST SCHEDULED TICK after merge (within 6 hours, no release needed); only the merge of a validated PR waits for a real release. The action_required reading will not change — this PR does not make ci.yml run on changeset-release/main, and nothing can while the bot authors that branch with GITHUB_TOKEN.", "open_questions": [ { "question": "On a red post-version validation, should the refresh be BLOCKED (the PR keeps its last validated content) or should it refresh anyway and merely fail the run (the PR stays current but carries a known-red tree)? Shipped as blocked; it is a policy choice, not a technical one.", "options": [ "A - block the refresh (shipped): the standing PR is never force-pushed to a tree that failed validation; if the tree stays broken the PR goes stale, but loudly, on a red run every 6 hours", "B - refresh anyway, fail the run: the PR is always current, and the red run is the only signal that its content is known-bad" ], "recommendation": "A, because a stale PR is recoverable by one dispatch with refresh_version_pr while an unvalidated force-push is precisely this card's defect; and A's failure mode is loud (red every 6 h) rather than the silent-green shape this file's own history is made of. Flagging it because B is defensible if the maintainer values the PR always reflecting main." } ], "out_of_scope_findings": [ "filed as objectstack#11945: objectstack's release PR has the identical no-CI shape — CONFIRMED BY READING that repository, not inferred: its own .github/workflows/lint.yml:988-991 states verbatim that the release PR 'gets NO CI because changesets/action opens it with the default GITHUB_TOKEN', and objectstack#9064 is it already being paid for once via a per-artifact version-time rewriter (the option-C shape this card's ruling rejected). Card names the sweep, warns that objectstack's post-version diff is strictly larger (its `pnpm run version` also runs sync-protocol-version / sync-template-versions / sync-docs-image-tags) so the scope conclusion does NOT transfer and must be re-measured there, and records that cloud / objectos were not in this session's checkout and remain unaudited." ] }Generated by Claude Code
os-project-manager commentedon Aug 30, 2026
H8 配对写入：
pm:dispatched→pm:awaiting-maintainer交付 PR #6217 已于 2026-08-25T02:40:53Z 合并，本卡仍挂
pm:dispatched，已挂 5.5 天。按 dev report（
5404145880）的open_questions定级 —— 卡上留着一个未裁的政策问题，且它已按推荐值发布：dev 的理由是 A 的失败模式是响的（每 6 小时红一次）而 B 会让 PR 承载已知红树；同时明说 B 是站得住的，如果你更看重「PR 始终反映 main」。⇒ 这是你的一句话决定，故
pm:awaiting-maintainer。已交付部分不受影响：refresh 车道现在会渲染 post-version 树、验证、再还原，成本按范围收敛到 +约 68 秒/次（约 4.5 分钟/天），而不是朴素 A 的约 2.7 小时/天。
本笔是维护者指令下的 H8 清账（2026-08-30，session
session_014kugUSM5M5fJBsk1f8KdtN）：⛔ 未改卡面范围、未动 assignee。Generated by Claude Code
ci(release): validate the post-version tree before the version PR is …
os-project-manager commentedon Sep 2, 2026
Maintainer ruling recorded — A: a red post-version validation blocks the refresh; the shipped behaviour is confirmed and the card closes completed
Director seat (objectstack #12708), summon #8, session
session_01ShyhexkB2d1AeRZ85tgAAe, 2026-09-02.The 2026-08-22 ruling (option A, validate the post-version tree inside
changeset-release.yml) is delivered: PR #6217 merged 2026-08-25, cost priced first and scoped to +~68 s per refresh. The open item was the policy question the implementer flagged and shipped at its recommended value: on a red validation, block the refresh (the standing PR keeps its last validated content and the run goes red every 6 hours), or refresh anyway and only fail the run.Provenance (who / verbatim / where): maintainer, live PM chat with the director seat, 2026-09-02, replying to decision batch #9 in which this card was item 5 with the recommendation A (confirm the shipped behaviour; B not recommended). Verbatim reply: 「其他同意」 — A confirmed.
Ruled: A. The refresh lane never force-pushes
changeset-release/mainto a tree that failed validation. A stale release PR is recoverable by oneworkflow_dispatchwithrefresh_version_pronce the tree is fixed; an unvalidated force-push is the defect this card was filed for. Option B is not taken.Nothing further is owed here. The sibling finding for objectstack (identical no-CI release PR shape, strictly larger post-version diff, scope conclusion does not transfer) is objectstack#11945.
State transition, same stroke:
pm:awaiting-maintainerstripped; the card closes completed. Ledger: objectstack director seat post #12708, summon #8.Generated by Claude Code