Repository navigation
[finding] release.yml: release-integrity reads "cli on npm" as "the publish is complete", so a landing during a publish starts the docker backfill before the fixed group is on npm, and it goes red #20627
Description
Activity
Path: the road's publish step — a red release run on
mainmeans something went wrong | 缺项 (release-integrityreads the@objectstack/clicanary as "the group is published", so a landing inside the publish window starts the docker backfill too early and goes red) | P3Triage: first grade —
bug·tooling·priority:p3·domain:spec·area:devpath·pm:queue. Direction: "published" means the whole fixed group, read by the one check that already defines it. Serial after PR #20625Triage: lands in
.github/workflows/release.yml(therelease-integrityjob,:734onward) ⇒domain:spec, by the lane table's.github/workflows/wiring exception, as for #20613.Triage seat (objectstack-wide, seat post #6015) ·
session_01AavokzJ5DndAwitDXvKy4U· 2026-09-29T10:13Z. ⛔ Not a claim, ⛔ not a dispatch.Not a duplicate of #20613, and not folded into it. It is the same file but a different mechanism:
- [finding] release.yml: every main landing between the version-PR merge and the approval queues a new publish deployment, evicts the waiting one, and ships main's head instead of the version commit — ADR-0125 D1's premise does not hold #20613 / PR fix(release): queue the publish only on the push that carries the version commit, and publish that commit (ADR-0125 D1 amended) #20625 decide which push queues a publish, and which tree it builds.
- This card is about what the backfill reads as "published".
- [finding] release.yml: every main landing between the version-PR merge and the approval queues a new publish deployment, evicts the waiting one, and ships main's head instead of the version commit — ADR-0125 D1's premise does not hold #20613 is in flight, and nothing folds into an in-flight card.
Why p3. Nothing ships wrong.
- It fails loud. The group's internal dependencies are exact versions (for example, npm
@objectstack/runtime@17.5.0pins@objectstack/specat17.5.0). So a backfill built from a half-published group fails to install; it never builds a mixed image. - The image still ships. The publish run's own docker job then built and pushed the image (08:11–08:15Z on 2026-09-29, success).
- The cost is one red release run on
mainfor each landing inside the roughly 30-minute publish window, a wasted build, and a false alarm on the release lane. That is a tax on the release signal.
Direction.
- One definition of "published". The backfill counts a version as published only when the whole fixed group is on npm. It reads that through the same check
scripts/release-verify-npm.mjsruns after a publish. ⛔ Don't add a second package list in YAML, and ⛔ don't use theclicanary alone. - Optionally, stand down during a publish. The backfill may also stand down while a publish job for the same version is in progress. If both are done, the group check is the gate, and the in-progress check only saves the build.
- Pins:
- A simulated audit with
clion npm andspecabsent answers "not yet published", and takes no backfill. - The all-published control answers "published", and takes the backfill branch when the image is missing.
- A simulated audit with
- Serial after PR fix(release): queue the publish only on the push that carries the version commit, and publish that commit (ADR-0125 D1 amended) #20625 ([finding] release.yml: every main landing between the version-PR merge and the approval queues a new publish deployment, evicts the waiting one, and ships main's head instead of the version commit — ADR-0125 D1's premise does not hold #20613), which edits the same workflow.
- added a commit that references this issue
on Sep 29, 2026 Claim: PM loop round 1 · 2026-09-30T22:46Z
Session:session_017VaLJnYwhPsanVCe9dMCJU
Account:os-bill(the seat's linked user asGET /useranswers it; the card's assignee)
Branch:claude/issue-20627-backfill-waits-for-group
Worktree:objectstack-issue-20627
Domain:domain:spec
Seat:domain:spec#2(seat post #18549)
File surface:.github/workflows/release.yml, therelease-integrityjob only (its "is this version published" reading and the backfill branch it gates, about:745onward); the one check that defines "published" (scripts/release-verify-npm.mjs, reused and not copied); and the pins for the audit's decision. ⛔ Noworkflow_dispatchofrelease.ymlor any publish-capable workflow, and no approval of areleasedeployment: verification is by pins and simulation only. Stop on breach and explain in the report.
Container & model:M,mode:subagent,model: opus(dispatch-gates--tier: no path-derived mandate; floor sonnet · default opus · ceiling fable).
Clause-②: no
Thread-read: 5888151858
Serial constraints cleared: read at this stamp againstorigin/main013f97df9. Triage's serial ("after PR #20625") is met: PR #20625 merged as73213a521. The only open PR touchingrelease.ymlis PR #20945 (#20765, in the merge queue), a comment-only edit above the console-dist restore step (about:1381), which is region-disjoint fromrelease-integrity; whichever lands second mergesmain. The fleet's one-tooling-card cap is lifted for this dispatch by the maintainer's ruling at5920990276on #18549.
Generated by Claude Code
os-dev-report
{
"issue": 20627,
"status": "done",
"branch": "claude/issue-20627-backfill-waits-for-group",
"pr": "#20978",
"session": "session_017VaLJnYwhPsanVCe9dMCJU (subagent, so this is the parent's id; the container's CLAUDE_CODE_REMOTE_SESSION_ID is cse_017VaLJnYwhPsanVCe9dMCJU, the same id)",
"premise_still_valid": true,
"summary": "release.yml's release-integrity audit now requests no backfill (GitHub Releases, ADR-0087 D4 asset, runtime image) until the WHOLE fixed group is on npm. It reads that through scripts/release-verify-npm.mjs's own definition of published, via a new --probe mode: one round with budget 0, no second package list, and a set derived with --root from the VERSION COMMIT's tree via git worktree add. The probe runs only when something is missing, and the cli canary stays the publish predicate. H1: the Releases/D4 backfill AND the image backfill both keyed on the canary, and on run 36540562567 both ran inside the window (Releases 08:06:13-08:08:22Z, D4 08:08:22-08:08:27Z on 2026-09-29, with spec landing at 08:09:33Z). The publish predicate, the publish-job stale guard and the stale-prompts sweep stay on cli, with reasons in the PR. H2: the default mode cannot be the probe (15-minute retry; exit 1 conflates absent, unreadable and refused; its text says 'publish ran but'), and it imports only node built-ins, so it runs pre-install. H3: npm time for 17.5.0 is first 07:54:59.851Z, cli 07:58:57.256Z (67th of 69), console 08:05:24.355Z, spec 08:09:33.521Z (last). Battery 11 replays these.",
"tests": "All runs are on final head d9ab734 unless noted. (1) node scripts/release-verify-npm.mjs --self-test :: exit 0, 'OK release-verify-npm self-test: 82 cases pass across 12 batteries' (52 across 9 at base 3693a1b). New batteries: 10 is the probe (9 cases), 11 is the 17.5.0 window on npm's clock (7), and 12 runs the audit step's real run: text from release.yml under bash --noprofile --norc -eo pipefail in a throwaway repo with a 127.0.0.1 stub registry and stub npm/gh/curl (14). In battery 12, the in-window case gives exit 0 with no image-missing and no releases-missing; the all-published control gives image-missing=true and releases-missing=true. (2) Ablations, from committed fix fe32441 via node scripts/ablation-replace.mjs (anchor hit 1 to 0, blob changed; each restore proven by blob == HEAD and an empty git diff HEAD). A1: release.yml 'case "$group_state" in' to 'case "published" in' turns 4 of 82 red (3 in-window cases plus the unknown case; controls stay green). A2: '--probe --root "$vc_tree"' to '--probe --root .' turns 2 of 82 red (the new-package pair; the probe asked @objectstack/fresh). A3: groupState reverted to cli-canary semantics turns 13 of 82 red across batteries 10, 11 and 12. All three went red in the expected direction. Nothing is built: a root .mjs run directly by node, so there is no dist preflight. (3) Gates: dispatch-gates --commands --repo objectstack-ai/objectstack with no paths derived 50 commands, the same set as the PM's gates-pm.txt. All 50 :: exit 0 on d9ab734; --ran gives 'Run reconciliation: 50 derived, 50 run, 0 NOT-MEASURED, 0 UNRUN' and 'a DERIVED zero: all 50 recorded an exit code'. pnpm check:pm-dispatch-gates (both halves) ran detached as its header prescribes: EXIT=0, 'dispatch-gates self-test: 1976 cases pass', 'the battery took 1228.3s'. All 50 also exited 0 on fe32441 before the merge of origin/main (10 commits, none touching these files; same 50-command set after). (4) Extra on d9ab734: node scripts/release-pending-publish.mjs --self-test :: exit 0 (49 cases); pnpm check:release-body :: exit 0. (5) Live read-only probe: RELEASE_VERSION=17.5.0 node scripts/release-verify-npm.mjs --probe :: exit 0, '{"version":"17.5.0","state":"published","checked":69,"missing":[]}', about 20s through the container proxy. RELEASE_VERSION=17.4.0 :: exit 1 (anchor mismatch); no RELEASE_VERSION :: exit 1. (6) ESLint, narrowed: --print-config resolves a config for scripts/release-verify-npm.mjs, so it is in the population; --format json gives files 1, errors 0, warnings 0; there is no type-aware linting (no parserOptions.project), so no untouched file's verdict can move. pnpm lint is CI's. (7) No workspace package is touched, so there is no build closure and no pnpm test/typecheck owed.",
"mcp_calls": "0. No MCP GitHub tool was called.",
"api_writes": "3 relay dispatches (POST /repos/objectstack-ai/objectstack/dispatches), executing 4 endpoint writes as objectstack-fleet[bot]. (a) pr_create: run 36793644436, POST /repos/objectstack-ai/objectstack/pulls, draft #20978, body read back identical (11466 bytes). (b) label-write: run 36793692018, POST /repos//issues/20978/labels (skip-changeset) and POST /repos//issues/20978/assignees (os-bill); read back as size/l, skip-changeset and os-bill. (c) this os-dev-report comment: POST /repos//issues/20627/comments. git push is not counted. Every other GitHub contact was an unauthenticated REST GET: the card, its comments, runs 36540562567 and 36536081716 jobs, the 17.5.0 Releases, and PR 20978.",
"deviations": "(1) check:pm-dispatch-gates ran detached (nohup), as its own header prescribes for agent containers: the battery takes about 1220s, which is over the foreground cap. It was awaited with foreground 'timeout 580 tail --pid' blocks, and the exit was read from its exit file. A first attempt inside my batch runner was killed by that runner's own timeout 590 mid-battery, with 0 failures before the kill; it is not counted as a measurement. (2) The worktree did not exist at dispatch, so it was created off a freshly fetched origin/main 3693a1b (main had moved past 013f97d). It was merged with origin/main before the PR, and removed after. (3) scripts/release-verify-npm.mjs dropped its 'dispatch-gates: no-path-population' marker: battery 12 now reads .github/workflows/release.yml, so the family has a real population. The dispatch-gates self-test is green with that change.",
"files_changed": [".github/workflows/release.yml", "scripts/release-verify-npm.mjs"],
"line_budget": "none owed (no skills/**)",
"clause2": "Clause-②: no, kept. The diff widens no accept set and no published surface: --probe and --root are flags on an unpublished root script.",
"open_questions": [],
"out_of_scope_findings": [
"class: b · reach: named real producer, release.yml release-integrity's 'Backfill spec-changes.json on the GitHub Release (ADR-0087 D4)' step. Run 36540562567 job 109314685158 step 8 (08:08:22-08:08:27Z on 2026-09-29) uploaded an asset built from landing 7510663, whose packages/spec/src differs from the shipped 0f6dcac by +108 lines (packages/spec/src/migrations/registry.ts and semantic/18.ui-html-page-div-refused.ts). The asset bytes are NOT MEASURED: the publish job's --clobber replaced them at 08:10:59Z. Contract, quoting that step's own comment: '--prepareregenerates the manifest against the previously published tarball exactly as the publish lane does, so the asset this repair uploads carries the same per-release section the npm artifact does.' Seam: release.yml declared contract → scripts/release-spec-changes.sh --prepare reads the checkout (github.sha), while the publish job builds the asset from the version commit (ADR-0125 D1 as amended). Consumer: the spec GitHub Release's spec-changes.json. The Releases backfill has the same shape (target_commitish and the CHANGELOG permalink are github.sha). Dedupe words: 'D4 backfill landing tree' · 'spec-changes.json backfill github.sha not version commit' · 'release-integrity backfill reads the landing tree'",
"carrier: none, noted in PR #20978's Acceptance notes and not filed. It is the residual in-flight race, the half of this card's triage direction marked optional ('stand down while a publish job for the same version is in progress'), and it is not implemented here: it needs Actions reads (release-pending-publish.mjs's version-commit and head_sha walk) and actions: read on release-integrity, both outside the claimed file surface. After the group completes, the publish job still runs its Releases (08:09:53-08:10:57Z), D4 attach (08:10:59Z) and docker (08:11:17-08:15:22Z) on 17.5.0. A landing audited then backfills alongside it: a duplicate image build (green), a find-then-POST Releases race (possible 422), and a D4 --attach --clobber that can land after the publish's. This PR narrows the race rather than introducing it: on 17.5.0 timings the persistent-clobber landing window was about 155s before and about 85s after, and the red-image window went from about 10 min to none. This is a timeline derivation, not a measured occurrence."
]
}
Generated by Claude Code
ACCEPT — PR #20978 at head
d9ab734556·domain:specseat 2 (session_017VaLJnYwhPsanVCe9dMCJU) · 2026-10-01T00:11ZReviewed against GitHub (the PR object, its file list, its diff and the head's check-runs), not against report
5921981370.- Shape: draft, base
main, first lineFixes #20627and no other closing keyword,Clause-②: noat line start. That matches the diff: a workflow step and flags on an unpublished root script. - Scope: 2 files inside claim
5921035518:.github/workflows/release.yml(therelease-integrityaudit step only) andscripts/release-verify-npm.mjs.skip-changesetand the PR assigneeos-billare on the PR. - The audit, read in the diff: the Releases/D4 and image probes run first, and nothing missing ⇒ exit 0 with no group read. When something is missing, the whole fixed group is read once through
release-verify-npm.mjs --probe, the one definition of "published" the publish job uses, with no second package list. The set is taken from the version commit's workspace (--rooton a detached worktree ofversion_commit).unpublished⇒ exit 0 with a step-summary note and no backfill;unknown⇒ exit 0 with no backfill; a probe that cannot be read or an unknown state ⇒ exit 1. Theclicanary stays the publish predicate, as triage kept it. - Release-lane fence held: no
workflow_dispatch, no deployment approval, no publish. The one live read was a read-only--probeof 17.5.0 (published, 69 of 69). - Evidence read: battery 12 runs the audit step's real text from
release.ymlunder bash against a stub registry. The 17.5.0 in-window case backfills nothing, and the all-published control requests both backfills. The ablations went red as expected: thecaseforced topublished(4 red),--root .(2 red), and canary semantics (13 red). Each was restored by blob equality. - Review face: none of the five faces is touched, so CI plus this seat read; no at-tier record is owed. 640 changed lines.
- CI at
d9ab734556, read at this stamp: 23 success, 11 roster skips (filter or label gated), andLint & Repo Gatesin progress. Landing waits for it. - Findings:
- The D4/Releases backfill builds from the landing push, not the version commit: filed [finding] release.yml: release-integrity's Releases / ADR-0087 D4 backfill builds from the landing push (
github.sha), not the version commit the publish built from #20982 (class a, named producer, measured run36540562567; bytes NOT MEASURED, as the card says). - The residual in-flight race (the optional "stand down during a publish" half of triage's direction): Acceptance notes. This PR narrows it and introduces none of it. Carrier: none.
- The D4/Releases backfill builds from the landing push, not the version commit: filed [finding] release.yml: release-integrity's Releases / ADR-0087 D4 backfill builds from the landing push (
Generated by Claude Code
- Shape: draft, base
Landed · PR #20978 MERGED through the merge queue as
587c84e6d5(single parent7fa67dada) ·domain:specseat 2 (session_017VaLJnYwhPsanVCe9dMCJU) · 2026-10-01T01:03Z- Verified by content on
origin/main, not by the merged flag:.github/workflows/release.ymlandscripts/release-verify-npm.mjshave the same blobs onorigin/mainas at the ACCEPTed headd9ab734556. The merge commit is an ancestor oforigin/main. - Closing keywords: the body carried
Fixes #20627only. This card closedcompleted, and no other card was named. - State:
pm:dispatchedcomes off in this act.domain:spec,area:devpath,bug,toolingand the priority stay. - What it changes: from the next landing on,
release-integritybackfills nothing for a version until the whole fixed group is on npm. The follow-up on the backfill's source tree is [finding] release.yml: release-integrity's Releases / ADR-0087 D4 backfill builds from the landing push (github.sha), not the version commit the publish built from #20982.
Generated by Claude Code
- Verified by content on
Filing gate: ① a defect with a named landing site and a measured
reach:. Finding class (a).reach:the release workflow onmain: run36540562567(push7510663c87, 2026-09-29T08:05Z) is red.Filed by the
domain:specexecution seat 2 (session_014EJ1ED8X4MMrT18BhVx4tx, seat post #18549), from the #20613 dev report (out-of-scope finding 3) and PR #20625's at-tier record5888029963(escalation 2). Both flagged it; the seat measured the timeline below. ⛔ Filed bare: routing and grading belong to triage. ⛔ Not a claim. It is outside #20613's surface: PR #20625 changes which push queues a publish, not what the backfill reads.What happened (measured, 2026-09-29)
36536081716: itsPublish 17.5.0 to npmjob ran from 07:40:26Z to 08:11:15Z.@objectstack/cli@17.5.0reached npm at 07:58:57Z (npmtime). Other packages of the fixed group landed at other times;@objectstack/spec@17.5.0landed at 08:09:33Z (npmtime).7510663c87, started run36540562567. Itsrelease-integrityaudit (08:05:53Z) readcli@17.5.0as published, found the runtime image missing, and took the no-mint backfill branch.Docker image / Build & pushjob'sBuild and pushstep ran 08:08:58Z–08:09:32Z and failed, one second before@objectstack/spec@17.5.0existed on npm. The job log is not readable from a seat container (the log download redirect is refused by the proxy), so the error text is NOT MEASURED; the timing is.mainskips the docker job. So nothing is missing now: the cost is a red release run onmain, a wasted build, and a false alarm on the release lane.Why
release-integrityin.github/workflows/release.ymltreats "the@objectstack/cliversion is on npm" as "the release is published". The publish job pushes about 69 packages over several minutes, andcliis not the last. Any landing inside that window reads a half-published group as complete, and the backfill then builds from npm too early.Suggested shape (⛔ not a ruling)
release-verify-npm.mjsruns after publish), not theclicanary; or it stands down while arelease-publish-*job is in progress.cliis on npm andspecis not answers "not yet published", with the all-published control.release-integrity, so the race stays until this is fixed.Dedupe
A REST listing of the 1,000 most recently updated issues and PRs (down to #19850), grepped locally for
backfillneardocker/image/in-flight/race,in-flight publishandimage-missing, found only PR #20625, which names this as out of scope. No issue carries it.Dedupe words:
release-integrity backfill in-flight publish·docker backfill spec not on npm·cli canary read as publishedGenerated by Claude Code