Repository navigation
publishPackageDrafts validation closure omits same-batch dataset drafts: a package shipping dashboard+dataset together can never publish (widget-dataset-unknown) #10377
Description
Symptom (P0 for AI builds — blocks epic objectstack-ai/cloud#1472 acceptance)
Live repro 2026-08-21, cloud local rig, all-latest mains (objectstack 0b3cf876+, cloud 0b3cf876). An AI-built package app.shyx carries object/*, view/*, dataset/shyx_customer_ds, dashboard/customer_dashboard (widget bound to that dataset), flow/*, seed/* — all drafted together. Every publishPackageDrafts attempt rolls back:
[protocol.publishPackageDrafts] batch publish of 'app.shyx' rolled back at dashboard/customer_dashboard:
[invalid_metadata] dashboard/customer_dashboard failed author-time validation: dashboards[0].widgets[0]:
[widget-dataset-unknown] dataset "shyx_customer_ds" does not resolve to a declared dataset.
The dataset exists in the same batch (sys_metadata row dataset/shyx_customer_ds, package app.shyx, state draft). The build agent retried with a second dataset name (shyx_customer → shyx_customer_ds) — same refusal both times, so a package that ships a dashboard together with its dataset can NEVER publish. Downstream: the entire AI magic-flow build stays draft (post-cloud#1467 the tools now honestly report "PUBLISH FAILED — NOT LIVE", which is how this surfaced immediately).
Root cause (source-level read, unverified by test yet)
validateWidgetBindings resolves widget.dataset against the caller-supplied datasets context collection (packages/lint/src/runtime-gate.ts — the #7529 comment says exactly that a missing collection makes every legitimate widget read as dangling). The publish gate's per-draft validation context appears to carry only ALREADY-LIVE dataset declarations, not the same batch's pending dataset drafts. The intra-batch closure works for flow→object (a flow referencing a same-batch object publishes fine — verified live on 2026-08-20), so this is a per-collection gap in the #9612/#10058 "judge a package write against its own closure" work: the closure includes objects but not datasets (and possibly not the other context collections — permissions, books — worth auditing in the same change).
Fix requirements
publishPackageDrafts's author-time validation context must include the batch's own pending drafts for every context collection the closure judgment carries (objects,permissions,books,datasets) — a package must be publishable as a self-consistent unit.- Regression test: a batch carrying dashboard + its dataset (dashboard validated before the dataset is promoted) publishes clean; a dashboard bound to a genuinely absent dataset still rolls back with
widget-dataset-unknown. - Audit the remaining context collections for the same gap in the same change (cheapest now, one closure seam).
Repro
Stage a package with dataset/x + dashboard/d (widget bound to x) as drafts, call publishPackageDrafts — rolls back at the dashboard regardless of item order.
Related: #9612 / #10058 (closure judgment), #7529 (datasets context), cloud#1472 (blocked epic), cloud#1467 (honest reporting that surfaced this).
Activity
os-zhuang commentedon Aug 20, 2026
Claim: PM loop round 3 (epic objectstack-ai/cloud#1472 — derived in-scope card: blocks epic acceptance)
Session:
session_3f79f27a-67a1-44c5-b9e3-f9e08b0aba46Branch:
claude/issue-10377-publish-closure-datasetsWorktree:
objectstack-issue-10377Domain: epic lane; lands in
packages/metadata-protocol(publish-gate context assembly) and/orpackages/lint(closure seam)File surface:
packages/metadata-protocol/**,packages/lint/src/runtime-gate*.ts+ tests + changeset (stop on breach; explain in the report)Container & model:
M/L, mode:subagent, model: opusClause-②: no (restores the declared closure semantics of #9612/#10058 — a package publishable as a self-consistent unit; no acceptance-set expansion beyond what the closure ruling already declared)
Serial constraints cleared: no sibling in flight; round-1/2 PRs all merged (#10291 touched publishMetaItem per-item path — read its diff, your seam is the BATCH door's validation context, adjacent but distinct).
fix(metadata-protocol): judge a package publish against the batch's o…
os-zhuang commentedon Aug 20, 2026
{ "issue": 10377, "status": "done", "branch": "claude/issue-10377-publish-closure-datasets", "pr": "https://github.com/objectstack-ai/objectstack/pull/10389", "premise_still_valid": true, "summary": "publishPackageDrafts built its author-time validation context from engine.registry alone (the already-LIVE universe), and the batch's own promotions do not reach that registry either: applyRegistryWriteThrough runs in Phase 2, AFTER the Phase-1 transaction that gates and promotes every draft. So no member of a batch was visible to any other member, in any order. The batch door now reads its own pending drafts ONCE before any promotion (a promote deletes the row it reads) and folds them into all four closure collections (objects, permissions, books, datasets), replacing a live declaration of the same name rather than sitting beside it; a name in neither the batch nor the live universe is still refused unchanged. PREMISE REFINEMENT (the issue's root-cause section is a source-level read, and one inference in it is wrong): the closure did NOT carry objects while omitting datasets - no collection carried same-batch drafts. flow->object publishes fine because validateFlowTriggerReadiness reports flow-trigger-unknown-object at severity 'warning', while validateWidgetBindings raises widget-dataset-unknown at severity 'error', which refuses the promotion and (the batch being all-or-nothing, ADR-0067 D2) aborts the package. The gap was uniform; only its severity differed per collection. The fix requirement is unchanged by that. AUDIT of all four (fix requirement 3): datasets = refusing gap; objects and permissions = the same gap at advisory severity (both pinned); books = NO observable gap today - both readers of stack.books iterate books to produce findings ABOUT books and nothing resolves a reference INTO the collection, so a pending book draft's findings cancel in the gate's differential either way; it is carried anyway so a future rule that does resolve into it cannot inherit the gap. #10291 (per-item door): its seam is onMetaItemPublished plus draft-org-scope resolution, not gate-context assembly, so the two doors stay coherent. The per-item door has NO analogous gap: publishMetaItem publishes one item, so its batch is itself and buildRuntimeWriteSnapshots already grafts the written item into the candidate snapshot. promoteDraftForPublish therefore accepts the new optional 'pending' argument and the per-item caller states nothing. PATCH ROUND (CI red on the first commit, now addressed at head 3495477a9): the new read introduced the batch door's FIRST dependency on repo.get, which silently widened what the door requires of an overlay repository - getOverlayRepo is the seam every publish double replaces, and the door's only previous body read (the seed capture) fires solely when a seed draft is in the batch, so a shape without get had never been asked for one. Nine cases in packages/objectql/src/protocol-publish-package-drafts.test.ts (a SIBLING package's suite) threw TypeError: repo.get is not a function BEFORE any promotion. Fixed with a declared capability check, not a wider try: typeof repo.get !== 'function' degrades to the pre-#10377 closure (live universe alone - the gate keeps judging and can only be MORE strict, never fail open), all-or-nothing so the verdict cannot depend on which org a draft lives in, and the degrade SAYS WHY once per process (which member was missing, where first reached, and that a same-batch sibling reference may therefore be refused). A read failure on a repository that HAS the member still propagates - unchanged. The batch's existing enumeration could NOT supply the bodies instead: listDrafts is a declared header projection whose other caller is the console's pending-changes list, and the doubles that lack repo.get stub listDrafts too, so a body-carrying projection would hand back headers there anyway - degrading SILENTLY rather than avoiding the guard. A second, self-caught regression in the same round: the new assertion's callback inferred implicit any, one new raw tsc error, caught by check:type-check-debt --re-measure as upward drift of a shrink-only ratchet (63 recorded, 64 measured); fixed at the source, ledger entry untouched.", "tests": "All commands run in the worktree on Node v25.9.0; exit codes captured before any pipe. FINAL HEAD 3495477a9 (3 commits: 85a1e89df fix, 9caa98fd1 repo-shape guard, 3495477a9 typecheck repair). ROUND 1. (1) REPRO FIRST, before any source edit: npx vitest run --maxWorkers=2 src/protocol-publish-drafts-closure.test.ts -> EXIT=1, the card's symptom verbatim including the gate's own hint \"Declared datasets: (none).\" on [widget-dataset-unknown] dataset \"shyx_customer_ds\" does not resolve to a declared dataset. (2) ABLATION of the merge (mergePendingDeclarations forced to return the live collection), run from the COMMITTED state, mutation confirmed on disk by grep anchored on BOTH texts (injected marker count 1, bypassed-body anchor supersededNames count 4): \"Tests 4 failed | 4 passed (8)\" - exactly the four discriminating expects-silence cases red (2 datasets, 1 objects, 1 permissions), the four boundary cases green. Restore leg: marker absent (0), git status --porcelain empty, 8 passed. No rebuild needed and none claimed there - that subject is imported relatively from the package's own src. PATCH ROUND. (3) The exact CI-failing file, in the package that owns it: cd packages/objectql && npx vitest run --maxWorkers=2 src/protocol-publish-package-drafts.test.ts -> EXIT=0, \"Test Files 1 passed (1)\", \"Tests 17 passed (17)\". Full package: pnpm --filter @objectstack/objectql test -> EXIT=0, \"Test Files 224 passed (224)\", \"Tests 3939 passed (3939)\". (4) pnpm --filter @objectstack/metadata-protocol test -> EXIT=0, \"Test Files 127 passed | 2 skipped (129)\", \"Tests 1734 passed | 10 skipped (1744)\". (5) pnpm --filter @objectstack/lint test -> EXIT=0, \"Test Files 77 passed (77)\", \"Tests 2137 passed (2137)\". (6) Own closure suite -> EXIT=0, \"Tests 9 passed (9)\" (8 from round 1 plus the new minimal-repo-double regression, which also asserts the degrade notice is emitted). (7) ABLATION OF THE GUARD, with the dist rebuild this cross-package path requires - objectql imports @objectstack/metadata-protocol BY PACKAGE NAME, so the mutation must reach dist/, not just src/. First attempt used a COMMENT marker and was correctly refused by scripts/ablation-dist-preflight.mjs: \"marker found ONLY in 2 sourcemap files and in no executable output -- treat this run as void\"; redone with an executable marker (the guard condition mutated to a never-true typeof comparison). Mutation leg: src greps confirm injected marker 1 and deleted text \"typeof repo.get !== 'function'\" 0; rebuild; preflight \"marker present in 2 built files -- the ablation is live in the artifact the suite consumes\"; suite -> EXIT=1, \"Failed Tests 9\", every one TypeError: repo.get is not a function - CI reproduced exactly. RESTORE leg: src greps confirm marker 0 and guard restored 1; rebuild EXIT=0; preflight --absent \"marker absent from all 22 built files\"; suite -> EXIT=0, 17 passed; git status --porcelain empty against the commit. (8) TYPECHECK, measured rather than assumed: metadata-protocol declares no typecheck script (pnpm --filter ... typecheck exits 1 with ERR_PNPM_RECURSIVE_RUN_NO_SCRIPT - loud, not the silent zero-match green), so npx tsc --noEmit was run directly. origin/main baseline 63 raw errors; the patch round measured 64; located as src/protocol-publish-drafts-closure.test.ts(436,43) error TS7006 (implicit any); fixed; back to 63. (9) GATE UNION re-derived AFTER the final commit with node scripts/pm/dispatch-gates.mjs (no paths) at 3495477a9. Every family it named ran green: check:changeset-gate-self-tests, check:cross-package-test-inputs, check:durability-log-level, check:filter-alias-parity, check:objectui-changeset, check:slot-lookup, check:query-options-erasure, check:type-check-coverage, check:engine-double-contract, check:where-matcher, check:nul-bytes (all EXIT=0); plus node scripts/check-adr-0087-registration.mjs, check-changeset-no-major.mjs, check-empty-changeset.mjs, docs-audit/check-affected-docs.mjs (all EXIT=0). The convention-triggered ratchet check:type-check-debt needs the built closure, so the workspace build ran first -> EXIT=0, \"Tasks: 70 successful, 70 total\"; then check:type-check-debt -> EXIT=0, its own verdict line: \"check-type-check-coverage --re-measure: OK - 33 ledger entr(ies) re-measured in 84.5s, 1924 raw tsc error(s) total, none above its recorded number.\" check:engine-double-contract had, in round 1, printed \"x RETAINED ... the ledger just has to learn about it\"; repaired with its own --write as instructed, and its verdict is now \"check-engine-double-contract: OK - 340 pinned, 133 in the DEBT ledger, 2 exempt.\"", "open_questions": [], "out_of_scope_findings": [] }fix(metadata-protocol): degrade the batch closure on a repository tha…
fix(metadata-protocol): type the degrade-notice assertion's callback
fix(metadata-protocol): judge a package publish against the batch's o…