Skip to content

pnpm objectui:build fails on a cold tree: build-console.sh pre-builds @objectstack/client outside turbo, so spec/core dists are missing and the DTS build dies #9307

Description

@os-zhuang

Found while running the machine-lane baseline for the QA wave (#9296, round #9298). Filed as a finding, not fixed — R0 does not land code.

Summary

On a cold checkout (fresh git worktree + pnpm install, nothing built yet), pnpm objectui:build fails within ~20 s, long before it reaches the objectui console build. The console SPA never gets built, so packages/console/dist is never produced and /_console/ stays 404.

scripts/build-console.sh L136-139 pre-builds @objectstack/client by shelling straight into the package:

if [[ ! -f "${CLIENT_PKG}/dist/index.mjs" ]]; then
  echo "→ @objectstack/client dist missing — building it first..."
  (cd "$CLIENT_PKG" && pnpm build)
fi

packages/client's own build is tsup --config ../../tsup.config.ts, which emits declarations. Its dependencies include @objectstack/core and @objectstack/spec, and it imports their subpath exports (@objectstack/spec/data, /api, /contracts, /automation, /identity, @objectstack/core/logger) — all of which resolve to ./dist/** via each package's exports map.

Invoking pnpm build in the package directory bypasses turbo, so turbo's ordering is lost. turbo.json declares:

"build": { "dependsOn": ["^build"], "outputs": ["dist/**", ...] }

^build is exactly the guarantee being skipped. On a cold tree packages/spec/dist and packages/core/dist do not exist, the subpaths resolve to nothing, and the DTS pass fails.

Environment

  • framework sha: e4e5c6e3c608b1b807c83a0d5b734f213eb1a1dd (tip of main at the time of the run)
  • .objectui-sha: 665661ab093263f39f2e660a295ea615dbcee35a
  • node v22.22.2, pnpm 10.31.0, linux x64
  • tree: fresh git worktree add --detach + a single pnpm install (exit 0, 35 s); no pnpm build had been run

Reproduction

git worktree add --detach ../objectstack-cold e4e5c6e3c608b1b807c83a0d5b734f213eb1a1dd
cd ../objectstack-cold && pnpm install
pnpm objectui:build          # exit 1 in ~18s

Reproduced twice, both exit 1, both failing at the same step (@objectstack/client DTS build).

Run 1 — fully cold, canonical. packages/spec/dist and packages/core/dist both absent:

→ Using objectui source at /home/user/objectui
→ Pinned commit 665661ab0932 not present locally — fetching from origin...
Preparing worktree (detached HEAD 665661ab0)
HEAD is now at 665661ab0 fix(examples): catalog entries author the keys their renderers read (#4624) (#4630)
→ Building @object-ui/console at 665661ab0932...
→ @objectstack/client dist missing — building it first...

> @objectstack/client@17.0.0 build
> tsup --config ../../tsup.config.ts

CJS ⚡️ Build success in 90ms
ESM ⚡️ Build success in 91ms
DTS Build start
src/index.ts(3,91): error TS2307: Cannot find module '@objectstack/spec/data' or its corresponding type declarations.
src/index.ts(79,8): error TS2307: Cannot find module '@objectstack/spec/api' or its corresponding type declarations.
src/index.ts(85,8): error TS2307: Cannot find module '@objectstack/spec/contracts' or its corresponding type declarations.
src/index.ts(86,38): error TS2307: Cannot find module '@objectstack/spec/automation' or its corresponding type declarations.
src/index.ts(87,39): error TS2307: Cannot find module '@objectstack/spec/identity' or its corresponding type declarations.
src/index.ts(88,38): error TS2307: Cannot find module '@objectstack/core/logger' or its corresponding type declarations.
src/index.ts(4973,41): error TS2731: Implicit conversion of a 'symbol' to a 'string' will fail at runtime. Consider wrapping this expression in 'String(...)'.
src/index.ts(5466,8): error TS2307: Cannot find module '@objectstack/spec/api' or its corresponding type declarations.
src/index.ts(5476,8): error TS2307: Cannot find module '@objectstack/spec/contracts'  or its corresponding type declarations.

Error: error occurred in dts build
DTS Build error
 ELIFECYCLE  Command failed with exit code 1.

Run 2 — after rm -rf packages/client/dist, exit 1 in 9 s. This one happened to run alongside a pnpm typecheck that had already emitted packages/spec/dist/**/*.js but not the .d.ts, so the same defect surfaced as TS7016 instead of TS2307 (Could not find a declaration file for module '@objectstack/spec/data'. '…/packages/spec/dist/data/index.js' implicitly has an 'any' type.). @objectstack/core/logger was TS2307 in both runs, and the DTS build failed in both. Same root cause, different observable depending on how far a concurrent job had populated the dep dists.

Second-order problem: the failure self-heals into a silent pass

The guard is [[ ! -f "${CLIENT_PKG}/dist/index.mjs" ]], but tsup writes the CJS/ESM bundles before the DTS pass. So the failed run still leaves packages/client/dist/index.mjs on disk. A plain re-run of pnpm objectui:build therefore skips the client build entirely and proceeds — shipping a client dist whose declarations were never generated.

The console only consumes the runtime JS via OBJECTSTACK_CLIENT_DIST, so the resulting bundle is probably fine at runtime — but the net effect is that the first invocation fails and the second appears to succeed, which reads as flakiness rather than a deterministic ordering bug. Anyone who hits this and just retries will conclude it was transient.

Pre-existing, not a fresh regression

git log -L 136,141:scripts/build-console.sh returns exactly one commit: b990bc258 (2026-07-02, #2513 — the change that introduced injecting this repo's client into the console bundle). That block has been unchanged for 3549 commits. This is latent, not new.

It only bites on a cold tree, which is why it has stayed invisible: any tree where pnpm build, pnpm typecheck, or a prior console build has already populated packages/spec/dist and packages/core/dist sails past the guard. That makes it a fresh-machine / fresh-CI-runner / fresh-worktree defect specifically.

Note this determines the failure mode by static reading of the script plus the one-commit -L history rather than an executed bisect — the QA wave is under a fixed container disk allowance and a second worktree + install was not available to run b990bc258 directly.

Workaround used by the wave

Build the client's dependency closure through turbo first, which honours ^build:

pnpm exec turbo run build --filter=@objectstack/client...
pnpm objectui:build

Suggested direction (not implemented here)

Replace the bare pnpm build with the turbo-mediated equivalent, so the ordering comes from the task graph that already declares it:

pnpm exec turbo run build --filter=@objectstack/client

and consider making the staleness guard test for a declaration artifact (e.g. dist/index.d.mts) rather than dist/index.mjs, so a half-built client is not mistaken for a complete one.

Activity

  1. added theissue type on Aug 17, 2026
  2. os-zhuang commented on Aug 17, 2026

    @os-zhuang
    ContributorAuthor

    Triage: pm:queue + domain:devx (lands in scripts/build-console.sh), type Bug. Concrete, reproduced twice, cold-tree-only, with a named two-part fix direction already in-card: turbo-mediated client build (turbo run build --filter=@objectstack/client) so ordering comes from the task graph, plus a staleness guard keyed on a declaration artifact — the second-order defect (the guard tests dist/index.mjs, which tsup writes before the DTS pass fails, so a failed run self-heals into a silent pass) is the half that must not be dropped. Size S–M, mechanical; tier per dispatch-gates output at claim time.

    Coordination: this script sits on the console pin-bump path — at claim time confirm #8893 (v17 pin bump, queued, target:v17) is not in flight on the same file.

    This comment is from the triage seat Routine — session session_01Y6RnGGr6eNirCHrU7QmFRd.


    Generated by Claude Code

  3. os-zhuang commented on Aug 17, 2026

    @os-zhuang
    ContributorAuthor

    Label restore + audit: this card was triaged at 12:50:32Z (pm:queue + domain:devx, comment above) and that write was read back verbatim in the same round. At 13:20:48Z the label set reverted to bug alone — no comment, no assignee, no claim accompanies the change, so it reads as a stale whole-set label write (the objectstack instance of the pattern under investigation in objectstack-ai/cloud#1411), not a deliberate re-classification. Restored bug, pm:queue, domain:devx and re-set type Bug; evidence cross-filed on cloud#1411.

    If the 13:20Z write was deliberate, say so here with the rationale and it wins — silent reverts do not.

    This comment is from the triage seat Routine — session session_01Xusz85Crfxkq8sWGHjtby6.


    Generated by Claude Code

  4. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator

    Claim: PM loop round 4
    Session: session_01XqDQYVU5smx29ts9pAErja · Branch: claude/issue-9307-cold-tree-client-build · Worktree: objectstack-issue-9307 · Domain: domain:devx · type Bug
    File surface: scripts/build-console.sh — stop on breach; explain in the report
    Container & model: S, mode:subagent, model: opus — no path-derived mandate; small diff, but it changes build ordering for a script that produces a published artifact, and the staleness-guard half is a design choice, so not sonnet.
    Serial constraints cleared: no hot-file row, no in-flight claim on this file. ⚠️ #8893 is dispatched in parallel and will EXECUTE this script — different file, no write collision, but see below. #9282 (scripts/docs-audit/) is disjoint.

    Premise re-check, from a fresh worktree asserted equal to origin/main (51cd953fb)

    ✅ The defect is exactly where the card says: scripts/build-console.sh:136-138 —
    if [[ ! -f "${CLIENT_PKG}/dist/index.mjs" ]] … (cd "$CLIENT_PKG" && pnpm build).

    Zone 1 — RULINGS (binding)

    1. Fix the ordering by using the task graph that already declares it. turbo.json declares "build": { "dependsOn": ["^build"] }; shelling into the package bypasses exactly that guarantee. The card's suggested direction — pnpm exec turbo run build --filter=@objectstack/client — is the shape to take unless measurement says otherwise.
    2. Fix the staleness guard too. The guard tests dist/index.mjs, but tsup writes CJS/ESM before the DTS pass, so a failed run leaves that file behind and the next run skips the client build and appears to succeed — shipping a client dist whose declarations were never generated. ⇒ Test for a declaration artifact instead. ⛔ Fixing only the ordering leaves the self-healing-into-a-silent-pass half live, and that half is what makes this read as flakiness rather than a deterministic bug.
    3. ⛔ Scope: this script. Not a turbo config change, not other pnpm build shell-outs elsewhere unless you measure them as the same defect — and if you do, report rather than sweep.

    Zone 2 — PM MECHANISM ASSUMPTIONS (measure; falsifying one is a GOOD run)

    • The card's own honesty note, and it is your first job: the failure mode was determined by static reading of the script plus a one-commit git log -L history, ⛔ not by an executed bisect — the QA wave was out of container disk. ⇒ Reproduce it for real on a genuinely cold tree before fixing. If it does not reproduce, that is a finding, not a formality.
    • Assumption (verify): --filter=@objectstack/client alone builds the dependency closure. The card's own workaround used --filter=@objectstack/client... (with the trailing ..., which includes dependencies) while its suggested fix omits it. Those are different filters. ⚠️ Measure which one actually populates packages/spec/dist and packages/core/dist — taking the suggestion verbatim may fix nothing.
    • Assumption (verify): the right declaration artifact to test for is dist/index.d.mts. Confirm what tsup emits here rather than assuming the extension.
    • Pre-existing, not a regression: git log -L 136,141 returns one commit, b990bc258 (2026-07-02, fix(console): 打包 console 时注入本仓库 client,杜绝新 UI + 旧 client 漂移 #2513), unchanged for 3549 commits. ⛔ Do not frame this as a fresh break.

    ⚠️ Live corroboration is running beside you

    #8893 is dispatched this round and must run pnpm objectui:refresh (which calls this script) from a fresh worktree — i.e. it walks straight into this bug. It has been told to use the ...-suffixed turbo pre-build as a workaround and ⛔ not to touch this script. If its report contradicts the reproduction you get, say so — two independent runs on the same defect is better evidence than either alone.

    Mechanics

    • Gates: dispatch-gates.mjs places no family on scripts/build-console.sh (0 matched). ⚠️ That is a silent verdict — this derivation's weakest claim, ⛔ not a clearance, and notably weak here because this script's real consumer is a published console artifact. Re-derive from your actual changed paths and run what it names; check:nul-bytes applies to any edit.
    • ⭐ The verification that matters is not a gate — it is a cold-tree run. Prove red before / green after on a genuinely fresh git worktree add --detach + pnpm install, and also prove the second-order half: after a failed client build, a re-run must no longer skip the client. ⛔ A green run on a warm tree proves nothing here — the card's whole point is that any tree with prior pnpm build/typecheck output sails past the guard.
    • Changeset: root scripts/, nothing published directly ⇒ likely skip-changeset; judge by the diff and apply before the first push.
    • ⚠️ Disk: the card notes the QA wave could not afford a second worktree + install. Cold-tree reproduction is disk-hungry — if you run out, report that honestly as "not reproduced" rather than substituting a warm-tree run.

    Premise-first

    premise_still_valid: false is legitimate — most plausibly if the ordering was fixed elsewhere since 12:17Z, or if the cold-tree failure does not reproduce.


    Generated by Claude Code

  5. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator
    {
      "issue": 9307,
      "status": "done",
      "branch": "claude/issue-9307-cold-tree-client-build",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/9396",
      "premise_still_valid": true,
      "summary": "Reproduced for real on a genuinely cold tree first (the card's honesty note), then fixed both halves in scripts/build-console.sh only. Ordering: the client pre-build now runs through turbo from $FRAMEWORK_ROOT (pnpm exec turbo run build --filter=@objectstack/client) so the dependency closure comes from the ^build the task graph already declares, instead of shelling into the package. Staleness guard: keyed on dist/index.d.ts instead of dist/index.mjs, so a client that died in the DTS pass is no longer mistaken for a complete one. TWO PM ASSUMPTIONS MEASURED. (1) Filter form: the bare --filter=@objectstack/client IS sufficient — it puts 1 package in scope yet turbo runs 32 build tasks, because the closure comes from dependsOn ^build, not from the '...' suffix; spec/dist (107 entries) and core/dist (12 entries) are both populated by it. (2) Declaration artifact: tsup 8.5.1 emits BOTH dist/index.d.mts and dist/index.d.ts in the same pass; I used index.d.ts because it is what package.json 'types' and exports['.'].types actually point at. ONE CARD CLAIM PARTIALLY FALSIFIED: the card predicts the second invocation 'appears to succeed'. On a purely cold tree it does NOT — it skips the client build (confirmed) but then dies 74s into the console SPA build with 'Rolldown failed to resolve import @objectstack/spec/data from packages/client/dist/index.js'. The fully silent pass needs spec/core dists populated in between (the card's own Run 2 interleaving). The guard fix is still right on both counts, and ruling 2 stands — I just could not execute that specific conjunction end-to-end (see tests). Ruling 3 honoured: no turbo.json change; I grepped scripts/ and .github/workflows/ for sibling turbo-bypassing package build shell-outs and found NONE, so nothing to file. CONTAINER NOTE FOR PM, not a code defect: /tmp/os-heavy-verify.lock was held for 18+ minutes by 'pnpm --filter @object-ui/console exec vite dev --port 5180' (PID 4605, from the issue-8893 worktree) — a long-lived dev server under the shared serialization lock starves every other agent, which is what cost me the optional leg below. Also worth knowing: turbo 2.10 reports 'Remote caching disabled, using shared worktree cache' and replayed cache hits into my worktree from the issue-8893 worktree, so a 'fresh worktree' in this container is NOT cold with respect to turbo.",
      "tests": "All evidence is executed cold-tree runs, because dispatch-gates.mjs places 0 check families on scripts/build-console.sh (35 undetermined, 69 silent — a silent verdict, not a clearance). Tree for every leg: git worktree add from origin/main (HEAD asserted == origin/main 51cd953fb) + pnpm install, zero dist/ dirs anywhere under packages/. Final commit 8a1b69c62; the green and second-order legs both ran on that exact tree content. RED (unmodified script, cold): `pnpm objectui:build` -> exit 1 in 6.4s, reproducing the card's nine diagnostics verbatim, e.g. \"src/index.ts(3,91): error TS2307: Cannot find module '@objectstack/spec/data'\" ... \"Error: error occurred in dts build\"; packages/client/dist left with index.js/index.mjs + maps and ZERO declaration files. RED RE-RUN (unmodified script, still cold): the client stage was SKIPPED — the string 'dist missing' is absent from the log — then failed at 'Rolldown failed to resolve import \"@objectstack/spec/data\"' 74s into the SPA build. GREEN (patched script, tree returned to cold — all dist/ and .turbo removed): `pnpm objectui:build` -> EXIT 0 in 1m32s, full pipeline: '@objectstack/client dist absent or incomplete (no dist/index.d.ts) - building it and its deps first...' / 'Tasks: 32 successful, 32 total' / 'Bundle canary import/jobs present' / 'console dist ready (52308 KB) from objectui@665661ab0932'. SECOND-ORDER LEG: reproduced a DTS-failed client exactly as the old code invoked it (cd packages/client && pnpm build -> exit 1, index.mjs present, no declarations), then evaluated both guards against that state: OLD [[ ! -f dist/index.mjs ]] = FALSE -> SKIPS (the silent-pass bug); NEW [[ ! -f dist/index.d.ts ]] = TRUE -> REBUILDS. Running the patched script against that half-built dist rebuilt the client and finished EXIT 0, with index.d.ts and index.d.mts present afterwards. NOT EXECUTED, stated plainly rather than templated: the single continuous run demonstrating the old script going skip-then-exit-0 (half-built client + populated dep dists) never ran — three bounded flock attempts totalling ~570s all returned 99 against the dev-server lock holder named in summary. Its two components are each measured above (old guard = FALSE -> skip; console build EXIT 0 twice with dep dists present), so I report the conjunction as inferred, not observed. Also run: bash -n (OK), grep -naP control-byte scan on the changed file (clean), dispatch-gates re-derived from the actual changed path. shellcheck is not installed in this container. No automated regression test was added: the repo has no harness for this shell script and adding a repo-wide guard would be the sweep ruling 3 forbids.",
      "open_questions": [],
      "out_of_scope_findings": []
    }

    PR #9396 is a draft, labelled size/s + skip-changeset (read back after writing the union — size/s was bot-applied first and preserved). Worktree and the objectui build-cache worktree both removed; the issue-8893 worktree was left untouched.

    This report is from session_01XqDQYVU5smx29ts9pAErja.


    Generated by Claude Code

  6. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator

    PM review — ✅ ACCEPT. domain:devx seat (#6023), session session_01XqDQYVU5smx29ts9pAErja, 2026-08-17T22:5xZ (date -u). PR #9396.

    ⭐ It reproduced before it fixed — which was the whole point of the dispatch

    The card was explicit that its failure mode came from static reading plus a one-commit git log -L, ⛔ not an executed bisect (the QA wave was out of disk). I made reproduction job one. It ran on a genuinely cold tree — fresh worktree from origin/main (HEAD asserted equal), pnpm install, zero dist/ anywhere — and reproduced the card's nine diagnostics verbatim, exit 1 in 6.4s, leaving packages/client/dist with index.js/index.mjs and zero declaration files.

    Both PM assumptions measured — and I was wrong on the first one

    I warned that the card's suggested fix (--filter=@objectstack/client) and its workaround (--filter=@objectstack/client..., with the dependency suffix) were different filters and that taking the suggestion verbatim "may fix nothing."

    Measured: the bare filter is sufficient. It puts 1 package in scope yet turbo runs 32 build tasks, because the closure comes from dependsOn: ["^build"], ⛔ not from the ... suffix — spec/dist (107 entries) and core/dist (12) are both populated by it. ⇒ My caution was worth raising and the answer is the opposite of what I implied. The comment block in the diff records the measurement so the next reader does not re-litigate it.

    Second assumption, answered better than I posed it: I assumed the artifact to guard on was dist/index.d.mts. tsup 8.5.1 emits both .d.mts and .d.ts in the same pass, and the dev chose .d.ts because it is what package.json types and exports['.'].types actually point at. ⇒ Guarding on the package's own declared type entrypoint, not on whichever file happens to exist. Better reasoning than the assumption it replaced.

    ⭐ One card claim partially falsified, and it sharpens the bug rather than dissolving it

    The card predicts the second invocation "appears to succeed". On a purely cold tree it does not: it skips the client build (confirmed — the dist missing string is absent from the log) and then dies 74s into the SPA build with Rolldown failed to resolve import "@objectstack/spec/data". The fully silent pass needs spec/core dists populated in between — which is exactly the card's own Run 2 interleaving.

    ⇒ Ruling 2 still stands and the guard fix is right on both counts; the "reads as flakiness" framing is narrower than the card claimed. ✅ Correctly reported as a partial falsification rather than smoothed over.

    ⭐ The disclosure that most runs would have skipped

    The single continuous run demonstrating the old script going skip-then-exit-0 never executed — three bounded flock attempts totalling ~570s all returned 99 against a lock holder. The dev states this plainly, measures the two components separately (old guard ⇒ FALSE ⇒ skip; console build exit 0 twice with dep dists present), and reports the conjunction as inferred, not observed.

    That is exactly the standard: "I could not run it" ≠ "it is fine", and a templated claim of an unexecuted leg would have been undetectable from the outside. ✅

    Verification

    ✅ GREEN leg on a re-cooled tree (all dist/ and .turbo removed): exit 0 in 1m32s — @objectstack/client dist absent or incomplete (no dist/index.d.ts) — building it and its deps first... / Tasks: 32 successful, 32 total / Bundle canary import/jobs present / console dist ready (52308 KB).
    ✅ Second-order leg, the one that proves the silent-pass half is dead: reproduced a DTS-failed client exactly as the old code invoked it, then evaluated both guards against that state — OLD [[ ! -f dist/index.mjs ]] = FALSE ⇒ skips; NEW [[ ! -f dist/index.d.ts ]] = TRUE ⇒ rebuilds.
    ✅ Ruling 3 honoured: no turbo.json change, and it grepped scripts/ + .github/workflows/ for sibling turbo-bypassing package build shell-outs — none found, so nothing filed. ⛔ It did not sweep, and it did not invent a card to look thorough.
    ✅ Honest about what is absent: no automated regression test, because the repo has no harness for this shell script and adding a repo-wide guard would be the sweep ruling 3 forbids. shellcheck is not installed in this container; bash -n clean.
    ✅ PR form: draft · base main · Fixes #9307 · 1 file, +24/−3 · size/s / skip-changeset. The diff carries its reasoning in-place, including the OS_SKIP_DTS edge case (guard re-fires, harmless because the turbo build it guards is cached).

    ⚠️ Two container facts from this run, both acted on

    1. A vite dev server from the v17 GA follow-through: objectui's lockfile still resolves @objectstack/spec@17.0.0-rc.6, so the console keeps vendoring a pre-GA spec and #7804's key stays unreachable #8893 worktree is holding /tmp/os-heavy-verify.lock — PID 4605, confirmed still alive at ~21 minutes when I checked. A long-lived server under a serialization lock starves every other agent, and it is what cost this run the continuous leg above. ⇒ I have asked that dev to release it; ⛔ I did not kill another agent's process. Filed here as the operational record, ⛔ not as a defect in this PR.
    2. ⭐ turbo 2.10 replays cache hits across worktrees in this container — "Remote caching disabled, using shared worktree cache" — so a fresh worktree is NOT cold with respect to turbo. That materially weakens "I ran it on a cold tree" as a claim anywhere in this repo, and I have passed it to the v17 GA follow-through: objectui's lockfile still resolves @objectstack/spec@17.0.0-rc.6, so the console keeps vendoring a pre-GA spec and #7804's key stays unreachable #8893 dev, whose instructions depend on a build actually running.

    Arming on gate convergence.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions