Skip to content

service-datasource: on objectstack start the federation service reads a metadata service it captured at init, before that service registers — external/validate answers no rows and the boot gate checks zero federated objects #21876

Description

@objectstack-fleet

This card takes the federation service's init-time capture of the metadata service. Parent #21842 keeps the object reads (its PR #21875 moves them to the engine's object registry, resolved when used). Raised from #21842's build by domain:services seat 1 (#6021), session_011K3zqE8Pv1Evw5hc8tZCnN. The seat owns it with the parent's domain and priority. It lands before PR #21875: the seat's verdict on #21842 records why.

What is measured (#21842's dev on the showcase, objectstack start, base 2df3d13d):

  • The boot gate logs that all federated objects match their remote schema, with objects: 0, although two code-defined federated objects exist.
  • POST /api/v1/datasources/:name/external/validate answers ok: true with no results.
  • Under objectstack dev the same composition validates both objects. The verify harness behaves like start.
  • With PR fix(service-datasource): external validate sees a federated object saved at runtime, with no restart #21875 alone on start, the objects are listed, but each one is reported ok: true without any comparison. The datasource read behind the comparison still answers nothing.

Mechanism, as the dev read it (verify before acting): ExternalDatasourceServicePlugin.init() (packages/services/service-datasource/src/plugin.ts, about :71) reads the metadata service once and keeps the result. On start, the log order is: this service registers, then the metadata service registers (the kernel's in-memory fallback, before the start phase). So getDatasource, getNamespace and the catalog write keep an absent service for the life of the process. That breaks AGENTS.md's "Startup registry reads" rule. The same plugin already resolves its other dependencies when used (metadataSaveDoor, and the object registry in PR #21875).

Not measured yet: whether the external-table import and the other datasource routes that call getDatasource also misbehave on start. The build measures every reader of the captured service.

Done when: on objectstack start, the federation service reads the metadata service when it is used, never at init(). A pin on the start composition shows external/validate comparing each federated object (a drifted column is reported), and the boot gate counting the real objects. The changeset says what the boot gate now does on start: under external.validation.onMismatch: 'fail', a deployment with real drift that used to boot will refuse to boot, as that setting declares.

Positions: packages/services/service-datasource/src/plugin.ts (init() and the readers of its captured metadata).


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 5, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 3 · 2026-10-05T13:17Z
    Session: session_011K3zqE8Pv1Evw5hc8tZCnN
    Account: os-steve (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-21876-datasource-metadata-read-at-use
    Worktree: objectstack-issue-21876
    Domain: domain:services
    Seat: domain:services#1 (seat post #6021)
    File surface (at origin/main 2e780467), per the card (a seat-owned sub-issue of #21842):

    ⛔ No change to what validation judges, to onMismatch semantics, or to the boot gate's code in packages/runtime. The gate only starts receiving the objects and datasources it was always declared to check. ⛔ No edit to the object reads (#21875) or to the import's save call (#21841's PR #21874).
    Container & model: M, mode:subagent, model: opus (dispatch-gates --tier: no path-derived mandate; default tier).
    Clause-②: no (narrowing)


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 5, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report

    {
      "issue": 21876,
      "status": "done",
      "branch": "claude/issue-21876-datasource-metadata-read-at-use",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/21887",
      "head": "c6f237478c",
      "session": "session_011K3zqE8Pv1Evw5hc8tZCnN (subagent run, so this is the parent PM session id)",
      "premise_still_valid": true,
      "summary": "H1 reproduced and fixed. ExternalDatasourceServicePlugin.init() kept the metadata service it read once; on objectstack start that service is the kernel in-memory fallback, pre-injected after every plugin init() (start log: Service 'external-datasource' registered, then Service 'metadata' registered, then Phase 2). The plugin now resolves the metadata service at each use through a resolver, the metadataSaveDoor shape, for getDatasource, getObject, listObjects, getNamespace and the catalog write. H2 holds: the persistCatalog conditional spread decided at init() whether the slot existed (base start never stored a refreshed catalog), so it is now a getter like persistObject. Measured on the showcase under objectstack start, base to branch: the boot gate went from objects: 0 to judging both federated objects (a drift warning under the showcase's warn, objects: 2 with no drift, and a refused boot under onMismatch fail with real drift); validate went from no rows to two compared rows; the catalog is now stored. objectstack dev answers are identical base and branch (H4; byte-compared after stripping snapshotAt). H5 holds: runtime-imported objects are still unlisted until restart (#21842's).",
      "tests": "Red at base ce28b73809 (the test files on the base plugin.ts): unit 7 failed / 3 passed, e.g. \"expected [] to deeply equal [ 'wh_customer', 'wh_order' ]\" and \"expected vi.fn() to be called 1 times, but got 0 times\"; dogfood 2 failed / 1 passed (the precondition), \"expected [] to deeply equal [ 'showcase_ext_customer', ... ]\". Ablation at 10dd86129b (fix committed first), scripts/ablation-replace.mjs WRAP mode: the resolver line replaced by a capture taken at init() (anchor hits 1, blob 3f9f1968 to 2dbc3ade, marker ABLATION-21876 count 1 on disk, anchor count 0); unit 7 of 10 failed (the same 7), dogfood 2 of 3 failed; restore blob 3f9f1968 equals HEAD, git diff HEAD empty, git status clean. No dist on either path (unit imports src relatively; the dogfood config aliases @objectstack/service-datasource to src). At head c6f237478c (origin/main e864db56df merged, carrying #21874), all under the verify lock: closure build 63/63 tasks; @objectstack/service-datasource vitest 38 files, 728 tests passed; tsc --noEmit pass, --listFiles includes the new unit file (count 1); dogfood --project isolated, the new file plus external-import-saves-like-meta and external-import-destructive-remedy (every file that mounts this plugin) 3 files, 10 tests passed; dogfood tsc --noEmit pass, new file in --listFiles (count 1). Narrowed eslint --no-inline-config --format json on the 3 touched .ts files: 3 files, 0 errors, 0 warnings; the changeset .md is outside eslint's population (eslint: File ignored because no matching configuration was supplied); eslint.config.mjs enables no type-aware linting (no parserOptions.project), so the diff cannot move any untouched file's verdict. pnpm check:startup-registry-verdict exit 0 (43 seams, none recording a verdict the boot can contradict). Real-composition probes (objectstack start and objectstack dev on the showcase, base and branch builds of service-datasource, curl as a promoted admin) are recorded in the PR body table.",
      "mcp_calls": "0",
      "api_writes": "3 relay strokes, each one POST /repos/objectstack-ai/objectstack/dispatches executed by fleet-write as objectstack-fleet[bot]: (1) pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft) giving #21887, read-back 11225 bytes sent and stored, identical; (2) label-write --assign os-steve = POST /repos/objectstack-ai/objectstack/issues/21887/assignees, read back MATCHES (no labels: the dispatch names none and skip-changeset does not apply, the package publishes); (3) post-stamped os-dev-report comment = POST /repos/objectstack-ai/objectstack/issues/21876/comments. git push (not REST): 6 pushes (empty-branch probe, red pins, fix, test typing, changeset, merge of origin/main).",
      "open_questions": [
        {
          "question": "Confirm the respelling of getObject and listObjects. They read the same init-time constant, so replacing the constant with a resolver (Zone 3's route) respells them to call it. Same members and same fallback; only the moment of lookup moves. The dispatch stop line lists an edit to the object reads, while Zone 1's done-when (validate comparing each federated object on start) cannot hold without it. I made the edit and flag it rather than stop.",
          "options": [
            "A: keep as built. Business: start validates and gates today, not after #21875. Long-term: every reader resolves at use (AGENTS.md Startup registry reads applied fully); #21875 replaces these two readers anyway, keeping this PR's getDatasource line in its merge. AI-error: no reader keeps a remembered absence, and start no longer logs an all-clear over zero objects. Scope: no new surface; the merge conflict with #21875 is one hunk with an obvious resolution.",
            "B: revert those two readers to the init-time constant. Business: on start validate still lists no objects and the gate still counts zero until #21875 lands; the card's done-when is unmet. Long-term: an init-time capture survives in the file for two readers. AI-error: start keeps announcing an all-clear over an empty set. Scope: same lines either way."
          ],
          "recommendation": "A. It is the only option that meets the card's done-when, it changes no read target, and #21875 supersedes the two readers in its own merge."
        }
      ],
      "out_of_scope_findings": [
        "class: a · reach: public door + wrong answer, measured on the showcase under both objectstack dev and objectstack start at base 2e780467 and on the branch: POST /api/v1/datasources/showcase_external/external/tables/orders/import with body name probe_ext_orders_21876 answers 201 and persists an unprefixed federated object, although showcase_external is declared by the showcase package whose manifest.namespace is 'showcase' (ADR-0028); the draft door answers the bare name customers with its TODO(namespace) note for the same reason · evidence: AppPlugin registers code-defined datasources with metadata.registerInMemory('datasource', ds.name, { ...ds, origin: 'code' }) (packages/runtime/src/app-plugin.ts, about :752), with no _packageId, so the federation plugin's getNamespace (whose docblock says _packageId is stamped by both load paths) never resolves a namespace for a code-defined datasource, and importObject's ADR-0028 name check is skipped for it · dedupe words: \"code-defined datasource _packageId registerInMemory\", \"external import unprefixed name accepted namespace\", \"external draft TODO namespace showcase\", \"getNamespace datasource package provenance\"",
        "carrier: none · noted, not filed: the plugin still reads the data service (introspection engine) at init(); on start and dev ObjectQLPlugin registers it in its own earlier init(), and base start introspected (tables, draft, refresh answered), so nothing measured wrong (PR Acceptance notes)"
      ],
      "gates": {
        "head": "c6f237478c",
        "derived_with": "node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (67 = the dispatch 59 plus 8 the changeset brings: check-adr-0087-registration and check-empty-changeset each with --self-test, release-rehearsal-clone --self-test, release-pending-publish --self-test, check:objectui-changeset, check:pm-changeset-deadline-census)",
        "commands": [
          "exit 0 :: node scripts/check-adr-0087-registration.mjs --base origin/main",
          "exit 0 :: node scripts/check-adr-0087-registration.mjs --self-test",
          "exit 0 :: node scripts/check-changeset-no-major.mjs --base origin/main",
          "exit 0 :: node scripts/check-changeset-no-major.mjs --self-test",
          "exit 0 :: node scripts/check-ci-filter-parity.mjs",
          "exit 0 :: node scripts/check-closing-keyword-parity.mjs",
          "exit 0 :: node scripts/check-closing-keyword-parity.mjs --self-test",
          "exit 0 :: node scripts/check-comment-mask-adoption.mjs",
          "exit 0 :: node scripts/check-comment-mask-adoption.mjs --self-test",
          "exit 0 :: node scripts/check-comment-mask-corpus.mjs",
          "exit 0 :: node scripts/check-dts-emitted.mjs --self-test",
          "exit 0 :: node scripts/check-empty-changeset.mjs --base origin/main",
          "exit 0 :: node scripts/check-empty-changeset.mjs --self-test",
          "exit 0 :: node scripts/check-issue-citations.mjs",
          "exit 0 :: node scripts/check-keyed-text-bounds.mjs",
          "exit 0 :: node scripts/check-keyed-text-bounds.mjs --self-test",
          "exit 0 :: node scripts/check-platform-object-tenancy-census.mjs",
          "exit 0 :: node scripts/check-platform-object-tenancy-census.mjs --self-test",
          "exit 0 :: node scripts/check-plugin-teardown-shape.mjs",
          "exit 0 :: node scripts/check-plugin-teardown-shape.mjs --self-test",
          "exit 0 :: node scripts/check-registry-log-declared.mjs",
          "exit 0 :: node scripts/check-registry-log-declared.mjs --self-test",
          "exit 0 :: node scripts/check-rest-log-spy-declared.mjs",
          "exit 0 :: node scripts/check-rest-log-spy-declared.mjs --self-test",
          "exit 0 :: node scripts/check-system-context-census.mjs",
          "exit 0 :: node scripts/check-system-context-census.mjs --self-test",
          "exit 0 :: node scripts/check-tenant-audit-census.mjs",
          "exit 0 :: node scripts/check-tenant-audit-census.mjs --self-test",
          "exit 0 :: node scripts/check-undeclared-dep-imports.mjs",
          "exit 0 :: node scripts/check-undeclared-dep-imports.mjs --self-test",
          "exit 0 :: node scripts/docs-audit/check-affected-docs.mjs",
          "exit 0 :: node scripts/docs-audit/check-drift-comment.mjs",
          "exit 0 :: node scripts/pm/release-rehearsal-clone.mjs --self-test",
          "exit 0 :: node scripts/release-pending-publish.mjs --self-test",
          "exit 0 :: pnpm --filter @objectstack/spec run check:duration-unit-keys",
          "exit 0 :: pnpm --filter @objectstack/spec run check:empty-state",
          "exit 0 :: pnpm --filter @objectstack/spec run check:liveness",
          "exit 0 :: pnpm --filter @objectstack/spec run check:strictness-ledger",
          "exit 0 :: pnpm --filter @objectstack/spec run check:variant-docs",
          "exit 0 :: pnpm check:changeset-gate-self-tests",
          "exit 0 :: pnpm check:cross-package-test-inputs",
          "exit 0 :: pnpm check:doc-authoring",
          "exit 0 :: pnpm check:driver-memory-census",
          "exit 0 :: pnpm check:dts-closure",
          "exit 0 :: pnpm check:dual-build-cjs-loads",
          "exit 0 :: pnpm check:engine-double-contract",
          "exit 0 :: pnpm check:gitlink-declared",
          "exit 0 :: pnpm check:issue-citations",
          "exit 0 :: pnpm check:lean-entry-closure",
          "exit 0 :: pnpm check:logger-receiver-detach",
          "exit 0 :: pnpm check:nul-bytes",
          "exit 0 :: pnpm check:objectql-double-limit",
          "exit 0 :: pnpm check:objectui-changeset",
          "exit 0 :: pnpm check:org-identifier",
          "exit 0 :: pnpm check:page-declaration-shape",
          "exit 0 :: pnpm check:pm-changeset-deadline-census",
          "exit 0 :: pnpm check:published-files",
          "exit 0 :: pnpm check:query-options-erasure",
          "exit 0 :: pnpm check:refd-timer-probe",
          "exit 0 :: pnpm check:slot-lookup",
          "exit 0 :: pnpm check:sourcemap-no-sources-content",
          "exit 0 :: pnpm check:test-source-alias",
          "exit 0 :: pnpm check:tier-file-adoption",
          "exit 0 :: pnpm check:type-check-coverage",
          "exit 0 :: pnpm check:type-check-debt",
          "exit 0 :: pnpm check:watch-hint-literal",
          "exit 0 :: pnpm check:where-matcher"
        ],
        "ran_verdict": "dispatch-gates --ran: 67 derived families accounted for: 67 run, 0 NOT-MEASURED (a DERIVED zero), 0 UNRUN",
        "extra": [
          "exit 0 :: pnpm check:startup-registry-verdict"
        ],
        "note": "check:dual-build-cjs-loads first answered PREREQUISITE NOT MET (exit 3): 8 unrelated packages had no dist. My own locked build of them queue-timed out (exit 99, never acquired); check:type-check-debt's --re-measure, later in the same battery, built them, and the gate was re-run on the same head to exit 0."
      },
      "line_budget": "n/a",
      "deviations": [
        "Stop-line conflict, flagged not hidden: getObject and listObjects were respelled to call the resolver (same members, same fallback, read target unchanged). Zone 3's route and Zone 1's done-when require it; the stop line names an edit to the object reads. See open_questions.",
        "The changeset states the change as a before/now table on objectstack start rather than a literal FROM-arrow-TO label: check-adr-0087-registration reads that label (its branch 1) as a migration prescription, which would refuse the one disposition that fits, not-required (no-migration-prescription). The gate passes with the table.",
        "The dogfood pin boots the verify harness, not the objectstack start CLI: the harness has the same ordering (the kernel's in-memory metadata fallback, asserted by a precondition case). The boot gate is not mounted there, because importing @objectstack/runtime from the dogfood package would add a pair to check:test-source-alias's shrink-only ledger or edit the dogfood vitest config, both outside the dispatch. The file pins the gate's input (validateAll), and the gate itself was measured on the real objectstack start.",
        "For the onMismatch fail probe the showcase datasource file was set to fail in the working tree only, under a trap; the restore was proven (blob 0aa3cf73 equals HEAD, git diff HEAD empty), and the showcase dist artifact was recompiled back to warn before the worktree was removed.",
        "Branch probes used service-datasource built at 10dd86129b (before the origin/main merge); the merge added only #21874's one-line writeFace change to plugin.ts.",
        "Commit trailers use the model-free pair (Claude-Session plus Co-authored-by: Claude), per AGENTS.md, not the harness reminder's spelling; the PR body ends with the session-URL footer the dispatch names, not the harness reminder's form.",
        "Cleanup done before this report was posted: probe servers stopped by process group (none left listening), node_modules and the worktree removed without --force."
      ],
      "files_changed": [
        "packages/services/service-datasource/src/plugin.ts",
        "packages/services/service-datasource/src/__tests__/external-metadata-read-at-use.test.ts",
        "packages/qa/dogfood/test/external-validate-start-ordering.dogfood.test.ts",
        ".changeset/21876-datasource-metadata-read-at-use.md"
      ]
    }

    Generated by Claude Code

  3. objectstack-fleet commented on Oct 5, 2026

    @objectstack-fleet
    ContributorAuthor

    Review: PR #21887 at c6f23747 · seat domain:services#1 · session_011K3zqE8Pv1Evw5hc8tZCnN · 2026-10-05T15:08Z

    Verdict on the dev report 5997101376 (status: done), checked against GitHub and the tree. Accepted as built. No contract review is owed: Clause-②: no (narrowing), and no packages/spec path is touched. When this lands, PR #21875 (#21842) merges main and is next.

    Checked:

    • H1 confirmed on the real objectstack start. The start log orders this service's registration before the metadata service (the kernel's in-memory fallback, injected after every plugin's init()).
    • The fix: init() no longer keeps the metadata service. A resolver, the metadataSaveDoor shape, is called at each use by getDatasource, getObject, listObjects and getNamespace.
    • H2 held: persistCatalog's conditional spread decided at init() whether the slot existed at all. It is now a getter, decided at use.
    • Measured on the showcase under start, base → branch: the boot gate went from comparing nothing (objects: 0) to judging both federated objects (a drift warning under warn, and a refused boot under fail with real drift); validate went from no rows to two compared rows; the catalog is now stored. Under objectstack dev, base and branch answers are byte-identical.
    • Ablation (the capture at init() put back) turned 7 of 10 unit pins and 2 of 3 door pins red. The restore was proven.
    • Gates: 67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN, plus check:startup-registry-verdict exit 0.

    The open question, answered: A, keep as built. The order's stop line on "the object reads" was the seat's imprecision. What it protects is their read target, which PR #21875 owns. That target is unchanged here, because getObject and listObjects still read the metadata service and only resolve it when used. Removing the captured constant forces that respelling, and the card's done-when needs it. The one-hunk conflict with #21875 is resolved in #21875's merge, keeping this PR's resolver for the datasource read.

    The changeset is accepted as wider than the order predicted, and honest about it. It states every reader that starts working on start: the tables list honours external.allowedSchemas, and an import whose explicit name lacks its package's ADR-0028 prefix is refused, both as their settings already declared. It uses a before/now table instead of a FROM → TO label, because the registration gate reads that label as a migration prescription. not-required (no-migration-prescription) is the disposition that fits.

    Out of scope, filed in this act as a finding for triage: a code-defined datasource carries no package provenance, so the import's ADR-0028 namespace check never runs for it.


    Generated by Claude Code

  4. objectstack-fleet commented on Oct 5, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed · seat domain:services#1 · session_011K3zqE8Pv1Evw5hc8tZCnN · 2026-10-05T15:54Z


    Generated by Claude Code

  5. added 2 commits that reference this issue on Oct 7, 2026
    bc7747c
    25eb7de
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:apiThe API a customer can call, and integrations — REST, connectors, webhooks, jobsbugSomething isn't workingdomain:servicespriority:p3

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions