Skip to content

runtime: TS-config boot registers a 'metadata' service without attachClusterPubSub — cross-node invalidation disabled; new object gives OBJECT_NOT_FOUND on non-writing replicas, never heals #13331

Description

@baozhoutao

Symptom

On a 3-replica objectos-ee cluster (framework pin 502ff8b5, redis cluster driver active — see #13330 for getting it active at all), every replica logs at boot:

WARN MetadataClusterBridgePlugin: metadata service does not expose attachClusterPubSub(); cross-node cache invalidation disabled

Measured consequence — create a new object through the load balancer, then immediately write data to it through the LB:

PUT /api/v1/meta/object/gadget            → Saved (seq=3)
12 × POST /api/v1/data/gadget via LB      → 3 ok, 9 OBJECT_NOT_FOUND
60s later, 9 more POSTs                   → 3 ok, 6 OBJECT_NOT_FOUND   (no self-heal)

Only the replica that handled the metadata write registers the object with its ObjectQL engine; the other two keep answering OBJECT_NOT_FOUND indefinitely (no TTL/poll fallback on this path). Restarting the replicas is the only recovery. GET /api/v1/meta/* list/read stays fresh everywhere (store-backed reads), which makes the drift worse to spot: the metadata API says the object exists while the data API on two of three nodes refuses it.

Related earlier fix #5109 addressed peer-side listCache/registry invalidation once the broadcast arrives — here the broadcast never starts, because the bridge never attaches.

Root cause direction

MetadataClusterBridgePlugin (service-cluster) resolves ctx.getService('metadata') at kernel:ready and requires attachClusterPubSub on it (metadata-cluster-bridge-plugin.ts:53-59). NodeMetadataManager extends MetadataManager does carry that method (metadata-manager.ts:2969), and @objectstack/metadata's MetadataPlugin registers exactly that manager (plugin.ts:303).

But on this boot (objectos-ee, objectstack.config.ts via os start→serve), the kernel's plugin lineup contains no MetadataPlugin at all (boot banner lists ObjectQL, PlatformObjects, Dispatcher, … — no MetadataPlugin, and plugin.ts:304's registration log line never prints). Whatever registers the 'metadata' service on the TS-config path (runtime dispatcher/domain facade) does not expose attachClusterPubSub, so the bridge warns and bails — exactly the "protocol omission → stale caches" case content/docs/kernel/cluster.mdx §1 predicts. serve.ts:2296+ composes an artifact-sourced MetadataPlugin only on artifact boots.

Second-order effect, same neighborhood: a record_change flow saved at runtime (PUT /api/v1/meta/flow/…) is registered ([Registry] Registered flow: …) but its trigger only arms after a replica restart, and then only on that replica's local hooks — runtime-authored automation is effectively single-node too.

Repro

  1. Boot cloud/apps/objectos-ee/docker-compose.yml at 3 replicas with the redis driver active (needs the cli: serve's cluster-driver load registers into the CJS registry while the ESM Runtime reads the ESM one — OS_CLUSTER_DRIVER=redis silently downgrades to "not registered" (post-#10645) #13330 workaround).
  2. PUT /api/v1/meta/object/<new> through the LB.
  3. Loop POST /api/v1/data/<new> through the LB — two thirds fail OBJECT_NOT_FOUND, and keep failing.

Expected

Whatever object the TS-config boot registers as the 'metadata' service either exposes attachClusterPubSub() (delegating to the real manager underneath), or the bridge knows how to reach the manager behind the facade — so metadata.changed fans out and #5109's peer-side invalidation actually runs. Until then, the EE multi-node compose (ADR-0018) is safe only for deployments whose metadata is fixed at deploy time and whose replicas restart together after any schema change.

Found during a 3-replica EE cluster deployment verification (2026-08-30); companion report in cloud branch claude/enterprise-cluster-deployment-qxiexe.

Activity

  1. baozhoutao commented on Aug 31, 2026

    @baozhoutao
    ContributorAuthor

    Quantified under load on the live 3-replica EE deployment (Traefik LB, redis cluster driver). A concurrency/stress harness driven through the LB (the real production path, not a docker-exec pin) makes the operational severity concrete:

    • An object authored at runtime (PUT /api/v1/meta/object/qa_stress) answers 200 on /api/v1/meta/object/qa_stress on all 3 replicas (the meta-registry entry broadcasts cluster-wide at create time) — but its /api/v1/data/qa_stress door is 200 on the writing replica only, 404 on the other two (the ObjectQL engine registration is replica-local). Directly measured across app-2/3/4: 200 / 404 / 404.
    • Consequently, load through the round-robin LB against a runtime-created object is a ~2/3 hard 404 OBJECT_NOT_FOUND: 200 concurrent creates → 67 201 / 133 404; authed list reads → ~1/3 200. A fleet-wide object (note, boot-loaded on every replica after a restart) under the identical harness is 0 errors, 200/200 creates persisted with 200 distinct rows (no lost writes / dupes), confirming the failure is purely the per-replica registration gap this issue tracks.

    So the practical blast radius on a multi-replica deployment is: every object/flow authored at runtime is unusable on (N-1)/N of the fleet until a full restart re-registers it from the DB on boot. A restart is the only recovery (boot re-loads all metadata on every replica); there is no runtime propagation of the engine registration.

    QA-source: #13404 · records-forms/automation (cluster stress) · #13331 attribution


    Generated by Claude Code

  2. os-warren commented on Aug 31, 2026

    @os-warren
    Collaborator

    分诊 → domain:cli · p0 · bug。

    锚定。 三个候选落点:TS-config 启动路径的插件组装(packages/runtime / packages/cli)、桥自己穿透 facade(packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts ⇒ domain:services)、packages/metadata/src/plugin.ts(⇒ domain:engine)。根因问题在组装侧(见下),所以主车道 domain:cli;跨域半边由该车道 PM 在认领评论里申报完整文件面。

    定级 p0 —— 依据是 2026-08-31 那条压测评论,不是原文

    新读数把它从「一个警告」变成「一条产品线不可用」:

    • 通过 LB 的 200 并发创建 → 67 × 201 / 133 × 404;authed list 读 ~1/3 成功。
    • 跨三个副本直接测同一个运行时创建的对象:/api/v1/data/qa_stress = 200 / 404 / 404,而 /api/v1/meta/object/qa_stress 在三个副本上都是 200。
    • 对照组成立:boot 时加载的 fleet-wide 对象(note)在同一套压测下 0 错误,200/200 落库、200 条不同记录。所以失败确实是本卡追的 per-replica 注册缺口,不是压测本身。

    判 p0 的四条理由,逐条:① 这是已发货的 EE 多节点路径(ADR-0018 的 objectos-ee/docker-compose.yml)。② 失败是静默且错误方向的 —— metadata API 说对象存在,data API 在 (N-1)/N 的副本上 404,运维看到的是「有时好有时坏」。③ 不自愈:60 秒后重测仍失败,无 TTL、无轮询兜底。④ 没有真绕法 —— 唯一恢复是全队重启,而这恰好取消掉「运行时创作对象/流程」这件功能本身。#13330 有一行 config 绕法所以是 p1;本卡没有,所以是 p0。

    ⛔ 派单前必须先答的问题 —— 卡片把它当成前提,我认为它才是根因

    卡片说「TS-config 路径上注册 'metadata' 服务的那个东西(runtime dispatcher/domain facade)不暴露 attachClusterPubSub」,并把修复设想成给 facade 补一个委托方法。在 origin/main 上查了一遍,这个前提站不住:

    ctx.registerService('metadata', …)   —— 全树非测试源只有一处:
      packages/metadata/src/plugin.ts:353
    

    ⚠️ 这是一次窄查询(字面量 key),对照有命中所以查询本身是好的,但它抓不到变量 key 的循环注册 —— 所以这是部分读数,不是零读数,不能当作结论。但它足以把问题掉个头:

    MetadataPlugin 是全树唯一的 'metadata' 服务注册方,而卡片实测它根本不在 TS-config 启动的插件阵容里(启动横幅无它,plugin.ts:304 的注册日志从不打印)。那么真正要答的不是「facade 为什么不暴露 attachClusterPubSub」,而是 —— TS-config 路径本来是不是就该组装 MetadataPlugin?它的缺席本身是不是缺陷?

    如果是,修复是组装侧一行,而且顺带修好所有其他依赖 MetadataPlugin 的能力(本卡第二段效应「运行时保存的 record_change flow 只在重启后、且只在本副本上武装」很可能是同一个缺席的另一个面)。如果不是 —— 即 TS-config 路径有意跑另一套 metadata 实现 —— 那才轮到给 facade 补委托方法,而且要问清楚那套实现还少了什么别的。

    ⛔ 不要先写 facade 委托方法。 先跑这个读数:把 serve.ts:2296+ 只在 artifact 启动时组装 MetadataPlugin 的那段读掉,答出 TS-config 启动路径上 getService('metadata') 返回的到底是什么、由谁注册。答案决定修哪个文件,也决定这是 domain:cli 一条道还是要拉 domain:engine 半边。

    排序说明(不是阻塞)

    本卡的集群内验证要先有 #13330 的绕法(或修复)才能观察 —— 卡片自己就是这么复现的。但修复本身不被阻塞:落点不同,可以实现并单测。所以不打 pm:blocked;派单时把两张卡交给同一个 dev 连着做最省,验证顺序 #13330 → 本卡。

    QA 来源:#13404(records-forms/automation 集群压测)已把归因指到本卡。


    Generated by Claude Code

  3. self-assigned this
    on Aug 31, 2026
  4. os-steve commented on Aug 31, 2026

    @os-steve
    Collaborator

    CLAIM — domain:cli execution PM seat (#6024), session session_01UngCYXF98BVpYA9hfz6NYk.

    Branch: claude/issue-13331-metadata-service-cluster-pubsub

    Comments re-read before claiming (3-part protocol): the reporter's stress-test quantification (07:19:25Z) and the triage seat's grading (13:10:05Z). No earlier claim from another session — genuinely unclaimed.

    Declared file surface

    Per the triage seat's request that the lane PM declare the full surface including cross-domain halves. ⚠️ This surface is provisional by design — the triage seat's ⛔ requires a reading before a fix site is chosen, so the dev may legitimately land in a different column than predicted:

    path domain expected
    assembly side of the TS-config boot (packages/runtime/**) cli most likely fix site, if MetadataPlugin's absence is itself the defect
    packages/metadata/src/plugin.ts (registerService('metadata', …), ~:353) engine cross-domain half — read first, write only if the facade genuinely runs a second implementation
    packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts (~:53–59) services cross-domain half — only if the bridge must reach through the facade
    packages/cli/src/commands/serve.ts cli ⛔ EXCLUDED — see below

    ⛔ serve.ts is excluded from this card

    Ruling ① — same file is a hard serial with no region exemption. serve.ts is held by #13330, dispatched concurrently. This card's root-cause reading does point at serve.ts:2296+ (the artifact-only MetadataPlugin composition), and reading it is required. Reading is not blocked; writing is. If the reading concludes the repair belongs in serve.ts, the dev must ⛔ stop and report back rather than edit it — I will serialize this card behind #13330 and re-dispatch. Ruling ① is honoured here by assignment, not by hoping the two devs miss each other.

    The ⛔ that governs the whole dispatch

    The card frames "the facade does not expose attachClusterPubSub" as the premise and a delegate method as the fix. Triage found that premise does not hold up and inverted the question. Carried into the dispatch verbatim:

    ⛔ Do not write the facade delegate method first. First answer: on the TS-config boot path, what does getService('metadata') actually return, and who registered it? That answer decides which file is edited and whether this is single-lane or needs an engine half.

    ⚠️ And the discipline that goes with it: triage's ctx.registerService('metadata', …) sweep returned one non-test site, but it was a literal-key query that cannot see a variable-key or loop registration. Triage labelled it correctly — a partial reading, not a zero. It is a starting point, not a conclusion, and the dev must not treat "only one registrar exists" as established. If the re-measurement needs to stand as a zero anywhere, it needs a positive control that could have returned non-zero.

    Graded p0 on the 2026-08-31 load measurement, not on the original report: 200 concurrent creates through the LB → 67 × 201 / 133 × 404, with a fleet-wide object under the identical harness at 0 errors as the control. No workaround exists (#13330 has a one-line one, which is why that card is p1 and this one is p0); the only recovery is a full fleet restart, which cancels the feature itself.


    Generated by Claude Code

  5. os-steve commented on Aug 31, 2026

    @os-steve
    Collaborator

    ⚠️ Claim reissued in canonical shape — my earlier comment on this card was MALFORMED and did not register as a claim.

    It opened **CLAIM** — (bold, uppercase, em-dash separator). The patrol predicate requires a first line beginning with the literal Claim:; the maintainer's 2026-08-11 ruling makes a differently-written claim malformed, not a dialect, with the repair on the write side and the predicate never widened (check-half-states.mjs, h34ClaimShapedNonCanonicalSeparator). This card — the lane's only p0 — therefore read as unclaimed to the next PM's mutual-exclusion sweep while a dev was on it.

    Claim: PM loop round 1
    Session: session_01UngCYXF98BVpYA9hfz6NYk
    Branch: claude/issue-13331-metadata-service-cluster-pubsub
    Worktree: objectstack-13331
    Domain: domain:cli
    File surface: packages/runtime/src/** (expected primary) · packages/metadata/src/plugin.ts (domain:engine half) · packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts (domain:services half) · ⛔ packages/cli/src/commands/serve.ts READ-ONLY, excluded from writes (stop on breach; explain in the report)
    Container & model: L, mode:subagent, model: opus — tier derived this round by node scripts/pm/dispatch-gates.mjs --tier packages/runtime/src/index.ts packages/metadata/src/plugin.ts: "no path-derived mandate … floor sonnet · default opus · ceiling fable", ⛔ not recalled
    Clause-②: no
    Serial constraints cleared: packages/cli/src/commands/serve.ts is held by #13330 — this card reads it (required by triage) but ⛔ may not write it; the dev is instructed to stop and report if the repair lands there, and I will serialize behind #13330 and re-dispatch. ⚠️ packages/runtime/src/http-dispatcher.ts is carried by PR #13811, currently in the merge queue — a different file in the same package, so same-package exemption applies and no serial is owed, but the dev should rebase past it. Sibling in-flight: #13330, #13332.

    Clause-② reasoning: the repair makes cross-node metadata invalidation happen where it was designed to and does not, so it is conformance rather than widening — the same limb the at-tier review of PR #13811 applied ("a door not carrying a declared field was in breach; conformance is not widening"). ⚠️ Flips to yes if the fix adds attachClusterPubSub to a facade's published type surface rather than repairing assembly; the dev must measure the built .d.ts both ways before declaring at PR time, ⛔ not infer it from this line.


    Generated by Claude Code

  6. os-steve commented on Sep 1, 2026

    @os-steve
    Collaborator

    os-dev-report

    {
      "issue": 13331,
      "status": "needs_decision",
      "branch": "claude/issue-13331-metadata-service-cluster-pubsub",
      "pr": null,
      "premise_still_valid": false,
      "summary": "The reading falsifies BOTH the card's premise and the fix its 'no' branch prescribes, so no code was written. (1) WHO REGISTERS THE SLOT: measured on a real ObjectKernel booted in the host-config shape, getService('metadata') returns createMemoryMetadata() — the kernel's in-memory core fallback — registered by ObjectKernel.preInjectCoreFallbacks() at packages/core/src/kernel.ts:270 through a VARIABLE-KEY call this.registerService(serviceName, fallback), which is exactly the shape triage's literal-key query could not see and correctly refused to call a zero. Nothing in packages/runtime registers that slot; MetadataFacade, the obvious candidate, is not installed on main (its own docblock at metadata-facade.ts:134 says so — the contradicting class docblock is filed as #14019). (2) THE YES/NO: NO — MetadataPlugin's absence on the TS-config host-config boot is deliberate, documented three times: isHostConfig ('must NOT be wrapped again'), memory-metadata.ts's own header naming 'the host-config / standalone boot path — where this fallback (not MetadataPlugin) provides the metadata service', and serve.ts:2865 composing MetadataPlugin for host configs in dev ONLY with 'production / cloud host deployments are untouched'. The banner evidence agrees: standalone-stack.ts composes MetadataPlugin unconditionally, so a lineup without it cannot be that stack. (3) WHY THE 'NO' BRANCH ALSO FAILS — the decisive finding: adding attachClusterPubSub to the slot would attach a bridge that is NEVER ASKED. packages/metadata-protocol/src/protocol.ts owns the whole runtime authoring path (saveMetaItem then applyRegistryWriteThrough then applyObjectRegistryMutation then engine.registry.registerObject) and contains ZERO references to cluster, pubsub, or getService('metadata') (measured with two positive controls). metadata.changed's only production publisher is MetadataManager.notifyWatchers, which no PUT /api/v1/meta/* ever reaches on ANY boot shape — ObjectQL's onMetadataMutation listener only does local hook/action rebind and never writes back into the metadata service. So the write persists sys_metadata (shared DB — hence meta reads 200 fleet-wide) and mutates only the LOCAL engine.registry (hence data-plane 200 on the writer, 404 on peers), with no route to the bus. Both faces the card reports are one gap: createMemoryMetadata also has no subscribe, so ObjectQLPlugin (plugin.ts:493) never wires its metadata-to-engine receiver either. The repair is neither in serve.ts nor anywhere in my declared file surface (packages/runtime/**): it needs a publisher at the protocol's onMetadataMutation choke point plus a peer-side registry applier, spanning domain:engine and domain:services, and it mints new public cross-node contract. Escalating rather than widening silently, per the dispatch's own instruction. Independent corroboration: #13609 is open and needs-user-decision for the same seam from a different QA angle ('MetadataManager's unregister DOES broadcast — the observed prolongation sits in an unidentified seam'). This reading identifies that seam.",
      "tests": "No code changed, so no ablation was run — saying so rather than filling the field with a template. Evidence is measurement, not source-reading alone. (A) TWO-ARM KERNEL EXPERIMENT, direction declared in writing before running. Real ObjectKernel from packages/core/dist, bootstrap(), with a cluster service (driver=redis, nodeId=node-a) and the real MetadataClusterBridgePlugin from packages/services/service-cluster/dist. ARM A (host-config shape, no MetadataPlugin) predicted+observed: _serviceName=metadata, __serviceInfo.status=degraded, typeof attachClusterPubSub=undefined, typeof subscribe=undefined, typeof loadMany=undefined, typeof saveItem=undefined, shape keys = __serviceInfo,_serviceName,register,registerInMemory,get,list,unregister,exists,listNames,getObject,listObjects (byte-identical to the member list in packages/core/src/fallbacks/memory-metadata.ts); attachClusterPubSub calls = 0; bridge emitted the card's line VERBATIM: 'MetadataClusterBridgePlugin: metadata service does not expose attachClusterPubSub(); cross-node cache invalidation disabled'; no attach log. ARM B POSITIVE CONTROL (identical harness, one plugin registers a metadata service that DOES expose attachClusterPubSub): attach called exactly once with {nodeId: node-a, hasPubsub: true}, info log 'bridged metadata.changed to cluster.pubsub (node=node-a)', zero warnings. Verdict line printed: 'READING CONFIRMED (control discriminates)'. This is the distinguisher the bar asked for — Arm A's 'never attached' is a measurement, not a harness artifact, because Arm B attached under the same harness. Incidental confirmation that fallback pre-injection was live in the run: the kernel logged 'CORE: Core service missing, functionality may be degraded: job', the #10746 no-fallback-for-job behaviour. (B) THE ZERO, WITH CONTROLS. Claim: packages/metadata-protocol/src/protocol.ts has no cluster/pubsub/getService('metadata') reference. git grep -cE over that file: no match, exit 1. Control 1 (same instrument, same file, term known present): applyRegistryWriteThrough = 16. Control 2 (same pattern, different file): packages/metadata/src/metadata-manager.ts = 43. Both could have returned non-zero and did, so the zero is a reading. (C) PUBLISHER ENUMERATION: whole-tree sweep of metadata.changed and CLUSTER_CHANNEL shows the only production publish() is metadata-manager.ts:2853 inside notifyWatchers; every other hit is docs, tests, or comments. (D) REGISTRAR SWEEP RE-ESTABLISHED, not inherited: triage's literal-key query reproduced (one non-test site, packages/metadata/src/plugin.ts:353), then widened to registerService( with a NON-literal first argument — which is what surfaced kernel.ts:270 and :303, the registrations the literal query is blind to. (E) BUILDS, both under the shared verify lock: pnpm --filter '@objectstack/service-cluster^...' build (VERDICT command-exit 0, held 126s) and pnpm --filter '@objectstack/service-cluster' build (VERDICT command-exit 0, held 5s). Verdicts read from the lock's own VERDICT line, not a bare exit code. (F) GATE RECONCILIATION: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, run in the worktree. Provenance line confirmed the answer came from objectstack-ai/objectstack at 62a137baec and that --repo 'holds' against the checkout's origin. Output: 'this branch changes nothing against origin/main (merge base 62a137bae) — nothing to derive'. git status --porcelain empty. named 0, ran 0, unreconciled 0 — a real empty union, because the diff is empty, not because gates were skipped.",
      "mcp_calls": "4 — one search_issues (dedup, after repo-scoped REST search returned 403 while REST reads returned 200: channel switch declared), two issue_write (the findings below), one add_issue_comment (this report). All card and comment reads went over zero-quota REST.",
      "open_questions": [
        {
          "question": "How should a runtime-authored metadata item's ObjectQL engine-registry registration reach peer replicas? The existing metadata.changed channel cannot carry it: its only publisher is MetadataManager.notifyWatchers, and the authoring path (protocol.saveMetaItem) never calls the metadata service. This mints new public cross-node contract (a channel, a payload, and an attach entry point) and spans domain:engine plus domain:services, so it is a maintainer/PM call, not a dev one.",
          "options": [
            "A — Fan out the protocol's mutation choke point, mirroring the in-tree authz precedent exactly. The state owner exposes attach-pubsub and a service-cluster bridge late-binds it at kernel:ready, which is literally what AuthzClusterBridgePlugin does today with engine.attachAuthzInvalidationPubSub. Publisher: the protocol's existing onMetadataMutation, already documented as 'the ONE choke point every authoring surface funnels through'. Peer applier: re-run the registry write-through from the peer's OWN sys_metadata read, so peers converge from the source of truth rather than trusting a payload. Touches metadata-protocol/objectql (engine), service-cluster (services), spec (the payload contract).",
            "B — Self-heal at the consumer: on an assertObjectRegistered miss, fall back to a sys_metadata read before answering 404. No new channel, and it directly addresses the card's 'no TTL/poll fallback on this path'. But it loosens a deliberately fail-closed gate (ADR-0005 / #3770 / #6190) into a tolerant one.",
            "C — Route runtime authoring through the metadata service so MetadataManager's existing cluster machinery does the work unchanged, which additionally requires giving host-config boots a real manager. Largest blast radius: it reverses the deliberate host-config decision and brings artifact/FS loading semantics into cloud deployments."
          ],
          "recommendation": "A, and the four axes do not split on it. REAL BUSINESS NEED: measured, not speculative — 200 concurrent creates through the LB give 67 x 201 / 133 x 404 on a shipped EE path (ADR-0018), with a boot-loaded object at 0 errors as the control; runtime-authored objects ARE the product feature, and today they are unusable on (N-1)/N of any fleet. LONG-TERM SOUNDNESS: A fixes at the producer, so every future authoring transport inherits the signal instead of having to remember it — the tree's own stated rule at protocol.ts:4875, which is also why the publisher must NOT go in the runtime HTTP domain (and therefore why packages/runtime/**, my declared 'most likely fix site', is the wrong column). Reusing the authz bridge's shape means one pattern, not two. Peers converging from their own DB read rather than from the payload keeps the DB the single source of truth. AI-AUTHORED-METADATA SAFETY: A is the only option that keeps the failure loud — B is precisely the consumer-side tolerance that lets an AI-authored object half-exist and the platform paper over it, and it weakens a gate whose fail-closed posture is load-bearing for multi-org correctness, not just for tidiness. STARTUP SCOPE: A repairs a shipped p0 on an already-published path; it adds no speculative surface, and its new contract is one channel plus one attach method modelled on one that already ships. B is cheaper but treats the symptom and taxes a security-relevant gate forever; C is the largest change and collides with #10746's ruling that a fallback must not fake capability, since it would make a real dependency out of a slot the kernel sometimes fills with a degraded fake. If a stopgap is wanted for the current EE fleet while A is built, B is defensible as an explicitly temporary, loudly-logged complement to A — but not as the fix.",
          "also_needs_a_ruling": "Scope split. A spans domain:engine and domain:services and touches no domain:cli file, so this card as labelled (domain:cli, file surface packages/runtime/**) cannot carry it. Recommend re-labelling and re-dispatching to the engine lane, and folding in #13609 — it is open, needs-user-decision, and is the same seam observed from the delete side ('MetadataManager's unregister DOES broadcast — the observed prolongation sits in an unidentified seam'). One fix closes both."
        }
      ],
      "out_of_scope_findings": [
        "filed as #14019: MetadataFacade's class docblock claims it is 'Registered as the metadata kernel service' while the same file at :134 says nothing on main installs it — the likely source of this card's inverted premise",
        "filed as #14021: MetadataClusterBridgePlugin logs 'bridged' on the in-process memory driver that fans out to nobody (its sibling AuthzClusterBridgePlugin guards exactly this with isInProcessClusterDriver), and has no test file at all"
      ],
      "notes": "serve.ts was READ, never written — the exclusion held. My reading does not conclude the repair belongs there: serve.ts:2851-2899 is dev-only HMR scaffolding and is not on the EE production path. Worktree removed cleanly (git status --porcelain empty, no --force). The branch was pushed empty as the write-route probe and carries no commits."
    }

    Generated by Claude Code

  7. os-steve commented on Sep 1, 2026

    @os-steve
    Collaborator

    PM review — ESCALATING. domain:cli → domain:engine, pm:dispatched → needs-user-decision. ⛔ No code was written, and that was correct.

    domain:cli execution PM seat (#6024) · session session_01UngCYXF98BVpYA9hfz6NYk

    The dispatch's ⛔ was "do not write the facade delegate method first — first produce a reading." The reading came back and falsified both branches of the question, so nothing was written. That is the dispatch working, not failing.

    ⭐ The premise is false, and triage's caution is what caught it

    The card says the TS-config path's 'metadata' service is a runtime dispatcher/domain facade lacking attachClusterPubSub. Measured on a real booted ObjectKernel, it is not:

    getService('metadata') returns createMemoryMetadata() — the kernel's in-memory core fallback — registered by ObjectKernel.preInjectCoreFallbacks() at packages/core/src/kernel.ts:270 through a variable-key call this.registerService(serviceName, fallback).

    ⭐ That is exactly the shape the triage seat said its literal-key query could not see, which is why it labelled its own result 「部分读数,不是零读数」 rather than a census. Had that caution not been written down, the obvious next step was to conclude "only one registrar exists" and build the facade delegate on top of a false premise. The refusal to call a partial reading a zero is what prevented a wrong fix.

    The yes/no question: NO, and it is documented three times

    MetadataPlugin's absence on the host-config boot is deliberate, not an oversight — isHostConfig in plugin-detection.ts ("must NOT be wrapped again"), packages/core/src/fallbacks/memory-metadata.ts's own header naming the host-config path as the one this fallback serves instead of MetadataPlugin, and serve.ts:2865 composing it for host configs in dev only ("production / cloud host deployments are untouched").

    ⭐ The decisive finding — the card's own fallback fix would have been a no-op

    This is the part that would not have been found by reasoning from the card:

    Adding attachClusterPubSub to the slot would attach a bridge that is never asked.

    packages/metadata-protocol/src/protocol.ts owns the entire runtime authoring path (saveMetaItem → applyRegistryWriteThrough → applyObjectRegistryMutation → engine.registry.registerObject) and contains zero references to cluster, pubsub, or getService('metadata'). metadata.changed's only production publisher is MetadataManager.notifyWatchers (metadata-manager.ts:2853), which no PUT /api/v1/meta/* reaches on any boot shape.

    ⇒ The write persists sys_metadata (shared DB — hence meta reads 200 fleet-wide) and mutates only the local engine.registry (hence data-plane 200 on the writer, 404 on peers), with no route to the bus at all. Both faces the card reports are one gap. ⇒ A seat that had "fixed" the facade would have shipped a change, seen the warning disappear, and left the 404s exactly where they were.

    The evidence bar was met, not asserted

    • Two-arm kernel experiment, direction declared in writing first. Arm A (host-config shape): attachClusterPubSub calls = 0, and the bridge emitted the card's warning verbatim. Arm B, positive control, identical harness with a metadata service that does expose it: attach called exactly once, info log emitted, zero warnings. ⇒ Arm A's "never attached" is a measurement, not a harness artefact — the control discriminates.
    • The zero carries two controls. protocol.ts cluster/pubsub sweep returns nothing; control 1 (same instrument, same file, applyRegistryWriteThrough) = 16; control 2 (same pattern, metadata-manager.ts) = 43. Both could have returned non-zero and did.
    • The registrar sweep was re-established, not inherited — triage's literal query reproduced first, then widened to non-literal first arguments, which is what surfaced kernel.ts:270.

    ⛔ The serve.ts exclusion held, and the conclusion vindicates it

    serve.ts was read, never written. And the reading concluded the repair does not belong there: serve.ts:2851-2899 is dev-only HMR scaffolding, not on the EE production path. ⇒ The serial I imposed cost nothing, and #13330 keeps sole ownership of that file.


    Why this becomes needs-user-decision rather than a re-dispatch

    The repair needs a publisher at the protocol's onMetadataMutation choke point plus a peer-side registry applier. That mints new public cross-node contract — a channel, a payload, and an attach entry point — and spans domain:engine + domain:services, touching no domain:cli file. Two consequences:

    1. Clause ② fires on content. New public cross-node contract is a widening, and clause ② bars enqueueing. ⛔ So I cannot put this back in a queue, and I am not going to route it through my lane as though the label were the question.
    2. This card as labelled cannot carry it. Re-labelled domain:engine, unassigned, so the engine lane can claim it cleanly.

    ⚠️ It stays priority:p0. The grading rests on the 2026-08-31 load measurement (200 concurrent creates → 67 × 201 / 133 × 404, with a boot-loaded object at 0 errors as control), and nothing in this reading weakens it — the reading makes the defect deeper, not smaller.

    四棱分析 —— 四面不分歧,一致指向 A

    A 在 protocol 的 onMetadataMutation 扇出,照搬树内 AuthzClusterBridgePlugin 的既有形状(state owner 暴露 attach-pubsub,service-cluster 在 kernel:ready 晚绑),peer 从自己的 sys_metadata 读回重放。B 消费侧自愈:assertObjectRegistered 落空时回落读一次 DB。C 让运行时创作走 metadata service。

    ⇒ 建议 A。 若 EE 现网需要止血,B 可以作为明确临时、大声记录的补充与 A 并行 —— ⛔ 但不能当作修复本身。

    ⭐ Fold in #13609 — one fix closes both

    #13609 is open and needs-user-decision for the same seam from the delete side ("MetadataManager's unregister DOES broadcast — the observed prolongation sits in an unidentified seam"). This reading identifies that seam. ⇒ Whoever takes the decision should take both; splitting them means designing the same channel twice.

    Findings filed on the way past


    Generated by Claude Code

  8. 7 remaining items

  9. claude commented on Sep 1, 2026

    @claude
    Contributor

    Claim: os-dev implementation seat, dispatched by PM loop round 11 (its claim comment is 5491366749)
    Session: session_01Q5WBDtaUnoz5XuJ6jk8pQ5 · mode:subagent os-dev (the dispatching PM seat's subagent; the branch below is this claim's identity bit)
    Branch: claude/issue-13331-metadata-registry-cluster-fanout (pushed empty as the write-route probe before any edit)
    Worktree: objectstack-issue-13331
    File surface: packages/metadata-protocol/src/** (expected primary) plus packages/services/service-cluster/src/** (bridge). READ-ONLY fences honoured: packages/objectql/src/engine.ts (held by PR #13864) and packages/spec/** (single owner, not this lane) — stop-and-report on breach, no writes there under any circumstance.
    Ruling being implemented: Option A per maintainer 2026-09-01 director batch A (comment 5486837547) — publisher at the metadata-protocol's onMetadataMutation choke point, service-cluster bridge late-binds at kernel:ready, peers converge from their own sys_metadata read; Option B not built.


    Generated by Claude Code

  10. claude commented on Sep 1, 2026

    @claude
    Contributor

    os-dev-report

    {
      "issue": 13331,
      "status": "done",
      "branch": "claude/issue-13331-metadata-registry-cluster-fanout",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/14183",
      "premise_still_valid": true,
      "clause_2": "yes — content limb as ruled (new public cross-node contract: channel METADATA_MUTATION_CLUSTER_CHANNEL = 'metadata.mutated', address-only payload ClusterMetadataMutationPayload, attach/detach entry point attachMetadataMutationPubSub on the protocol). Path limb: NO, measured from the actual diff — packages/spec untouched, packages/objectql/src/engine.ts untouched. needs:contract-review attached at PR creation (additive REST POST; response echoed the label; re-read at report time pending the delayed-confirm discipline).",
      "zone2_verdicts": {
        "1_state_owner_is_protocol": "CONFIRMED — the registry write-through choke point AND the sys_metadata read the ruled receipt needs both live in ObjectStackProtocolImplementation; the engine precedent (attachAuthzInvalidationPubSub, engine.ts:3298 read-only) was mirrored in shape only. engine.ts has zero writes; the card proceeded this round.",
        "2_onMetadataMutation_choke_point": "CONFIRMED by symbol on this ref: onMetadataMutation at protocol.ts:4899, docblock 'the ONE choke point every authoring surface funnels through' at :4887 (drifted from the cited ~4875 — verified by symbol as instructed). PARTIAL NUANCE, measured: the funnel covers saveMetaItem / runPublishSideEffects / deleteMetaItem, but the two recovery doors (rollbackMetaItem, revertCommit) run the registry write-through WITHOUT emitting the event — pre-existing gap, local listeners miss it too; filed as #14179 with Blocked-by on this card.",
        "3_extend_existing_bridge": "CONFIRMED — MetadataClusterBridgePlugin gained an independent lane 2 (protocol service, duck-typed) beside the untouched lane 1; Runtime's default composition already mounts this plugin, so no new wiring anywhere. Lane 2 carries the isInProcessClusterDriver guard from birth; #14021's lane-1 misreading NOT absorbed (lane 1 byte-identical behavior, warn line verbatim).",
        "4_reuse_applyRegistryWriteThrough": "CONFIRMED — the peer applier is its deliberate FIFTH caller (active row present), with restoreArtifactRegistryView for the no-active-row leg; the caller-count pin, header trace and a route-5 wire-plural fold case are updated in protocol.object-registry-write-through-spelling.test.ts."
      },
      "spec_fence_answer": "No split owed — the payload lives entirely in metadata-protocol types, matching both shipped precedents (authz payload in core's channel module; ClusterMetadataChangedPayload in the metadata package); IPubSub from spec/contracts is the existing generic transport. The spec's dormant MetadataChangedEventPayloadSchema: zero producers, zero consumers in-tree, declared version-compare semantics opposite to the ruled re-read receipt, z.bigint() version that cannot cross JSON — filed as #14180 for the spec lane.",
      "summary": "Option A implemented as ruled. The protocol's post-persistence funnel now publishes the mutation's address on the new metadata.mutated cluster channel (drafts never published, originNode loopback suppression, partitionKey type:name); on receipt a peer re-reads the row from its OWN sys_metadata and re-runs the registry write-through (or the delete heal walk when no active row remains), then replays the event into its local onMetadataMutation listeners without re-publishing — authored hook/action rebinds converge on peers too. The bridge late-binds the new seam at kernel:ready as an independent second lane, so the shipped EE boot shape (host-config, fallback metadata slot, lane 1 warns) gets the data-plane fix. Option B not built. Both fences held: engine.ts and packages/spec have zero writes.",
      "files_changed": [
        ".changeset/metadata-mutation-cluster-fanout.md",
        "packages/metadata-protocol/src/index.ts",
        "packages/metadata-protocol/src/protocol.ts",
        "packages/metadata-protocol/src/protocol.cluster-mutation-fanout.test.ts (new)",
        "packages/metadata-protocol/src/protocol.object-registry-write-through-spelling.test.ts",
        "packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts",
        "packages/services/service-cluster/src/metadata-cluster-bridge-plugin.test.ts (new — first test file for this plugin)",
        "scripts/engine-double-contract.pinned.json (gate's own --write)"
      ],
      "tests": "TWO-ARM, directions declared in the test-file header before running: Arm B (bridge attached, two protocol replicas over one shared store) — peer converges from its own read, 'Arm B' case asserts peer registeredObjects == [gadget]; Arm A (control, no attach) — identical write leaves peer empty, so Arm B is the bridge's doing. 11 cases total in protocol.cluster-mutation-fanout.test.ts: address-only payload key-set, draft silence, loopback, duplicate-delivery convergence, re-attach idempotency, detach, delete fan-out heal (unregisterObject + removeObjectOverlay on the peer), draft-discard keeping the peer's active registration, listener replay ordered after registry convergence. COMMITTED-TREE ABLATION at 44a2e59b, direction declared first: deleting the single publisher call — mutation proved ON DISK by anchored greps (anchor count 0, marker count 1), not by editor exit codes — went red on exactly the 7 declared bus-crossing cases, 15 green (Arm A, loopback, draft silence, idempotency, route-5 all held); restore proved by state: HEAD-blob equality d7ba0225 both sides, empty git diff HEAD, marker count 0; restored run 22/22 green; trap EXIT INT TERM restore with absolute REPO_ROOT paths was armed but the state checks are the proof. REBUILD NOTE: these tests import ./protocol.js relative from the same package src — the subject never resolves through dist/exports, so no build leg exists on either ablation leg; stated rather than skipped. SUITES at final head e4601935: metadata-protocol vitest 'Test Files 152 passed | 2 skipped (154), Tests 2094 passed | 10 skipped'; service-cluster 'Test Files 6 passed (6), Tests 79 passed (79)'; tsc --noEmit clean, with both edited test files confirmed IN the program via --listFiles (2 hits — the exclude trap checked, not assumed). All heavy runs through scripts/pm/os-verify-lock.sh; verdicts read from its VERDICT lines (command-exit 0), never bare dollar-question.",
      "gates": "Derived at the ACTUAL diff, no paths passed: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands; provenance line names objectstack-ai/objectstack at e4601935 and the --repo assertion holds. 44 commands derived (grew from 37 after the ledger touch pulled scripts-family gates — re-derived and re-run at final head). 41 exit 0 (exit captured by redirect before any pipe, per-gate logs kept). 3 NOT MEASURED by their own printed verdicts, never read as pass or fail: check-test-completeness (exit 3, grades a CI turbo log none exists locally), check:dual-build-cjs-loads (exit 3, needs full workspace build — CI-owned), check:type-check-debt (exit 3, full re-measure — CI-owned). Two first-pass reds fixed and re-run to their own OK lines: check:engine-double-contract 'OK — 741 pinned, 134 in the DEBT ledger, 3 exempt' (ledger learned the fanout file's pinned doubles via the gate's own --write); check:objectql-double-limit 'conformance holds: 299 doubles graded' (the new find double now holds the caller's bound by presence). check-nul-bytes OK (7740 files). Repo-level lint scan not run locally — CI-owned per standing rule.",
      "mcp_calls": "0 — card body + all 9 comments read via the zero-quota page payload (first 4) then repo-scoped REST after a green probe (HTTP 200 on /issues/13331); all writes (claim comment 5491428456, findings #14179/#14180/#14181, draft PR #14183, needs:contract-review label, this report) via the REST core bucket; dedup via REST open-issues list (5 pages, 443 issues, completeness reconciled against open_issues_count 452 incl PRs) + local grep with a firing positive control (attachClusterPubSub: 3 hits incl #13331).",
      "deviations": [
        "Channel switch, declared: the page-payload archive carried only 4 of 9 comments (SSR backTimelineItems ships empty; timeline totalCount 36) — switched to repo-scoped REST after the probe answered 200. No comment or body showed sanitizer truncation (every tail ends in its footer).",
        "Receipt behavior beyond the ruling's minimum, declared in the PR for contract review: after registry convergence the peer replays the event into its LOCAL onMetadataMutation listeners (never re-published) — the remote-replay shape MetadataManager.attachClusterPubSub already ships for watchers; it is what makes authored hook/action rebinds converge on peers.",
        "The applyRegistryWriteThrough caller-count pin was re-opened 4 to 5 — the pin's own designed procedure (human re-reads the trace): trace updated, route-5 spelling case added, fold via canonicalMetaType at the wire boundary.",
        "PR body footer: platform appended a second bare footer after my session-URL footer at create time; left as-is per create-only treatment (no re-edit loop).",
        "origin/main advanced twice during the run (2bff79ae3 to 45b90512, PR #13864 landed among others); measured ZERO file intersection with my merge-base diff, so no merge was performed — CI tests the merge result. The anticipated #14114 barrel conflict cannot arise: no export was added to service-cluster's index.ts."
      ],
      "out_of_scope_findings": [
        "filed as #14179: rollbackMetaItem / revertCommit run the registry write-through but never emit onMetadataMutation — local boot-cached rebinds and (post-#13331) cluster fan-out both miss recovery writes; Blocked-by: #13331",
        "filed as #14180: spec's MetadataChangedEventPayloadSchema declares a MUST-emit metadata persistence contract with zero producers/consumers in-tree and a z.bigint() version that cannot cross JSON — ADR-0049 enforce-or-remove candidate, spec lane",
        "filed as #14181: service-cluster tests compile under no tsc program (services-family instance of #14062's class) — a pre-existing TS2322 sits in memory.contract.test.ts, the package's contract witness"
      ],
      "open_questions": []
    }

    Generated by Claude Code

  11. claude commented on Sep 1, 2026

    @claude
    Contributor

    os-dev-report

    {
      "issue": 13331,
      "status": "done",
      "round": "rework — CI census red on PR #14183, repaired and validated",
      "branch": "claude/issue-13331-metadata-registry-cluster-fanout",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/14183",
      "premise_still_valid": true,
      "clause_2": "restated as instructed: content limb YES per the ruling (new public cross-node contract — channel 'metadata.mutated', address-only payload ClusterMetadataMutationPayload with originNode, attach/detach entry point on the protocol); path limb NO, confirmed from the landed diff — ClusterMetadataMutationPayload lives in packages/metadata-protocol, packages/spec has zero writes, packages/objectql/src/engine.ts has zero writes. The spec fence held; no contract-first split is owed.",
      "summary": "check-system-context-census red at e4601935 was pure line rot from this PR's own insertion: the metadata.mutated publisher block sits above stripReadonlyForInsert in protocol.ts, moving its context.isSystem read 1737 to 1741 while content/docs/permissions/system-context.mdx row 21 still anchored 1737. Reproduced locally byte-for-byte (same two problems, exit 1 captured by redirect), repaired with the gate's own --fix, and VERIFIED as a SHIFT not a population change: the tool rewrote exactly one line (the anchor's line number), zero prose, and did not refuse; census population identical before and after (109 sites / 145 anchors). Bare re-run green: 'OK — 109 elevation read sites in 20 packages across 45 files, all anchored; 145 anchors resolve, 27 declared non-read.' Committed as cdbe5f0b, pushed after validation — one validated push. PR body notes the docs file as the re-anchor, not scope creep.",
      "gates": "Union re-derived at cdbe5f0b (provenance line names this repo at that commit; --repo holds): the docs file grew the family 44 to 70. All 70 run at cdbe5f0b: 68 exit 0 — including check-system-context-census green, and check:skill-examples upgraded from a prerequisite refusal to a REAL green ('260 prose examples type-check across 3 surfaces') after building the client-SDK closure it reads (rest, runtime, plugin-auth, verify, client, client-react; two closure quirks en route, neither mine: runtime's dts needs @objectstack/rest dist via a dev-edge outside the dependency closure, and verify races runtime's dts write under workspace-concurrency 2 — resolved by serial builds). 2 remain NOT MEASURED by their own printed verdicts, CI-owned: check-test-completeness (exit 3, grades a CI turbo log) and check:dual-build-cjs-loads (exit 3, needs the full 47-package workspace build). Suites re-anchored at cdbe5f0b: fanout+spelling+mutation-listeners 36/36, service-cluster 79/79 (VERDICT command-exit 0 under the verify lock); the full metadata-protocol suite's 2094-test green stands cited at e4601935 — the only delta to cdbe5f0b is one line number in a docs table, no package source touched.",
      "docs_drift_assessment": "content/docs/kernel/cluster.mdx section 6.2 is now INCOMPLETE, as the coordinator suspected: it states cross-node metadata invalidation 'already works' through the manager's metadata.changed lane alone — exactly the account #13331 falsified for the runtime-authoring path (the protocol never reaches the manager; host-config boots carry no manager at all). It needs a paragraph on the metadata.mutated protocol lane and the bridge's second attach. Its own closing principle there ('the peer re-reads the shared store, which is the source of truth') is the rule the new receipt implements, and its 'Target spec (planned)' block already marks MetadataChangedEventPayloadSchema as not wired — corroborating finding #14180. Same incompleteness class for the section 5/7 bridge mentions and content/docs/concepts/metadata-lifecycle.mdx. Stated in the PR body and here; corpus NOT rewritten per dispatch; releases pages untouched.",
      "files_changed_delta": [
        "content/docs/permissions/system-context.mdx (one line: anchor 1737 to 1741 — the census gate's own --fix; noted in the PR body)"
      ],
      "mcp_calls": "0 this round (cumulative 0) — repro, fix, gates, push all local/git; PR body PATCH and read-backs via repo-scoped REST (PATCH sent body field only; draft read back true before and after).",
      "deviations": [
        "check:skill-examples' local reading required building six packages outside my change's own closure — done under the verify lock rather than recorded NOT MEASURED, because the gate's refusal text names the exact prerequisite and it was locally satisfiable; the two remaining exit-3 gates are not (CI-owned inputs).",
        "PR body PATCH: the session-URL footer survived this PATCH (both footers present at tail); no re-paste loop performed either way."
      ],
      "out_of_scope_findings": [],
      "open_questions": []
    }

    Generated by Claude Code

  12. claude commented on Sep 1, 2026

    @claude
    Contributor

    os-dev-report

    {
      "issue": 13331,
      "status": "done",
      "round": "merge-conflict resolution on PR #14183 — resolved with the tooling, one validated push",
      "branch": "claude/issue-13331-metadata-registry-cluster-fanout",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/14183",
      "premise_still_valid": true,
      "summary": "git merge origin/main AUTO-completed (58321d2d) — no textual conflict locally — but the auto-merged census page was WRONG: it kept this branch's pre-#14171 anchor rows beside my row-21 edit, and the bare census gate flunked the merged tree 26 ways (paired stale-anchor / unanchored-site entries in engine.ts, deltas +64..+95, and share-link-service.ts, delta +11 — the coordinator's predicted mechanism, adjacent rows of one machine-maintained table). Resolved exactly per instruction: took origin/main's page wholesale, re-derived with check-system-context-census --fix on the merged tree. The tool performed a SHIFT, not a population change, and did not refuse: it rewrote exactly ONE anchor — row 21, protocol.ts 1737 to 1741 (re-derived on the merged tree; same number as before but not carried forward on assumption). Delta vs origin/main's page is that single line, zero prose. The repo's own os-regen commit hook confirmed the deferred generated artifact current and cleared its marker. Committed as 0cf0a437, pushed once after full re-validation. PR now reads mergeable true / mergeable_state blocked (normal draft-plus-pending-checks aggregate), head 0cf0a437, draft still true — not flipped ready, no enqueue, no auto-merge.",
      "readings_retaken_at_0cf0a437": [
        "census gate bare: 'OK — 109 elevation read sites in 20 packages across 45 files, all anchored; 145 anchors resolve, 27 declared non-read' (red 26 problems before resolution, green after — both readings on the merged tree)",
        "metadata-protocol full suite: Test Files 152 passed | 2 skipped (154), Tests 2094 passed | 10 skipped — after rebuilding the dependency closures on the merged tree (main's five commits touch packages/spec sources, so pre-merge dists were stale; 判断任何事之前先 build)",
        "service-cluster full suite: Test Files 6 passed (6), Tests 79 passed (79)",
        "metadata-protocol tsc --noEmit: clean (same lock hold, VERDICT command-exit 0)",
        "check-nul-bytes: OK (7757 files)"
      ],
      "readings_carried_forward": "Everything else is carried, stated as such and not restated as fresh: the 70-gate union (68 green / 2 CI-owned NOT MEASURED) stands at cdbe5f0b; the two-arm experiment and the committed-tree publisher ablation stand at 44a2e59b; the full-suite green also previously taken at e4601935/cdbe5f0b is superseded by the fresh 0cf0a437 run above. Per the coordinator, the union was not re-run — the merge brings no new files into this PR's diff (the only diff-file the five main commits touched is the census page itself, now resolved to main-plus-one-line).",
      "files_changed_delta": [
        "merge commit 58321d2d (origin/main into the branch — no rebase, no force-push, no stash)",
        "0cf0a437: content/docs/permissions/system-context.mdx re-derived on the merged tree (9 lines vs the merge commit — 8 rows restored to main's re-anchors, 1 row re-shifted for this branch's insertion; exactly 1 line vs origin/main)"
      ],
      "mcp_calls": "0 this round (cumulative 0) — merge, resolution, validation, push all local/git; PR state read via repo-scoped REST.",
      "deviations": [
        "The coordinator's step 1 anticipated a conflicted merge; git auto-merged instead — but the auto-merge was semantically wrong in exactly the predicted file, caught by running the census gate on the merged tree BEFORE any push. Steps 2-4 were then followed verbatim (main's page wholesale, --fix re-derive, shift verified, no refusal).",
        "Resolution was committed as its own commit (0cf0a437) after the merge commit rather than folded into it: the census tool was run against a committed merge tree, mirroring the repo's commit-then-regenerate rule for generated artifacts; the repo's os-regen hook endorsed the sequence (deferred-artifact marker cleared at commit)."
      ],
      "out_of_scope_findings": [],
      "open_questions": []
    }

    Generated by Claude Code

  13. os-support-ai commented on Sep 1, 2026

    @os-support-ai
    Collaborator

    ✅ Contract review PASS at CONTRACT_REVIEW_TIER → needs:contract-review cleared, PR #14183 ready and enqueued

    domain:engine lane PM, session session_01Q5WBDtaUnoz5XuJ6jk8pQ5, R11.

    ⛔ This seat is below tier, so it did not self-review. Clause-② was ruled to fire on this card, and the enqueue gate forbids landing below CONTRACT_REVIEW_TIER — so the review was delegated to a reviewer running at that tier, per the in-seat review rule. Its full verdict is on the PR at 5494333461. Verdict: PASS, no ESCALATE, three findings, all non-blocking.

    What the review measured (its readings, not mine)

    • Content limb: YES, declared honestly and completely. The minted surface is exactly what the PR declares — METADATA_MUTATION_CLUSTER_CHANNEL = 'metadata.mutated' and ClusterMetadataMutationPayload (the only two new barrel exports) plus attachMetadataMutationPubSub / detachMetadataMutationPubSub. Base-tree grep at 9dd022ae for all three: zero hits, with a firing positive control. No undeclared widening.
    • Two lanes are correct, not duplicate — measured: lane 1's only production publisher is MetadataManager.notifyWatchers, and protocol.ts at head has zero code references to the metadata service (the 3 hits are doc comments this PR adds). Host-config boots carry no manager at all, which is why lane 2 duck-types the real implementation.
    • Address-only verified at three levels: the payload type, all three emit sites, and the receiver — which consumes the wire purely as an address and registers from its own DB read. That is the ruling's load-bearing clause and it holds.
    • Failure directions safe: duplicates idempotent (pinned), out-of-order converges because the DB read decides, a lost signal degrades to the pre-existing boot-reload bound, a failed apply fails closed (404 staleness, never wrong data).
    • Option B was not built, as ruled.

    What I verified myself before enqueueing (⛔ not taken from the review)

    Check Reading
    ⛔ packages/objectql/src/engine.ts fence HELD — 0 in the diff's file list
    ⛔ packages/spec/** fence HELD — 0; no contract-first split owed
    Governed surface 0 across all 9 files ⇒ ready→enqueue path, not the maintainer fork
    Mergeable vs current main (b992b1d6) git merge-tree --write-tree exit 0, clean — measured locally rather than trusting the API's cached flag, since main moved 8 commits after this head was built
    ⚠️ Generated ledger scripts/engine-double-contract.pinned.json 0 commits touched it since the merge base (8 landed overall; control confirms the path is tracked with 2 prior commits) ⇒ no silent-drop risk, so the regen four-step is not owed here
    CI at head 0cf0a437 33 checks, 32 success + 1 skipped, zero failures

    ⭐ The ledger check is the one I would not have skipped: a clean textual merge on a generated artifact is exactly where one side's entries vanish without a conflict — this lane paid for that lesson earlier today on the census page. Here it was genuinely clean, and measured rather than assumed.

    The three findings, and their dispositions — all non-blocking

    1. Peer apply race — no per-key serialisation at the receiver, so two rapid same-row mutations can under DB-response jitter leave a peer one version behind until the next signal or boot. Same class as two racing local writers, and inside the PR's own declared staleness bound. Receiver-internal, fixable later with no contract change ⇒ not a landing blocker.
    2. deleteMetaItem's legacy control-plane branch never emitted even locally (pre-existing, marked deliberate), so it fans out to no one. Same family as metadata-protocol: rollbackMetaItem / revertCommit registry write-throughs never emit onMetadataMutation — boot-cached rebinds and (post-#13331) cluster fan-out miss recovery writes #14179 but not named there ⇒ recommendation is to append it to metadata-protocol: rollbackMetaItem / revertCommit registry write-throughs never emit onMetadataMutation — boot-cached rebinds and (post-#13331) cluster fan-out miss recovery writes #14179, ⛔ not widen this PR.
    3. Wording — the payload docblock's "at-least-once … harmless" is a tolerance statement, not a delivery guarantee. The changeset and PR body already state the at-most-once loss bound plainly.

    ⛔ None of the three is a design fork the ruling leaves unsettled, so none goes to the maintainer.

    State transition

    needs:contract-review cleared (read back: the other four labels intact, nothing else stripped) · draft → ready · auto-merge enabled, in the merge queue.

    ⚠️ Owed at landing — do not let this one slip

    #13609 is pm:blocked on this card per the ruling, and the ruling is explicit: when this lands, re-verify the deleted-entry prolongation on a multi-node deployment before closing it. ⭐ That is not a formality — this PR's own reading is what identified the seam #13609 observed from the delete side, so #13609 is the check that the identification was right. I will carry it at the landing.

    Also queued behind this landing: #14179 (recovery doors rollbackMetaItem / revertCommit run the write-through but never emit the mutation event even locally) carries Blocked-by this card, and finding 2 above belongs on it.


    Generated by Claude Code

  14. claude commented on Sep 1, 2026

    @claude
    Contributor

    os-dev-report

    {
      "issue": 13331,
      "status": "done",
      "round": "second merge-conflict resolution on PR #14183 — mergeable again, one validated push",
      "branch": "claude/issue-13331-metadata-registry-cluster-fanout",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/14183",
      "premise_still_valid": true,
      "summary": "git merge origin/main auto-completed again (d2e4679a); per the known trap, textual cleanliness was treated as evidence of nothing. Followed the procedure verbatim: took main's content/docs/permissions/system-context.mdx wholesale (main's #14199 re-anchored 11 rows of that machine-maintained table), then re-derived on the merged tree with check-system-context-census --fix. The tool performed a SHIFT, not a population change, and did not refuse: exactly ONE anchor rewritten — row 21, protocol.ts 1737 to 1741. The number equals last round's but was RE-DERIVED, not carried: measured zero main-side commits on packages/metadata-protocol/src/protocol.ts since 0cf0a437, so this branch's insertion shift is unchanged — the differing-anchor warning was prudent and the measurement answered it. Bare gate green: 'OK — 109 elevation read sites in 20 packages across 45 files, all anchored; 145 anchors resolve, 27 declared non-read.' Delta vs origin/main's page is that single line, zero prose. The os-regen commit hook again confirmed the deferred artifact current. Committed 7ffeb726, pushed once after full re-validation. PR now: mergeable true, mergeable_state blocked (ready PR awaiting checks — auto-merge armed by the coordinator), head 7ffeb726, draft false — draft state and auto-merge untouched by me as instructed.",
      "pinned_ledger_recheck": "scripts/engine-double-contract.pinned.json re-measured after the merge as instructed: ZERO main-side commits touched it (git rev-list count over 0cf0a437..origin/main), and the gate run on the merged tree prints its own OK — 'check-engine-double-contract: OK — 741 pinned, 134 in the DEBT ledger, 3 exempt' — so the merge dropped nothing and no --write regeneration was owed.",
      "readings_retaken_at_7ffeb726": [
        "census gate bare: OK line quoted above (on the merged tree, after --fix)",
        "check:engine-double-contract: OK — 741 pinned, 134 DEBT, 3 exempt (merged tree)",
        "metadata-protocol full suite: Test Files 152 passed | 2 skipped (154), Tests 2094 passed | 10 skipped — after rebuilding both dependency closures on the merged tree (main moved ~15 commits incl. spec/objectql/runtime sources)",
        "service-cluster full suite: Test Files 7 passed (7), Tests 93 passed (93) — the package GREW under the merge (#14114's multi-node-gate-mount.test.ts landed) and passes alongside the bridge tests; the two features coexist with no barrel conflict, as predicted",
        "metadata-protocol tsc --noEmit: clean (same lock hold, VERDICT command-exit 0)",
        "check-nul-bytes: OK (7769 files)"
      ],
      "readings_carried_forward": "Stated as carried, not fresh: the 70-gate union (68 green / 2 CI-owned NOT MEASURED) stands at cdbe5f0b; the two-arm experiment and committed-tree publisher ablation stand at 44a2e59b. The union was not re-run per the coordinator's earlier scoping — this merge brings no new files into the PR's diff beyond the census page already handled.",
      "files_changed_delta": [
        "merge commit d2e4679a (origin/main into the branch — no rebase, no force-push, no stash)",
        "7ffeb726: content/docs/permissions/system-context.mdx re-derived on the merged tree (11 lines vs the merge commit — main's #14199 re-anchors restored, row 21 re-shifted; exactly 1 line vs origin/main)"
      ],
      "mcp_calls": "0 this round (cumulative 0) — merge, resolution, validation, push all local/git; PR state read via repo-scoped REST.",
      "deviations": [
        "The merge again auto-completed rather than conflicting; the semantic wrongness was assumed rather than trusted-away, and the census gate was run on the merged tree before any push — red-to-green sequence observed locally (the --fix rewrite is itself the red's proof: main's page anchored 1737 for a tree where the read sits at 1741)."
      ],
      "out_of_scope_findings": [],
      "open_questions": []
    }

    Generated by Claude Code

  15. os-support-ai commented on Sep 1, 2026

    @os-support-ai
    Collaborator

    Conflict resolved — PR #14183 mergeable again at 7ffeb726. Contract-review PASS and auto-merge untouched.

    domain:engine lane PM, session session_01Q5WBDtaUnoz5XuJ6jk8pQ5. Maintainer flagged the conflict; routed to the branch's own dev rather than a second actor on the branch.

    ⚠️ The conflict was invisible to every textual check — again

    git merge auto-completed with no conflict, for the second time on this branch. ⛔ That was treated as evidence of nothing, which is the only correct reading here: main's #14199 (34ce8e7d) re-anchored 11 rows of content/docs/permissions/system-context.mdx while this branch re-anchored 1. Different rows, so git merges them happily — but both sides' line numbers were derived against different protocol.ts trees, and the merged tree matches neither.

    ⭐ The --fix rewrite is itself the proof the merge was wrong: main's page anchored 1737 for a tree where the read sits at 1741.

    Resolution per procedure: main's page taken wholesale, re-derived with check-system-context-census --fix on the merged tree. A shift, not a population change — exactly one anchor rewritten, zero prose. Bare gate green: "OK — 109 elevation read sites in 20 packages across 45 files, all anchored; 145 anchors resolve, 27 declared non-read."

    ⭐ And the number that came back was the same as last round — which the dev measured rather than waved through. I warned to expect a different anchor and not to carry 1741 forward. It re-derived, got 1741 again, and then answered why: zero main-side commits touched packages/metadata-protocol/src/protocol.ts since 0cf0a437, so this branch's own insertion shift is unchanged. A matching number reached by measurement is a reading; the same number carried forward would not have been.

    The two things I asked to be re-measured after the merge

    • Generated ledger scripts/engine-double-contract.pinned.json — re-measured on the new merge base: zero main-side commits touched it, and the gate prints its own OK on the merged tree (741 pinned, 134 DEBT, 3 exempt). Nothing dropped; no --write regeneration owed. My pre-enqueue reading was 15 commits stale, so this was the right thing to re-take rather than inherit.
    • Suites on the merged tree — metadata-protocol 2094 passed, tsc --noEmit clean, after rebuilding both dependency closures (main moved ~15 commits including spec/objectql/runtime sources).

    ⭐ A predicted risk resolved itself, and the measurement says so

    service-cluster grew under the merge — 7 files / 93 tests, up from 6 / 79 — because #14114's multi-node-gate-mount.test.ts landed. That is the PR I named at dispatch as the barrel-conflict risk, with the standing ⛔ never to resolve such a conflict by dropping its exports. It never arose: this card added no barrel export, the two features coexist, and both suites pass together.

    State

    mergeable: true, ready, auto-merge still armed from the contract-review pass — the dev left draft state and auto-merge untouched as instructed. It will re-queue and land on its own once CI is green on 7ffeb726. ⛔ Nothing about the review outcome was reopened.

    📌 Still owed at landing: #13609's multi-node re-verification of the deleted-entry prolongation, per the ruling.


    Generated by Claude Code

  16. os-support-ai commented on Sep 1, 2026

    @os-support-ai
    Collaborator

    ✅ LANDED — PR #14183 merged as 1403d943, verified by content on origin/main

    pm:dispatched → stripped. pm:blocking → stripped (both blockees released below). The lane's only priority:p0 is closed.

    The landing probe, and why it discriminates

    ⛔ Per this seat's own rule, the API's merged field is not the evidence. The probe was run against the tree, and it had to be able to come back negative:

    parent 1134e47a origin/main 1403d943
    METADATA_MUTATION_CLUSTER_CHANNEL | ClusterMetadataMutationPayload | attachMetadataMutationPubSub across packages/ absent (exit 1, empty) protocol.ts ×7, index.ts ×1, metadata-cluster-bridge-plugin.ts ×4, plus two new test files

    ⭐ Positive control on the same parent commit, same instrument, same path: a control string in protocol.ts returned 4 on 1134e47a. So the zero above is a reading — the probe was pointed at a file that existed and was being read — not a mis-spelled query against a tree the grep never reached.

    Independently: git log origin/main -- <file> names 1403d943 as the last touching commit for both subject files (packages/metadata-protocol/src/protocol.ts, packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts).

    Fences — both held post-merge

    Declared at dispatch as untouchable; re-checked against the merged tree rather than against the pre-merge diff:

    What actually landed (9 files, +1199 / −45)

    packages/metadata-protocol/src/protocol.ts                            +245
    packages/metadata-protocol/src/protocol.cluster-mutation-fanout.test.ts +466 (new)
    packages/metadata-protocol/src/protocol.object-registry-write-through-spelling.test.ts +61/−
    packages/metadata-protocol/src/index.ts                                 +7
    packages/services/service-cluster/src/metadata-cluster-bridge-plugin.ts +182/−45
    packages/services/service-cluster/src/metadata-cluster-bridge-plugin.test.ts +214 (new)
    .changeset/metadata-mutation-cluster-fanout.md                          +52
    scripts/engine-double-contract.pinned.json                              +15
    content/docs/permissions/system-context.mdx                              ±1
    

    ⚠️ Two of these are the hazard classes this seat has been tracking, and both came through intact:

    • content/docs/permissions/system-context.mdx — the census anchor table, ±1 line. This is the file that merges cleanly and lands semantically wrong when two PRs derive their anchors against different protocol.ts trees. It was re-derived with the gate's own --fix on the merged tree, not hand-edited.
    • scripts/engine-double-contract.pinned.json — a generated ledger, where a zero-conflict merge silently drops one side. Concurrent-commit count on that path was measured twice: before enqueue and after the merge, zero both times.

    Emit-site inventory on the landed code

    emitMetadataMutation has exactly three call sites, one per authoring verb — saveMetaItem (:15071), publishMetaItem (:16305), deleteMetaItem (:20043) — with the local half (emitMetadataMutation → listeners) and the cluster half (publishMetadataMutation → METADATA_MUTATION_CLUSTER_CHANNEL) split so a received peer event replays locally without re-publishing. That split is the loop guard; originNode on the payload is the second.

    📌 Released by this landing

    Both were pm:blocked on this card and both move to pm:queue in this same pass:


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdomain:enginepriority:p0Critical: blocker, must ship before MVP

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions