Read a subscribed record once per update instead of once per subscriber

[kriszyp]

Live subscription delivery now reads a record once per update instead of once per subscriber. Every subscriber to a record checked its event against the record's current version with its own primaryStore.getEntry call (eventFromAudit in resources/Table.ts), so an update to a record with N subscribers on a thread did N identical reads. The read is now memoized by currentEntryForAudit on the audit record every one of those subscribers receives, for the duration of that synchronous notify pass. The invariant it relies on is recorded in resources/DESIGN.md (subscription version selection) and its root index entry. For a record with many subscribers this cuts CPU per delivery by roughly 10–15% and moves the saturation point for update-driven fan-out: at the rate where main delivers 58–86% of messages with seconds of p99 latency, this branch delivers 97–99% with p99 under 400 ms.

For the human reviewer

1. Requirement. This came out of WebSocket scale benchmarking: CPU profiles of record-update fan-out showed the per-subscriber getEntry as the largest single JavaScript cost (~10% of worker CPU in a hot-record scenario). The change is warranted for any "many clients watching one record" workload (live scores, dashboards). It is independent of other in-flight work and small enough to land on its own.
2. Where the memo lives. Chosen: module-level slots in Table.ts, keyed on audit-record identity, store and record id, cleared by a queueMicrotask after the pass. Alternatives: a per-record slot on the audit-record object (adds a field to every decoded audit entry on every read path, replication included), or an entry passed from the notify loop in transactionBroadcast.ts (couples the two modules). Easy to change later; a "no" costs only the refactor.
3. Cost left on the default path. One queueMicrotask per synchronous turn that fills the memo, even when a key has a single subscriber and saves nothing. It is per turn, not per record or subscriber. The zero-allocation alternative is the caller-owned clear from item 2.
4. LMDB shares the value object. On HARPER_STORAGE_ENGINE=lmdb, subscribers of a key now receive one shared event.value object instead of one decode each (RocksDB already shared it through its record cache). A rowFilter freezes that value, so a listener that mutates event.value would now throw on LMDB as it already could on RocksDB. Records are immutable by contract.
5. Scope. Only the default current-version path is memoized. includeSuperseded patch reconstruction (durable MQTT receiving patches) still reads once per subscriber; covering it would be an additive follow-up.
6. Release line. Milestoned v5.4 (main). The benefit applies to deployments running 5.2/5.3 today; backporting is a customer-need call.

Verification

• Unit: subscriptionEntryRead.test.js counts getEntry calls for the record while 1 and then 20 subscribers receive an update, and asserts the count does not grow with subscriber count and that every subscriber gets the new value. Fails on main (22 reads for 20 subscribers vs 3 for one), passes here. Subscription suites (subscriptionEntryRead, subscriptionSuperseded, subscriptionValueIdentity, subscriptionReplay) pass on RocksDB (64 passing) and LMDB (75 passing).
• Full gates at 827247cd9 (fresh npm ci): test:unit:main 6289 passing; test:unit:resources 3707 passing; test:integration:all 2285 passing, 0 failing, 6 cancelled — the integrationTests/server/ollama-backend.test.ts suite, which only runs when a live Ollama server is present and fails at module load (systemSchema.json needs an import attribute) identically on main (726dd1e).
• End-to-end (live measurement): one record with 50,000 MQTT-over-WebSocket subscribers connected over Harper's per-worker Unix sockets (as behind a TLS-terminating proxy), updated by REST PUT; 8 worker threads pinned to 8 CPUs capped at 2 GHz; two interleaved main/branch pairs. CPU per delivery at 2 updates/s: 18.0 / 17.1 µs on main vs 16.2 / 14.7 µs here; at 5 updates/s: 17.3 / 17.5 µs vs 15.6 / 14.1 µs. At 10 updates/s (the saturation point) main delivered 58% / 86% of the fan-out with p99 5.0 / 0.96 s, this branch 97% / 99% with p99 0.37 / 0.31 s. The load generator is a benchmark harness to be submitted separately.

Complexity: medium

_{Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex; adjudicated=domain; declined=cursor-grok,cursor-kimi,cursor-muse; rounds=2; full=1 @ afc9b2f}

_{Human-Review-Need: 3 (decisions: memo-lifetime-owner, memo-scope-default-path-only) @ ca64a6e}

[gemini-code-assist]

Code Review

This pull request optimizes subscription version selection in Table.ts by introducing a memoization mechanism (currentEntryForAudit) that ensures the current entry is read only once per audit record during a synchronous notify pass, rather than once per subscriber. The changes also include corresponding updates to the design documentation and a new unit test to verify the optimization. The review feedback correctly identifies a TypeScript type definition issue where memoizedEntryId is declared as Id but is initialized and reset to undefined, which would cause compilation errors under strict null checks.

[claude]

Reviewed; no blockers found.