Resume analytics aggregation from the last raw record it rolled up

[kriszyp]

An analytics aggregation cycle rolls up one analytics.aggregatePeriod window of hdb_raw_analytics at a time, and it used to resume from a marker stamped with the cycle's own end-of-run Date.now(). Because the next cycle starts after that marker, every raw report between the end of the window and the end of the cycle was discarded: the remainder of a backlog left by a late tick, and anything a worker reported while the cycle was running. get_analytics under-counted by whatever fell in the gap, and the lost samples never came back.

The resume point is now rawCursor, which moves only to a record a cycle actually read — never to a clock — and stays where it is when a cycle reads nothing. lastAggregationTime goes back to being only the cadence marker, so an idle node still aggregates once per period.

For the human reviewer

1. Is this the right task at all? It came from main going red on harper-pro at 8b570b7db — one leg, Cluster Integration Tests 5/6 (Node.js v22), in integrationTests/cluster/replicatedAnalyticsUnion.test.mjs. The same shard passed on Node 24 and 26.5.0 at that SHA, so the cheap reading was "flake, rerun it". The trace says otherwise: that test writes five records and then waits 45 s for one db-write aggregate row, and when the raw report carrying those writes lands in the dropped gap, no later cycle can produce it. Rerunning would have turned CI green and left a metrics-loss bug in place, so the fix is warranted; whether it must ship as part of getting main green, rather than on its own schedule, is the call to make.
2. The cursor stays in memory. The planning round returned better-alternative-exists and asked for a cursor persisted in hdb_analytics atomically with the rows derived from each window. Half of that alternative is adopted here — cadence and raw progress are separate — but not the durable half. Two cases therefore stay open: a restart mid-drain reseeds from an end-of-cycle stamp and skips the rest of the backlog, and storeMetric discards table.put()'s result, so a failed aggregate write still advances the cursor. I overruled the durable half for this change because hdb_analytics is user-visible — get_analytics reads it, metric discovery enumerates it, and the replicated union fan-out concatenates it across peers — so a new record shape there is API surface that needs its own design round, and neither case is what reddened CI. Both are recorded in DESIGN.md and the adjudicator classed them as pre-existing. Reversing this means a second PR, not a revert.
3. Catch-up skips the cadence guard. A cycle that stops at its window edge sets aggregationBehind, and the next cycle runs without waiting out the period, so a stall of S seconds clears in about S seconds. Keeping the cadence instead means a backlog never drains; rolling the whole backlog up in one cycle gives up the bounded main-thread decode that #1538 exists to keep. The cost is that while draining, the per-cycle metrics (resource usage, DB sizes, RocksDB stats) are written twice as often. One line to reverse.
4. A cycle's error still reaches the interval callback unhandled. Raised in three review rounds and deliberately left: base has the same unguarded await, the process-level handler logs rather than exits, and the cursor is not advanced on a throw, so the window is retried. It belongs with the durable-cursor work, not here.
5. runAggregationCycle is a new module export. The regression tests drive the real cycle rather than a helper, which is what makes them fail on the previous code; the cost is an exported function that is harder to take back later than an internal one. findLastAggregationTime was exported for the same reason in #1541.

Changes

resources/analytics/write.ts — the scan resumes from rawCursor rather than from the cadence marker, and the cursor is only assigned from a key the scan read. An empty scan leaves it alone, which matters because recordAnalytics does not await its primaryStore.put: a report can be uncommitted, and therefore invisible to the scan, while carrying a key below any clock reading the cycle could take. Cycles also run under a single-flight flag, since setInterval does not await its async callback and two cycles reading the same cursor roll the same window up twice — a real exposure now that catch-up keeps the cadence guard open.

unitTests/resources/analytics/aggregationCycle.test.js — three properties over the real cycle: an empty cycle does not pass a report seeded behind it, a backlog longer than one window drains across consecutive cycles, and a cycle entered while another runs does not re-aggregate the window.

DESIGN.md — the two cursor rules, the no-overlap rule, and the losses that remain open. docs/design/analytics-aggregation-resume-marker.md — the design note, its spanning set, and the planning- and review-round dispositions.

Verification

Core unit gate npm run test:unit:resources: 2879 passing, 46 pending, 3 failing — unitTests/resources/conditionDeleteVisibility.test.js (×2) and unitTests/resources/rangeReadActivity.test.js. All three fail identically with resources/analytics/write.ts reverted to its base revision in the same sandbox, whose borrowed node_modules carries rocksdb-js 2.8.0 where this checkout asks for 2.9.1.

Fails-on-base, each with a rebuild between swaps because the tests load dist:

• scan and cursor reverted to the single lastAggregationTime marker → all three tests fail;
• single-flight flag removed → refuses a cycle that starts while another is running fails with AggWindowConcurrent stored twice.

End to end: harper-pro's integrationTests/cluster/replicatedAnalyticsUnion.test.mjs against a dist built from this core, through the companion PR's CI. That test reproduces the defect at roughly one run in 25, so it corroborates rather than proves; the unit-level fails-on-base checks are the mechanism proof.

Complexity: medium

_{Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex; adjudicated=domain; declined=cursor-grok; rounds=6; full=2 @ 3c12595}

_{Human-Review-Need: 3 (decisions: in-memory-cursor, catch-up-bypasses-cadence, cold-start-seed, test-export) @ 3c12595}

[gemini-code-assist]

Code Review

This pull request addresses an issue where raw analytics records could be skipped during aggregation cycles by transitioning the aggregation marker from a wall-clock timestamp to a raw-record cursor (lastTime ?? cycleStart). It introduces a single-flight guard (runAggregationCycle) to prevent overlapping cycles, updates the scheduled tasks to use this guard, and adds comprehensive design documentation and unit tests to verify the backlog aggregation and concurrency prevention. There are no review comments to evaluate, and we have no additional feedback to provide.

[claude]

Reviewed; no blockers found.

[github-actions]

Release cherry-pick v5.2: conflict

Cherry-pick onto v5.2 produced conflicts on commit(s): 5dd2092442f79022337964ee209d3a2d674f92cd 823e5f0e895821b29789dd4307719af102dd045b

The conflict markers are committed on branch cherry-pick/v5.2/pr-2692.
A pull request has been opened to land this patch: #2987