Keep active RocksDB scans alive during eviction

[kriszyp]

Keep actively progressing RocksDB scans alive during TTL eviction and report a defined read-snapshot expiration when an abandoned scan resumes. Keep the restart transaction-log regression focused on filesystem reclamation under rocksdb-js 2.9.0.

For the human reviewer

Planning review adopted Gemini's monitor-consumed activity flag, keeping owner lookup and renewal-policy chain walks out of the per-row path. Final planning verdict: Framing-Verdict: chosen-approach-sound.

Look hardest at read ownership versus native commit ownership: a handle handed to the commit retry loop is still live, and retry handlers must be able to open fresh ranges on it. Existing range wrappers retain their original owner so they cannot silently switch snapshots. A named 503 asks callers to retry only the read; progressing reads still cannot extend pending write holders. These preserve the requester-approved snapshot and rollback policy.

Review rulings: retained the measured activity wrapper (+0.43% median in one local workload; V8 inlines its guard) and intentional lifecycle comments. Declined defensive cleanup suppression because the 2.9.0 native return path explicitly tolerates invalidated/already-closed iterators, also covered by expiry tests. A separate paced secondary-index HTTP case remains optional coverage: both adapters have real monitor-tick unit coverage, while the HTTP regression targets the failing base dump. The claimed WeakMap.get(undefined) crash is factually false (get returns undefined).

The PR bot’s wrapper suggestions are left for human adjudication: cleanup repeats the ruling above; detached next() calls were already unsupported by the native driver’s receiver-dependent iterator; and native return and throw invoke the same idempotent close macro, so return-then-rethrow preserves cleanup. This helper wraps those native iterators, not arbitrary generators with custom recovery. No post-review code change.

Release follow-up: the milestone was externally changed to v5.2 after this draft opened. The automated cherry-pick report records conflicts and says release integration did not run, although the automation job reports success. This task is scoped to main; release conflict resolution remains separate.

Changes

• Failure A: Harper usage. Native range progress did not renew the read-only idle budget, so the monitor aborted a snapshot while its HTTP scan was progressing. RocksDB 2.9.0 correctly binds ranges to transactions and exposed that lifetime bug; 2.8.0 ignored the supplied transaction. trackReadRange records activity without extending idle write holders, preserves commit retry handoffs, and raises ReadSnapshotExpiredError (503) before invalid native access. Poisoned write holders retain the existing 422 rollback behavior.
• Primary range adapter and secondary-index adapter wrap native ranges before their transformations, covering primary keys, secondary indexes, and entries later filtered out. Snapshot lifetime design records snapshot ownership, expiry, and read-only retry semantics. A local five-sample, million-row comparison measured +0.43% median wrapper overhead (overlapping timing ranges); V8 tracing confirmed guard inlining. This is one workload, not a general performance guarantee. No configuration or dependency changes; no separate public documentation is needed for this internal lifetime correction.
• Failure B: over-specified assertion. RocksDB 2.9.0 adds a recovery purge in TransactionLogStore::load; Harper's startup purge remains valid, and the counter includes no-op scans. The restart oracle permits at least one purge while requiring 100 MB of removed allocation and 75% reconciled filesystem reclaim. Its seed volume increases to keep more than 100 MB purgeable after rotation.

Verification

• New range lifetime regressions cover real monitor ticks, primary/index scans, filtered rows, snapshot isolation, read-your-writes, early cancellation, expired iterators, native conflict retry, and write-holder rollback: 16 passing.
• The HTTP regression and its paced resource fixture add a paced HTTP scan. On unmodified base bdd4f37 it fails with HTTP 500 / Next failed: Iterator not initialized; with this fix both target integration files pass all 5 tests. txnlog-restart-reclaim.test.ts removed 106,516,480 allocated bytes and reconciled 100% through statfs.
• Main unit gate: 5,463 passing, 196 skipped after removing inherited GIT_CONFIG_GLOBAL/GIT_PAGER from the test environment. RocksDB resource gate: 2,352 passing, 31 skipped. LMDB resource gate: 1,833 passing, 370 skipped.
• Build, changed-file formatting and lint pass. Repository lint has 13 existing warnings across 12 files byte-identical to base; no changed-file warnings.
• Requested integration shard 2/6: 268 passing, zero failures/cancellations. Shard 6/6: 266 passing, one skipped, zero failures/cancellations.
• Full integration gate: 2,067 passing, 20 skipped, 6 cancelled in the optional Ollama suite because its source-module setup imports JSON without the required attribute on Node 26. The same hook failure reproduces on unmodified base bdd4f37; all database integrations, including both target files, pass. The gate exits nonzero because of that baseline setup failure.

GitHub CI at 03:50 UTC: 41 passed, 6 skipped, 2 failed. All builds, unit gates (Node 22/24/26 and Windows), and all 24 integration shards are green. CI logs confirm the new paced HTTP scan and original Q1 execute successfully on Node 24, Bun, and uWS; reclaim executes successfully on Node 24 and uWS (intentionally guarded out on Bun/Windows). The two Next.js adapter jobs fail before tests at npm ci: the existing lockfile omits @harperfast/rocksdb-js-linux-x64-musl@2.9.0. Dependency repair is explicitly outside this task; this PR changes neither dependency manifest.

Refs #2506, #2337, #2549.

Reviewer substitution: ran=codex,gemini; declined/blocked=claude-quota. Codex authored and adjudicated; Gemini supplied the independent planning/code review. The Gemini code adapter needed a local supplied-diff-only prompt prefix after its first attempt requested a denied command; permissions were not expanded.

Co-Authored-By: GPT-5 Codex noreply@openai.com

Complexity: complicated

_{Review-Coverage: authored=codex; ran=gemini; adjudicated=domain; declined=claude,cursor-grok,cursor-composer; rounds=2 @ 881855c}

_{Human-Review-Need: 3 (decisions: iterator-wrapper-boundary, activity-sampling-model, write-holder-liveness, expired-snapshot-contract, startup-purge-oracle) @ 881855c}

[gemini-code-assist]

Code Review

This pull request introduces tracking for RocksDB range read activity and snapshot expiration. It wraps range iterators in PrimaryRocksDatabase and RocksIndexStore using a new trackReadRange helper in DatabaseTransaction, ensuring active range scans renew read timeouts and throwing a ReadSnapshotExpiredError if a scan resumes after its snapshot is released. The review feedback suggests robust improvements to the trackReadRange iterator wrapper, specifically addressing potential error masking during cleanup, this context binding issues, and ensuring delegation to the underlying iterator's throw method.

[github-actions]

Release cherry-pick v5.2: conflict

Cherry-pick onto v5.2 produced conflicts on commit(s): b349f8ec88769860467fb7404b63da7429807d55

The conflict markers are committed on branch cherry-pick/v5.2/pr-2556.
A pull request has been opened to land this patch: #2559

[claude]

Reviewed; no blockers found.