EACL cache: consistency-aware recursive, lookup, and count acceleration

[theronic]

Summary

Follow-up to #79 that removes repeated-prefix recursive traversal and adds bounded authorization caching without adding permanent consumer storage.

• Recursive pagination resumes bounded traversal continuations and caches already produced pages.
• Pending relationship-index scans retain scalar descriptors plus at most 64 internal EIDs, never a Datomic DB value or lazy index sequence.
• Missing continuations replay deterministically only while the cursor's complete schema and relationship proof still matches. After a relevant relationship change, EACL returns an already retained exact page or fails closed.
• Recursive state is keyed by schema plus relationship proof rather than general basis-t, so unrelated application transactions do not make an identical walk cold.
• One typed store also supports opt-in live can?, lookup, and count results, plus cache-resident exact snapshots. Keys and result identities use internal EIDs.
• No new lookup API variants, Datomic schema attributes, derived-grant tuples, cache markers, pages, or continuations are written to the consumer database.

Adversarial review fixes

The performance review found and fixed several issues that were hidden by the original single-size benchmark:

1. Local-store expiry scanned every entry on every cache operation. TTL is now checked only for the requested key; stale physical entries remain globally capacity-bounded.
2. Continuations could retain lazy Datomic index streams. Index work is now represented by bounded, resumable chunks.
3. Continuation admission walked the queued frontier to estimate weight on every page. Weight is now a constant-time conservative estimate using maintained counters.
4. Recursive pages were still namespaced by unrelated Datomic basis churn. They now use the relationship proof, while public exact-result entries remain keyed by transaction t.
5. Cursor retry, eviction, rejected admission, provider misses, alternate stores, deleted boundary entities, and relevant-write races now have explicit tests and fail/recompute semantics.
6. Completed-result keys canonicalize once, and hot can? keys use compact internal scalar identities.

Coherence and consistency

Relationship helpers hold the configured coordinator barrier through transaction completion and publish the exact committed Datomic t only for relation definitions whose tuples changed. Unrelated application transactions, no-op writes, and writes outside a permission's dependency set preserve eligible entries.

Every cache-enabled client has a local coordinator for its own cursor proofs. Cross-client live memoization remains opt-in and requires one explicitly shared coordinator for every participating reader and relationship writer. Direct transactions of EACL-owned relationship or schema data are outside the coherence contract and are not polled.

fully-consistent uses the DB visible to the local Peer without forcing zero-argument d/sync. at-least-as-fresh may perform only the targeted wait requested by its token. at-exact-snapshot is cache-resident and never invokes d/as-of.

Performance

The revised benchmark measures actual v7.3 (2ccc6c4) in a separate JVM, cache-disabled replay, continuation advancement, completed-page hits, deterministic graph work, five graph sizes, and five page sizes.

At 4,000 recursive results with page size 25:

workload	median
v7.3	1,562.04 ms
candidate, cache disabled	1,351.92 ms
continuation walk	45.75 ms
repeated completed-page walk	23.30 ms

Continuations perform 77.46× less graph work, make the first walk 34.1× faster than v7.3, and make a repeated walk 67.0× faster. The continuation path stays below 1.04N derived grants; replay grows approximately with N²/page-size.

On a 2,000-result graph:

page size	pages	replay	continuation	completed pages
10	200	884.65 ms	40.48 ms	27.82 ms
25	80	347.23 ms	22.49 ms	12.02 ms
100	20	90.69 ms	12.88 ms	3.94 ms
250	8	39.58 ms	10.97 ms	2.26 ms
1,000	2	13.25 ms	10.02 ms	1.46 ms

The remaining recursive cost is linear cold traversal plus work proportional to returned results and public page count. Profiling 1,000 results over 40 pages measured about 5.9 ms in the continuation engine, 0.5 ms in cache I/O, and 3.7 ms in cursor encryption/decryption; response coercion overlaps most cursor encryption.

The same warmed heavy run measured:

• can?: 12.00 µs default, 3.88 µs completed live hit;
• 15,000-result count: 24.48 ms cold, 0.008 ms live hit;
• 15,000-resource first lookup page: 1.54 ms median;
• cache-disabled 20,000-result count: at most 16,384 EIDs retained at once.

Verification

All Clojure tests were run through nREPL.

• Fresh-JVM regular suite: 149 tests, 4,427 assertions, 0 failures, 0 errors
• Fresh-JVM heavy benchmark/load suite: 7 tests, 3,301 assertions, 0 failures, 0 errors
• Deterministic cached-vs-disabled mutation model: green
• Forward and reverse traversal across 64-EID chunk boundaries: green
• Strict OpenSpec validation: green; 89/89 tasks complete
• No persistent Datomic storage or schema additions

Release

Implemented as the v7.4 candidate while remaining suitable for consideration as a v7.3 patch because v7.3 is recent. This PR remains intentionally stacked on #79.