Add access cache to support efficient lookup #17697
Description
Task Description
What needs to be done:
- Add the basic cache layer abstraction
- Introduce hotspot entries cache: cache A
- Introduce inflight entries cache(for uncommitted checkpoint): cache B
- Introce smart eviction strategy for cache A & B
- support point queries with optional mini-batch access(one access with a set of keys)
- add tests
Why this task is needed:
ms level point query access is required for streaming, otherwise it's very prone to trigger backpressure then volatility of the pipeline.
Task Type
Code improvement/refactoring
Related Issues
Parent feature issue: (if applicable )
#17647