[Enhance]: Buffer Storage

[iaojnh]

Affected Component

buffer manager, indexes who use buffer storage.

Current Behavior

Current buffer storage performance is poor, limited by excessive concurrency lock overhead.
Lack of write capability.

Desired Improvement

Optimize the LRU implementation using a data structure with strong concurrency capabilities to improve performance.Minimize the overhead of locking.
Add write capability.

Impact

faster queries.

[SYaoJun]

Hello @iaojnh, I'd like to know how to measure the efficiency of a new data structure. Are there any mature tools or benchmarks available to evaluate a new implementation?

[iaojnh]

Hello @iaojnh, I'd like to know how to measure the efficiency of a new data structure. Are there any mature tools or benchmarks available to evaluate a new implementation?

For our indexes, there are some recall or bench tools located in "tools/core/". You can read README.md for detail.
For external tools, you can learn about ann-benchmark or VectorDBBench.