Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Bifrost] Read stream support multi-segment logs
This introduces a new read stream implementation that operates under a multi-segment bifrost world. Notable features include: - Support for reading from multiple segments seamlessly - Reading unsealed segments while watching the tail state to determine the safe boundaries with minimal efficiency loss - Handling of on-going reconfiguration, the stream waits for the loglet to be sealed. - Handles prefix trims on metadata-level when detected (partial support, more on that in follow up PRs) Running bifrost-benchpress read-to-write latency tests show that the new read-stream doesn't introduce any meaningful regression in latency in the the unsealed close-to-tail case (note that P100 should be discarded due to shutdown-related noise) Write-to-read latency: ``` New Old Total records read: 98317 Total records read: 97871 P50: 67.455µs P50: 67.519µs P90: 77.951µs P90: 77.183µs P99: 96.447µs P99: 94.143µs P999: 129.215µs P999: 122.815µs ```
- Loading branch information