Organize WAL logs of all indexes into one file #210
Labels
enhancement
New feature or request
performance
All issues related to Tatris performance, including reporting, optimization, and regressions.
Describe This Problem
Now Tatris maintains a WAL file for each shard for the sake of clear classification, but this brings the I/O resource consumption of the same order of magnitude as the number of shards, the waste seems particularly unreasonable especially when most of shards only have sporadic write traffic.
Proposal
These scattered WAL logs can be organized into the same file, as long as we can distinguish the index and shard it belongs to for each line of log.
Additional Context
Please consider this issue together with issue #209 : when multiple shards' WAL logs are converged to a single file, maintaining a
POINT
for truncating expired logs seems to become more complicated, since you need to know exactly how many logs are consumed by all indexes in a certain period of time.The text was updated successfully, but these errors were encountered: