always retry syncing index files from indexshipper when the index list cache is stale #6327
Conversation
|
./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell. + ingester 0%
+ distributor 0%
+ querier 0%
+ querier/queryrange 0%
+ iter 0%
+ storage 0%
+ chunkenc 0%
+ logql 0%
+ loki 0% |
|
./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell. + ingester 0%
+ distributor 0%
+ querier 0%
+ querier/queryrange 0%
+ iter 0%
+ storage 0%
+ chunkenc 0%
+ logql 0%
+ loki 0% |
| // syncWithRetry runs a sync with upto maxSyncRetries on failure | ||
| func (t *indexSet) syncWithRetry(ctx context.Context, lock, bypassListCache bool) error { | ||
| var err error | ||
| for i := 0; i <= maxSyncRetries; i++ { |
There was a problem hiding this comment.
double checking: the PR title says that it will retry syncing files when the index list cache is stale, but on this implementation you are retrying regardless of the error (although you're only refreshing when the cache is stale). Is that the intended behavior?
What this PR does / why we need it:
From
indexshipper, we detect and retry failed sync due to a stale index list cache. However, it is not done in all cases like query readiness or syncing local files on startup, which causes some unexpected problems. This PR updates the code to always refresh the stale list cache and retry sync to keep it consistent.To keep the code simple, I have made the code retry sync in all failure cases and not just the stale index list cache.
Checklist