-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[ENH] If the dirty log fails with LogContentionDurable, do not fail the operation. #4953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Handle LogContentionDurable Errors Gracefully in Dirty Log Operations This PR modifies both the log-service and the wal3 implementation so that operations which would fail due to a LogContentionDurable error when appending to the dirty log now succeed instead, treating this error as a non-fatal, recoverable state. This improves reliability during high garbage collection rates by preventing spurious failures when the log is already in a durable state. Key Changes: Affected Areas: This summary was automatically generated by @propel-code-bot |
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
…he operation. Both roll_dirty_log and mark_dirty append to the dirty log. At high gc rates, this can fail, and the operation will fail. We can simply return Ok(()) for any case where the log is durable.
6a7c800
to
69c7963
Compare
…he operation. (#4953) ## Description of changes Both roll_dirty_log and mark_dirty append to the dirty log. At high gc rates, this can fail, and the operation will fail. We can simply return Ok(()) for any case where the log is durable. ## Test plan CI ## Documentation Changes N/A
## Description of changes Included changes - **[ENH] Purge dirty log in background at the end of scheduled compaction (#4915)** - **[ENH] Move Log GC to operator (#4919)** - **[BUG] Do not leak tokio tasks in the log service. (#4936)** - **[BUG] Log GC offset should be one above minimum compaction offset (#4938)** - **[ENH] Make roll dirty log always converge to coalesce everything. (#4927)** - **[BUG] Coalesce when multiple collections return the same info to compact (#4946)** - **[BUG] Enrich from the manifest if a cursor doesn't exist. (#4947)** - **[ENH] If the dirty log fails with LogContentionDurable, do not fail the operation. (#4953)** - **[ENH] Warn, not error, if dirty log has no cursor. (#4952)** - **[ENH] Cancellation safety for append_batch. (#4959)** ## Test plan CI ## Documentation Changes N/A --------- Co-authored-by: Macronova <60079945+Sicheng-Pan@users.noreply.github.com>
…he operation. (chroma-core#4953) ## Description of changes Both roll_dirty_log and mark_dirty append to the dirty log. At high gc rates, this can fail, and the operation will fail. We can simply return Ok(()) for any case where the log is durable. ## Test plan CI ## Documentation Changes N/A
Description of changes
Both roll_dirty_log and mark_dirty append to the dirty log. At high gc
rates, this can fail, and the operation will fail. We can simply return
Ok(()) for any case where the log is durable.
Test plan
CI
Documentation Changes
N/A