[ENH][wal3] Make GC be delete-free for log/gc/GARBAGE

[rescrv]

Description of changes

AWS doesn't support delete of things with an If-Match header except in
directory buckets and we are not using those. Instead overwrite using
the same hand-over-hand pattern as the manifest. Load a GC, decide
what to do while holding on the the ETag, and then overwrite the garbage
with a sentinel rather than deleting it.

Test plan

CI

• [wal3] Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

N/A

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[propel-code-bot]

Refactor GC to Overwrite 'log/gc/GARBAGE' Instead of Deleting (AWS S3 Compatibility)

This PR updates the WAL3 garbage collection logic to accommodate AWS S3's lack of support for conditional deletes (i.e., delete with If-Match header) outside of directory buckets. Instead of deleting the 'log/gc/GARBAGE' object, the implementation now uses a 'sentinel' overwrite pattern: the file is overwritten (using ETag for concurrency safety) with an empty/sentinel value after processing, similar to the manifest update mechanism. The change adds a recursive GC mechanism and enhances the handling of garbage object state transitions for durability and correctness.

Key Changes:
• Introduced a sentinel overwrite/reset mechanism instead of delete for 'log/gc/GARBAGE', using ETags and safe atomic update patterns.
• Refactored garbage collection in rust/wal3/src/writer.rs to include a recursive helper (garbage_collect_recursive), handling contention and concurrent updates.
• Expanded the Garbage struct and implementation (rust/wal3/src/gc.rs): now supports empty/sentinel state, equality, and safe overwrite semantics.
• Added Garbage::reset and a generalized transition method to atomically replace the garbage file, reusing logic for manifest updates.
• Altered file handling so object deletion is replaced by an update to an empty 'GARBAGE' object, improving resilience to S3 semantics.

Affected Areas:
• rust/wal3/src/writer.rs (garbage collection logic)
• rust/wal3/src/gc.rs (Garbage struct, file I/O logic, atomic garbage overwrite)

This summary was automatically generated by @propel-code-bot