feat(index): add leased whole-set GC execution and canonical SQLite authority#154
Merged
Conversation
…uthority Add library-owned whole-set authority, mutable canonical SQLite binding with connection-owned WAL/journal/SHM reservation, exact-epoch capture retention, and leased whole-set GC execution. Wire CLI/build/list/doctor paths and document the authority and recovery contract. Changes: - Add indexcoord lease/authority and set maintenance surfaces - Add OpenLocalMutableCanonical VFS with exclusive sidecar reservation - Retain discoverable quarantine residue without path-unlink reclaim - Add GC execute path and related CLI/docs/tests Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
CI on the durable-set GC PR failed two independent gates without changing the exact-epoch policy: Windows reserved-absent interposition left the empty CREATE_NEW reservation behind, and a Linux restore-over-live fixture truncated planted capture content when remove+recreate recycled the inode. Changes: - Close the write-target binding before SameFile-gated reservation removal so Windows can abandon an untouched empty epoch after denied interposition - Plant exact-epoch mismatch substitutes via rename of a pre-created peer file so the fixture cannot recycle the live inode number on Linux tmpfs Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
Hosted Windows jobregistry failed every durable whole-set GC execution test with "GC plan changed before execution" after acquire, while Linux and planning-only coverage stayed green. Held durable write leases were keyed by the planned target path, but revalidation looked up Dir(latest.json) which can use a different Windows path form; the miss fell through to an exclusive availability probe against our own LockFileEx, blocked the set, and rewrote the plan digest. Changes: - Key held durable leases by WriteLease.SegmentSetRoot (Abs+Clean) - Look up held leases via Abs/identity-tolerant helper before availability probe - Assert held scope against the lease canonical root - Soften Unix mode-bit assertions in the dry-run receipt test on Windows - Add lookup helper unit coverage Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
Windows leased GC execution still failed after held-lease path lookup with "GC plan changed before execution" for every durable whole-set run. Plan revalidation re-hashes segment-set trees while AcquireWriteLeaseForMaintenance holds an exclusive LockFileEx on .durable-write.lock; content-opening that artifact under the held lock is unreliable on Windows and the lock payload is documented diagnostic-only, not published index authority. Changes: - Contribute write-lease files to GC tree digests as name/mode/size + lease-meta without opening content (plan hash, quarantine snapshot, remainder validation) - Export WriteLeaseFileName for the shared special-case - Enrich plan-mismatch errors with candidate/warning counts for future triage - Add regression: pre-acquire plan SHA equals plan under held maintenance lease Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
… rename After plan revalidation stabilized, Windows leased GC failed at mutation with "quarantine GC target: renameat ... Access is denied." The exclusive write-lease FD pins .durable-write.lock inside the segment-set directory, and Windows cannot rename a directory while that handle remains open. Changes: - Release the matching durable write lease immediately after final pre-mutation tree revalidation and before quarantine rename of segment-set targets - Keep set maintenance authority held for exclusion during the rename window Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
Devrev P1 on PR #154: isIndexGCWriteLeaseRel matched any basename .durable-write.lock, so nested same-named files and root-level same-named files under identity/journals targets skipped content binding. Same-size substitution preserved the immutable plan digest and synthetic lease-meta remainder proof, violating exact-artifact deletion authority. Changes: - Apply lease-meta only when target kind is segment-set and rel is exactly the package root lock name - Content-open/hash nested and non-segment same-basename artifacts - Add regressions for nested segment-set and journals/identity root cases Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
Windows amd64 jobregistry suite flaked on PR CI while arm64 and the leased GC suite stayed green: waitHelperCompletion required !isProcessAlive after the helper already persisted JobStateSuccess+EndedAt. On busy Windows runners PID reuse can keep that condition false for the full timeout. Changes: - Treat Success with EndedAt as terminal for helper completion waits - Fall back to process liveness only when EndedAt is missing - Slightly widen poll interval/timeout for wait helpers Generated by Grok 4.5 via Grok Build under supervision of @3leapsdave Co-Authored-By: Grok 4.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: Dave Thompson <dave.thompson@3leaps.net> [@3leapsdave]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pkg/indexcoord) and leased whole-set GC execution for durable index sets.