Skip to content

fix(iceberg,backup): complete #639 items 2, 5, 7, 8 - #693

Merged
xe-nvdk merged 1 commit into
mainfrom
fix/639-sweep-completion
Sep 2, 2026
Merged

xe-nvdk merged 1 commit into
mainfrom
fix/639-sweep-completion

Conversation

@xe-nvdk

@xe-nvdk xe-nvdk commented Sep 2, 2026

Copy link
Copy Markdown
Member

Summary

Part 1 of finishing the #639 audit sweep (item 3, catalog cleanup on database drop, follows separately as the one destructive-path change):

  • item 2: a reconciler gated off on a node (no compactor role) warns ONCE at first occurrence with actionable text; previously only a per-tick debug line, so a cluster misconfiguration meant Iceberg export silently never ran
  • item 5: warehouseRelKey resolves symlinks before comparing, symmetrically across the metadata location, warehouse, and storage root (constant sides cached; parent-dir fallback for not-yet-written locations; s3/azure schemes bypass untouched), so a symlinked storage root no longer lands every commit in warn-once hint-unaddressable mode
  • item 7: after each commit, the freshly committed table's local metadata tree is tightened to owner-only permission bits (iceberg-go writes at umask); TIGHTEN-ONLY, group/other bits are cleared and owner bits are never added, so deliberately restrictive modes stay (the review of the first attempt caught it un-breaking a test's read-only directory)
  • item 8: restore staging streams from backup storage via ReadTo instead of buffering multi-GB databases in memory twice

Process

Batched adversarial plan review before implementation (which set the tighten-only requirement's groundwork, the symmetric-resolution shape, and confirmed the chmod walk is bounded by the retain cap); unit tests per item.

Verification

  • new tests: symlinked-warehouse resolution, tighten-only permission semantics (umask-wide file hardened, 0400 file untouched), once-per-process gated warning over repeated passes
  • item 8 is exercised by the existing staging tests through the streaming path
  • full iceberg/backup/api suites pass

Refs #639

Item 2: once-per-process WARN when the reconciler is gated off on a
node, naming the no-compactor-role failure mode. Item 5: warehouseRelKey
resolves symlinks symmetrically (cached constant sides, dir fallback for
not-yet-written locations, remote schemes untouched). Item 7: after each
commit, tighten local warehouse metadata written by iceberg-go to
owner-only bits; tighten-only, restrictive modes stay. Item 8: restore
staging streams via ReadTo instead of double-buffering the database.

Refs #639
@xe-nvdk
xe-nvdk merged commit 8bfb9bb into main Sep 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant