Skip to content

feat(iceberg): clear catalog artifacts on database drop - #694

Merged
xe-nvdk merged 1 commit into
mainfrom
fix/639-item3-catalog-cleanup
Sep 2, 2026
Merged

xe-nvdk merged 1 commit into
mainfrom
fix/639-item3-catalog-cleanup

Conversation

@xe-nvdk

@xe-nvdk xe-nvdk commented Sep 2, 2026

Copy link
Copy Markdown
Member

Summary

  • completes low(iceberg): audit sweep — interval validation, silent cluster no-run, dropped-DB residue, per-pass hygiene #639 (item 3, the final open item): DELETE /api/v1/databases/{name} now also removes the database's Iceberg catalog tables, its namespace, and its warehouse metadata directory, previously all of it persisted forever
  • ordering is files-first (the existing delete flow) so a racing reconcile pass can only empty tables, never resurrect them via EnsureTable; catalog-first was explicitly rejected in plan review for that reason
  • the SQL catalog's DropNamespace refuses non-empty namespaces, so tables are dropped individually first; ErrNoSuchNamespace reads as success (database never exported), making the operation idempotent
  • cleanup also runs best-effort on the 404 path, so re-running the DELETE after a crash between file deletion and catalog cleanup converges, without changing the API's not-found contract
  • warehouse directory sweep is deepest-first (RemoveDirectory is non-recursive, which is the very behavior the issue documents); spoke pseudo-databases are refused (their lifecycle belongs to edge-sync, and the review confirmed a bare spoke-ID delete hits a nonexistent namespace harmlessly)
  • nil-safe wiring: deployments without Iceberg export are untouched

Verification

  • exporter test with a real SQL catalog: reconcile creates the table, drop empties iceberg_tables and namespace properties and removes the warehouse namespace tree, second drop and never-exported drop are no-ops, namespaced names refused
  • full iceberg/api suites pass

Refs #693 (items 2/5/7/8). Closes #639.

Completes #639 (item 3, the last open item). Exporter.DropDatabase drops
each table in the database's namespace (the SQL catalog's DropNamespace
refuses non-empty namespaces), the namespace itself (ErrNoSuchNamespace
tolerated for never-exported databases), and the warehouse metadata
tree, sweeping the now-empty directories deepest-first since
RemoveDirectory is non-recursive. Wired into the database-delete API
after file deletion, files-first so a racing reconcile can only empty
tables, and also best-effort on the not-found path so re-running the
DELETE converges after a crash. Spoke pseudo-databases are refused.

Closes #639
@xe-nvdk
xe-nvdk merged commit 77e2193 into main Sep 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

low(iceberg): audit sweep — interval validation, silent cluster no-run, dropped-DB residue, per-pass hygiene

1 participant