Skip to content

draft: feat: add Delta Lake connector#8916

Draft
royendo wants to merge 8 commits intomainfrom
royendo/add-delta-connector
Draft

draft: feat: add Delta Lake connector#8916
royendo wants to merge 8 commits intomainfrom
royendo/add-delta-connector

Conversation

@royendo
Copy link
Contributor

@royendo royendo commented Feb 23, 2026

Adds a Delta Lake connector that lets users query Delta tables stored on S3, Azure Blob Storage, or local filesystem, powered by DuckDB's delta extension.

  • Backend: installs and loads the delta DuckDB extension at startup; adds project schema validation for Delta sources
  • Frontend: multi-step form schema with storage type selector (S3 / Azure / local), conditional field visibility via x-grouped-fields, and disabled-option enforcement when cloud connectors are missing
  • Source rewriting: maybeRewriteToDuckDb() normalizes Delta form data into a DuckDB model with delta_scan() SQL, bridging cloud credentials via create_secrets_from_connectors
  • Icons: Delta Lake logo and icon Svelte components
  • Docs: new Delta Lake connector page with storage backend matrix, UI walkthrough, and manual YAML configuration examples

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

Developed in collaboration with Claude Code

@royendo royendo changed the title draft: feat/add Delta Lake connector draft: feat: add Delta Lake connector Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant