Skip to content

feat: Add iceberg connector + Tests + Docs#8902

Open
royendo wants to merge 10 commits intomainfrom
add-iceberg-connector
Open

feat: Add iceberg connector + Tests + Docs#8902
royendo wants to merge 10 commits intomainfrom
add-iceberg-connector

Conversation

@royendo
Copy link
Contributor

@royendo royendo commented Feb 21, 2026

Adds Apache Iceberg as a new connector, leveraging DuckDB's native Iceberg support. Tested with local filesystem and GCS catalogs.

Backend:

  • Register `iceberg` as a DuckDB source connector in `runtime/drivers/duckdb/duckdb.go`
  • Add Iceberg connector schema to `runtime/parser/schema/project.schema.yaml` with catalog type, storage type, and credential properties

Frontend:

  • Multi-step form schema (`iceberg.ts`) for configuring Iceberg connections (catalog URI, storage type, credentials)
  • Apache Iceberg logo SVG icons and connector icon mapping
  • Dynamic field rendering in `JSONSchemaFormRenderer` and `GroupedFieldsRenderer` for grouped/conditional form fields
  • GCS and Azure Blob Storage icon components

Documentation:

  • User guide at `docs/developers/build/connectors/data-source/iceberg.md`
  • Reference docs for Iceberg connector YAML properties

Integration tests:

  • Resolver test (`connector_iceberg.yaml`) validating Iceberg → DuckDB materialization with TPC-H lineitem data (51,793 rows, 16 columns)
Screenshot 2026-02-20 at 20 59 40 Screenshot 2026-02-20 at 20 59 43 Screenshot 2026-02-20 at 21 00 00

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

@royendo royendo requested a review from mindspank February 21, 2026 02:00
@royendo
Copy link
Contributor Author

royendo commented Feb 21, 2026

Make changes:

Tab 1: Direct Table Path
• Storage dropdown
• Storage connector
• Table path input
• Done

Tab 2: Managed Table (Catalog)
• Catalog dropdown (REST, PG, MySQL)
• Catalog connector
• Storage dropdown
• Storage connector
• Table identifier input (db.table)
• Done

Results look correct:

51,793 rows — matches TPC-H lineitem SF0.01 with the l_extendedprice < 10000 rows deleted (per the README, full SF0.01 has ~60K rows)
16 columns — standard TPC-H lineitem schema (l_orderkey, l_partkey, ... l_comment)
Types are reasonable — INTEGER, DECIMAL(15,2), VARCHAR, DATE
Both models (with and without secrets) return the same 51,793 count
@royendo royendo changed the title feat: Add iceberg connector feat: Add iceberg connector + Tests Feb 23, 2026
@royendo royendo changed the title feat: Add iceberg connector + Tests feat: Add iceberg connector + Tests + Docs Feb 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant