Skip to content

Support hierarchical namespaces in Datafusion integration #1976

@eickler

Description

@eickler

Is your feature request related to a problem or challenge?

Currently, while Iceberg and iceberg-rust support hierarchical namespaces, the Datafusion integration in iceberg-rust only makes tables in top-level namespaces available. My use case includes tenancy and fairly complex schemas with thousands of tables, so our lake is quite much nested.

Describe the solution you'd like

A fairly simple addition to the IcebergCatalogProvider can make deeply nested tables using a dot notation (i.e., the namespace is presented as parent.child.… and is available for querying using select … from catalog."parent.child".table). This is apparently also how the same use case is solved in Datafusion itself currently.

Willingness to contribute

I can contribute to this feature independently

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions