Support for Iceberg Tables in Snowflake with AWS Glue Catalog Integration

[amirbtb]

Describe the feature

Support for Iceberg tables in Snowflake with AWS Glue catalog integration.

Describe alternatives you've considered

Currently, there is no existing alternative that would allow us to directly reference Iceberg tables (managed by catalog integration with AWS Glue catalog) as a dbt (external) source. We would need to create tables in Snowflake that reference the Iceberg tables and then we could use them as dbt sources.

Additional context

• Database-specific: The feature is specific to Snowflake. It could be extended to BigQuery as I heard it also supports Iceberg via BigLake but I'm not very familiar with the latter.
• Context: Our organization uses AWS Glue catalog for cataloging and S3 for storage.

Who will this benefit?

This feature would be useful for anyone :

• Using Iceberg with Snowflake
• Using AWS Glue catalog integration

[amirbtb]

I found this comment, but I'm not sure if this feature is supported and if that works with AWS Glue catalog integration :

Note, the way that #138 is implemented, option 1 via "external tables" should already work.

    - name: iceberg_tbl
         description: "External table using Iceberg files"
         external:
           location: "@stage"                      # reference an existing external stage
           file_format: "( type = parquet )"       # fully specified here, or reference an existing file format
           table_format: iceberg                   # specify the table format
           auto_refresh: false                     # requires configuring an event notification from Amazon S3
           

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[dataders]

@amirbtb do you still have this use case? Have you seen the new pattern of EXTERNAL TABLE + ICEBERG TABLE compared to this packages use of EXTERNAL STAGE + EXTERNAL TABLE?