Skip to content

Update Iceberg catalog docs for Glue example #815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 27, 2025
+11 −0
Merged

Update Iceberg catalog docs for Glue example #815

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fb7d1b0
Update Iceberg catalog docs for Glue example
phillipleblanc Jan 27, 2025
0ea0150
clarify
phillipleblanc Jan 27, 2025
0764a51
Merge branch 'trunk' into phillip/250127-iceberg-glue
ewgenius Jan 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions website/docs/components/catalogs/iceberg.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,20 @@ catalogs:
iceberg_s3_role_arn: arn:aws:iam::123456789012:role/my-role # Optional. ARN of the IAM role to assume when accessing the S3-compatible endpoint.
iceberg_s3_role_session_name: my-session # Optional. Session name to use when assuming the IAM role.
iceberg_s3_connect_timeout: 60 # Optional. Connection timeout for the S3-compatible endpoint (default: 60).

# AWS Glue Catalog
- from: iceberg:https://glue.us-east-1.amazonaws.com/iceberg/v1/catalogs/123456789012/namespaces
name: glue
params:
iceberg_sigv4_enabled: true
```

## `from`

The `from` field is used to specify the catalog provider. For Iceberg, use `iceberg:<namespace_path>`. The `namespace_path` is the URL to the Iceberg namespace in the catalog provider to load the tables from. It is formatted as `http[s]://<iceberg_catalog_host>/v1/{prefix}/namespaces/<namespace_name>`.

For AWS Glue catalogs, the URL format is `https://glue.<region>.amazonaws.com/iceberg/v1/catalogs/<account_id>/namespaces`, where `<account_id>` is your AWS account ID.

The selected namespace must have sub-namespaces where the tables are stored.

Example: With this Iceberg catalog structure:
Expand Down Expand Up @@ -98,6 +106,9 @@ The following parameters are supported for configuring the connection to the Ice
| `iceberg_s3_role_arn` | ARN of the IAM role to assume when accessing the S3-compatible endpoint. |
| `iceberg_s3_role_session_name` | Session name to use when assuming the IAM role. |
| `iceberg_s3_connect_timeout` | Connection timeout in seconds for the S3-compatible endpoint. Default: `60` |
| `iceberg_sigv4_enabled` | Enable SigV4 (AWS Signature Version 4) authentication when connecting to the catalog. Automatically enabled if the URL in `from` is an AWS Glue catalog. Default: `false` |
| `iceberg_signing_region` | Region to use for SigV4 authentication. Extracted from the URL in `from` if not specified. |
| `iceberg_signing_name` | Service name to use for SigV4 authentication. Default: `glue`. |

## Cookbook

Expand Down
Loading