Skip to content

Document table_changes Iceberg table function #21406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

findinpath
Copy link
Contributor

@findinpath findinpath commented Apr 4, 2024

Description

Documentation follow-up from #15677

Additional context and related issues

Fixes #21227

Iceberg Spark counterpart https://iceberg.apache.org/docs/latest/spark-procedures/#create_changelog_view

Delta Lake table_changes table function documenation https://trino.io/docs/current/connector/delta-lake.html#table-changes

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@cla-bot cla-bot bot added the cla-signed label Apr 4, 2024
@findinpath findinpath self-assigned this Apr 4, 2024
@findinpath findinpath added the iceberg Iceberg connector label Apr 4, 2024
@github-actions github-actions bot added docs and removed iceberg Iceberg connector labels Apr 4, 2024
@findinpath findinpath added the iceberg Iceberg connector label Apr 4, 2024
FROM
TABLE(
system.table_changes(
"SCHEMA" => 'default',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCHEMA and TABLE are reserved words, so we'll have to quote them to avoid unwanted sql syntax issues.

cc @martint

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats kinda bad .. why did those parameters not use the same approach as in https://trino.io/docs/current/connector/iceberg.html#migrate-table

so schema_name
and table_name

and also stick with lower case parameter names

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we maybe fix the implementation ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#21698 fixes the issue.

Thank you @ebyhr

Reviving this PR

@findinpath findinpath force-pushed the findinpath/iceberg-table-changes-docs branch from 9db2912 to 22bd647 Compare April 4, 2024 21:44
@findinpath findinpath force-pushed the findinpath/iceberg-table-changes-docs branch 3 times, most recently from 838a6f5 to 983b48f Compare April 26, 2024 15:15
@findinpath findinpath requested a review from ebyhr April 26, 2024 15:16
VALUES
('url1', 'domain1', 1),
('url2', 'domain2', 2),
('url3', 'domain1', 3);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the domain values here and below different on purpose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the page_url & domain are supposed to be the primary key so to say of the table that can be used in downstream processes to identify the history of changes performed on a tuple.

Note that the same example slightly adapted is employed in Trino Delta Lake connector as well.

@findinpath findinpath force-pushed the findinpath/iceberg-table-changes-docs branch from 983b48f to db0edd1 Compare April 29, 2024 12:26
@findinpath findinpath requested a review from mosabua April 29, 2024 12:26
Copy link
Member

@mosabua mosabua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now. Ideally add the sql language identifier to all the query code snippets .. also not sure if you want approval from another maintainer.

@ebyhr ebyhr merged commit 7b7993f into trinodb:master Apr 29, 2024
@github-actions github-actions bot added this to the 446 milestone Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

Document table_changes table function in Iceberg connector
3 participants