Skip to content

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Dec 21, 2024

Description

The 1st commit allows properties after the target table name in DML:

    | INSERT INTO qualifiedName ('@' branch=string)?
       columnAliases? rootQuery                                        #insertInto
    | DELETE FROM qualifiedName ('@' branch=string)?
         (WHERE booleanExpression)?                                    #delete
    | UPDATE qualifiedName ('@' branch=string)?
        SET updateAssignment (',' updateAssignment)*
        (WHERE where=booleanExpression)?                               #update
    | MERGE INTO qualifiedName (AS? identifier)? ('@' branch=string)?
        USING relation ON expression mergeCase+                        #merge

Avoided session property approach because users may forget the current branch and modify unexpected branch.

The 2nd commit introduces branching support and adds the following procedures in Iceberg connector:

  • ALTER TABLE table EXECUTE create_branch(name)
    • creating a new branch
  • ALTER TABLE table EXECUTE drop_branch(name)
    • removing a branch by name
  • ALTER TABLE table EXECUTE fast_forward(from, to)
    • performs a fast-forward of from up to the to snapshot if from is an ancestor of to.

TestIcebergBranching uses that new syntax and procedures.

Relates to #12844

References

Release notes

# SPI
* TBD. ({issue}`issuenumber`)

# Iceberg
* Add support for `create_branch`, `drop_branch` and `fast_forward` table procedures. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Dec 21, 2024
@github-actions github-actions bot added the iceberg Iceberg connector label Dec 21, 2024
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch 3 times, most recently from 2a6ee7f to 60394b8 Compare December 21, 2024 13:05
@github-actions github-actions bot added the hive Hive connector label Dec 21, 2024
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch 6 times, most recently from 42baa70 to a797e04 Compare December 24, 2024 00:38
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch from a797e04 to 571c906 Compare January 7, 2025 01:05
@ebyhr ebyhr changed the title PoC v2: Add support for DML with properties in engine + Add support for branching in Iceberg Add support for DML with properties in engine + Add support for branching in Iceberg Jan 8, 2025
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch from 571c906 to 4948981 Compare January 8, 2025 07:43
@ebyhr ebyhr marked this pull request as ready for review January 14, 2025 07:59
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch from 4948981 to 9edb316 Compare January 31, 2025 06:29
Comment on lines +5868 to +5871
if (table.getBranch().isPresent()) {
verify(table.getQueryPeriod().isEmpty(), "query period must be empty");
Optional<TableVersion> endVersion = table.getBranch().map(branch -> new TableVersion(TARGET_ID, VARCHAR, utf8Slice(branch)));
return metadata.getRedirectionAwareTableHandle(session, name, Optional.empty(), endVersion);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add another parameter instead of reusing the existing end-version.

@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes-v2 branch from 9edb316 to 732ac7e Compare February 7, 2025 22:51
@ebyhr
Copy link
Member Author

ebyhr commented Feb 26, 2025

Superseded by #25152

@ebyhr ebyhr closed this Feb 26, 2025
@ebyhr ebyhr deleted the ebi/iceberg-tag-branch-writes-v2 branch February 26, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

1 participant