Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask] support row-level operations to iceberg Table #2543

Closed
caican00 opened this issue Mar 15, 2024 · 1 comment · Fixed by #3243
Closed

[Subtask] support row-level operations to iceberg Table #2543

caican00 opened this issue Mar 15, 2024 · 1 comment · Fixed by #3243
Assignees
@caican00
Labels
0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 subtask Subtasks of umbrella issue
Milestone
Gravitino June Re...

Comments

@caican00
Copy link
Collaborator

caican00 commented Mar 15, 2024 •
edited
Loading

Describe the subtask

  1. support merge operation to iceberg Table.
    For commands:
    update catalog.db.table set c1=x, c2=y, ... where xxx

merge into xxx

  1. inject iceberg extension

Parent issue

#1571
@caican00 caican00 added the subtask Subtasks of umbrella issue label Mar 15, 2024
@caican00
Copy link
Collaborator Author

will work on it.

@FANNG1 FANNG1 mentioned this issue Mar 19, 2024
Merged
@caican00 caican00 changed the title [Subtask] support merge operation to iceberg Table [Subtask] support row-level update operation to iceberg Table Mar 19, 2024
@caican00 caican00 changed the title [Subtask] support row-level update operation to iceberg Table [Subtask] support row-level operation to iceberg Table Mar 21, 2024
@caican00 caican00 changed the title [Subtask] support row-level operation to iceberg Table [Subtask] support row-level operations to iceberg Table Mar 21, 2024
@caican00 caican00 mentioned this issue Mar 21, 2024
Closed
caican00 added a commit to caican00/gravitino that referenced this issue Mar 22, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
741f9a8 
…iceberg Table
@caican00 caican00 mentioned this issue Mar 22, 2024
Closed
@jerryshao jerryshao added this to the Gravitino June Release milestone Apr 24, 2024
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
e9ff76d 
…iceberg Table
@caican00 caican00 mentioned this issue May 1, 2024
Merged
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
6c97f57 
…iceberg Table
caican00 added a commit to caican00/gravitino that referenced this issue May 1, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
d66ff10 
…iceberg Table
caican00 added a commit to caican00/gravitino that referenced this issue May 2, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
4d334aa 
…iceberg Table
caican00 added a commit to caican00/gravitino that referenced this issue May 5, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
ae71559 
…iceberg Table
FANNG1 pushed a commit that referenced this issue May 13, 2024
@caican00
[#2543] feat(spark-connector): support row-level operations to iceber…
f207466 
…g Table (#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
github-actions bot pushed a commit that referenced this issue May 13, 2024
@caican00 @web-flow
[#2543] feat(spark-connector): support row-level operations to iceber…
e40c5da 
…g Table (#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
@jerryshao jerryshao mentioned this issue May 13, 2024
Closed
caican00 added a commit to caican00/gravitino that referenced this issue May 14, 2024
@caican00
[apache#2543] feat(spark-connector): support row-level operations to …
c811897 
…iceberg Table (apache#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: apache#2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
@caican00 caican00 mentioned this issue May 14, 2024
Merged
FANNG1 pushed a commit that referenced this issue May 14, 2024
@caican00
[#2543] feat(spark-connector): support row-level operations to iceber…
315ffc4 
…g Table (#3382)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: #2543

### Does this PR introduce any user-facing change? 
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
@FANNG1 FANNG1 added 0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 labels May 15, 2024
diqiu50 pushed a commit to diqiu50/gravitino that referenced this issue Jun 13, 2024
@caican00 @diqiu50
[apache#2543] feat(spark-connector): support row-level operations to …
2c1d6df 
…iceberg Table (apache#3243)

### What changes were proposed in this pull request?

- refactor table implementation, make `SparkIcebergTable` extend Iceberg
`SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.

- support row-level operations to iceberg Table

```
1. update tableName set c1=v1, c2=v2, ...

2. merge into targetTable t
   using sourceTable s
   on s.key=t.key
   when matched then ...
   when not matched then ...

3. delete from table where xxx
```

### Why are the changes needed?

1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
identify whether it is an Iceberg table, so the `SparkIcebergTable` must
extend `SparkTable`.

2. support row-level operations to iceberg Table.

Fix: apache#2543

### Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...

### How was this patch tested?
New ITs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@caican00 caican00

Labels
0.5.1 Release v0.5.1 0.6.0 Release v0.6.0 subtask Subtasks of umbrella issue
Projects
None yet
Milestone
Gravitino June Release
3 participants
@jerryshao @FANNG1 @caican00