[Feature] support create iceberg table #21378
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stephen-shelby
force-pushed
the
create_iceberg_table
branch
from
d15bbe7 to
866907f
Compare
DorianZheng
reviewed
Apr 12, 2023
fe/fe-core/src/main/java/com/starrocks/connector/iceberg/IcebergApiConverter.java
Show resolved
Hide resolved
DorianZheng
reviewed
Apr 12, 2023
fe/fe-core/src/main/java/com/starrocks/connector/iceberg/IcebergCatalog.java
Outdated
Show resolved
Hide resolved
stephen-shelby
force-pushed
the
create_iceberg_table
branch
from
866907f to
5154b20
Compare
Signed-off-by: stephen <stephen5217@163.com>
stephen-shelby
force-pushed
the
create_iceberg_table
branch
from
5154b20 to
db3185d
Compare
stephen-shelby
commented
Apr 12, 2023
| @@ -1966,6 +1966,7 @@ optimizerTrace | |||
| partitionDesc | |||
| : PARTITION BY RANGE identifierList '(' (rangePartitionDesc (',' rangePartitionDesc)*)? ')' | |||
| | PARTITION BY LIST identifierList '(' (listPartitionDesc (',' listPartitionDesc)*)? ')' | |||
| | PARTITION BY identifierList | |||
There was a problem hiding this comment.
this grammar can cover hive/hudi/iceberg/deltalake creating table
Astralidea
reviewed
Apr 12, 2023
fe/fe-core/src/main/java/com/starrocks/connector/iceberg/IcebergApiConverter.java
Outdated
Show resolved
Hide resolved
Astralidea
reviewed
Apr 12, 2023
fe/fe-core/src/main/java/com/starrocks/connector/iceberg/IcebergApiConverter.java
Outdated
Show resolved
Hide resolved
Astralidea
reviewed
Apr 12, 2023
|
SonarCloud Quality Gate failed. |
Astralidea
approved these changes
Apr 12, 2023
HangyuanLiu
approved these changes
Apr 12, 2023
numbernumberone
pushed a commit
to numbernumberone/starrocks
that referenced
this pull request
May 31, 2023
Signed-off-by: stephen <stephen5217@163.com>
Support create iceberg table under iceberg catalog. This table will sync to the user meta service. The Spark/hive can also query it.
usage:
create external table iceberg_catalog.iceberg_db.iceberg_table (c1 int, c2 int);
create external table iceberg_db.iceberg_table (c1 int, c2 int); // current context catalog is iceberg catalog
create external table iceberg_table (c1 int, c2 int); // current context catalog and db are both iceberg.
create external table iceberg_table (c1 int, c2 int ) engine = iceberg;
create external table iceberg_table (c1 int, c2 int, c3 int ) engine = iceberg partition by (c2, c3);
create external table iceberg_table (c1 int, c2 int ) partition by (c2) properties ("location"="hdfs://hadoop:9000/user/warehouse/iceberg_db.db/iceberg_table", "file_format"="parquet");
abc982627271
pushed a commit
to abc982627271/starrocks
that referenced
this pull request
Jun 5, 2023
Signed-off-by: stephen <stephen5217@163.com>
Support create iceberg table under iceberg catalog. This table will sync to the user meta service. The Spark/hive can also query it.
usage:
create external table iceberg_catalog.iceberg_db.iceberg_table (c1 int, c2 int);
create external table iceberg_db.iceberg_table (c1 int, c2 int); // current context catalog is iceberg catalog
create external table iceberg_table (c1 int, c2 int); // current context catalog and db are both iceberg.
create external table iceberg_table (c1 int, c2 int ) engine = iceberg;
create external table iceberg_table (c1 int, c2 int, c3 int ) engine = iceberg partition by (c2, c3);
create external table iceberg_table (c1 int, c2 int ) partition by (c2) properties ("location"="hdfs://hadoop:9000/user/warehouse/iceberg_db.db/iceberg_table", "file_format"="parquet");
southernriver
pushed a commit
to southernriver/starrocks
that referenced
this pull request
Nov 28, 2023
Signed-off-by: stephen <stephen5217@163.com>
Support create iceberg table under iceberg catalog. This table will sync to the user meta service. The Spark/hive can also query it.
usage:
create external table iceberg_catalog.iceberg_db.iceberg_table (c1 int, c2 int);
create external table iceberg_db.iceberg_table (c1 int, c2 int); // current context catalog is iceberg catalog
create external table iceberg_table (c1 int, c2 int); // current context catalog and db are both iceberg.
create external table iceberg_table (c1 int, c2 int ) engine = iceberg;
create external table iceberg_table (c1 int, c2 int, c3 int ) engine = iceberg partition by (c2, c3);
create external table iceberg_table (c1 int, c2 int ) partition by (c2) properties ("location"="hdfs://hadoop:9000/user/warehouse/iceberg_db.db/iceberg_table", "file_format"="parquet")
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this：
Which issues of this PR fixes ：
Fixes #
Support create iceberg table under iceberg catalog. This table will sync to the user meta service. The Spark/hive can also query it.
usage:
create external table iceberg_catalog.iceberg_db.iceberg_table (c1 int, c2 int);create external table iceberg_db.iceberg_table (c1 int, c2 int); // current context catalog is iceberg catalogcreate external table iceberg_table (c1 int, c2 int); // current context catalog and db are both iceberg.create external table iceberg_table (c1 int, c2 int ) engine = iceberg;create external table iceberg_table (c1 int, c2 int, c3 int ) engine = iceberg partition by (c2, c3);create external table iceberg_table (c1 int, c2 int ) partition by (c2) properties ("location"="hdfs://hadoop:9000/user/warehouse/iceberg_db.db/iceberg_table", "file_format"="parquet");#21502
Problem Summary(Required) ：
Checklist:
Bugfix cherry-pick branch check: