Update Iceberg version to 1.8.0 #24713
Conversation
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
from
5efe851 to
7bfb179
Compare
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
3 times, most recently
from
f562041 to
a67e5a9
Compare
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
3 times, most recently
from
1903d4b to
a524df5
Compare
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
from
a524df5 to
5b49311
Compare
| @@ -91,7 +91,8 @@ protected QueryRunner createQueryRunner() | |||
| throws Exception | |||
| { | |||
| Network network = Network.newNetwork(); | |||
| minio = closeAfterClass(Minio.builder().withNetwork(network).build()); | |||
| // Use new MinIO version because Iceberg enables strong integrity checks | |||
| minio = closeAfterClass(Minio.builder().withImage("minio/minio:RELEASE.2025-01-20T14-49-07Z").withNetwork(network).build()); | |||
There was a problem hiding this comment.
this will break iceberg in storage like ECS. Can we force iceberg not to use storng integrity checks?
There was a problem hiding this comment.
I reported the issue at Iceberg Slack today: https://apache-iceberg.slack.com/archives/C03LG1D563F/p1739434861646329?thread_ts=1739417985.509649&cid=C03LG1D563F
|
I'm fixing product test failures. |
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
from
5b49311 to
849c3f5
Compare
ebyhr
force-pushed
the
ebi/iceberg-1.8.0
branch
from
849c3f5 to
98b1267
Compare
|
@ebyhr do we plan to merge it? What's the scope of the strong integrity checks issue? |
|
@wendigo I think we should wait for 1.8.1. There are 2 issues - REST catalog compatibility apache/iceberg#12294 and S3 strong integrity checks. The scope of S3 issue is limited because most IO happens with our file system module. However, the scope of REST catalog issue is unclear because it depends on each REST catalog's implementation. I believe this update breaks integration not just with Polaris and Unity, but with most REST catalogs. |
|
Anther PR with 1.8.1 RC #25094 |
|
Closing as we will update to 1.8.1 instead #25094. |
Description
The following PR broke our integration with older REST catalogs:
AWS SDK bump broke our integration with S3-compatible storages (pre-2025 Minio, Vast, Dell EC etc):
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.