35 Pull requests merged by 16 people

• GCP: Use catalog endpoint as base when refreshing OAuth2 token

  #12638 merged Mar 28, 2025

• AWS: Use assertThat instead of JUnit4 assertions

  #12668 merged Mar 28, 2025

• Build: Revert AWS SDK from 2.30.31 to 2.29.52

  #12649 merged Mar 28, 2025

• Azure: Support vended credentials refresh in ADLSFileIO.

  #11577 merged Mar 28, 2025

• Docs: Fix Latest Iceberg Support version of Hive

  #12640 merged Mar 27, 2025

• Build: Bump jetty from 11.0.24 to 11.0.25

  #12618 merged Mar 27, 2025

• update status page for pyiceberg as of 0.9.0

  #12645 merged Mar 27, 2025

• Core: Enhance TestRemoveSnapshots

  #12662 merged Mar 27, 2025

• Spark 3.4 : Use correct statistics file in SparkScan::estimateStatistics(Snapshot)

  #12647 merged Mar 27, 2025

• Spark 3.4: Migrate SparkRowLevelOperationsTestBase related tests to JUnit 5

  #12656 merged Mar 27, 2025

• Docs: Fix ASF sponsorship links

  #12646 merged Mar 26, 2025

• Docs: Update block spacing guideline in contribute.md

  #12641 merged Mar 26, 2025

• AWS: fix incorrect parent session when calling delegate auth manager

  #12582 merged Mar 26, 2025

• API, Core: Add geometry and geography types support

  #12346 merged Mar 25, 2025

• Core: child HTTPClient should not close shared resources

  #12566 merged Mar 25, 2025

• Core: Add option to fallback to thread classloader

  #12613 merged Mar 25, 2025

• Added New Blog Post: Loading Data into Apache Iceberg

  #12587 merged Mar 24, 2025

• Build: Bump calcite from 1.10.0 to 1.39.0

  #12617 merged Mar 24, 2025

• Build: Bump parquet from 1.15.0 to 1.15.1

  #12616 merged Mar 24, 2025

• Docs: Fix lifecycle and versions in multi-engine-support

  #12370 merged Mar 24, 2025

• Spark 3.4: Propagate snapshot properties / Add max allowed failed commits

  #12632 merged Mar 24, 2025

• Data: Refactor PartitionStatsHandler

  #12550 merged Mar 24, 2025

• Core: Add commit metrics for rewriting manifests

  #12630 merged Mar 24, 2025

• Build: Bump com.google.errorprone:error_prone_annotations from 2.36.0 to 2.37.0

  #12622 merged Mar 24, 2025

• Build: Enforce error message check on Exception assertions

  #12624 merged Mar 24, 2025

• Core: Add update event for rewrite manifests

  #12627 merged Mar 24, 2025

• Spark 3.4: Backport Spark actions changes in Spark rewrite_table_path procedure (#12006 #12172 #11929 #12282 #12569)

  #12568 merged Mar 24, 2025

• Spark 3.5: Adjust repeated INFO logs to DEBUG in SparkWrite and SparkPositionDeltaWrite

  #12404 merged Mar 24, 2025

• Spec: Geo spec simplifications

  #12533 merged Mar 24, 2025

• Build: Bump nessie from 0.103.0 to 0.103.2

  #12615 merged Mar 23, 2025

• Build: Bump mkdocs-material from 9.6.8 to 9.6.9

  #12614 merged Mar 23, 2025

• Spark 3.4: Rewrite V2 deletes to V3 DVs / Detect dangling DVs properly

  #12606 merged Mar 22, 2025

• Spark 3.4: Rewrite data files with high delete ratio

  #12601 merged Mar 22, 2025

• Parquet: Implement Variant metrics

  #12496 merged Mar 21, 2025

• Spark: Backport Spark 3.5 DVs related part to Spark 3.4

  #12603 merged Mar 21, 2025

27 Pull requests opened by 24 people

• ORC: Implement initial default values for readers

  #12604 opened Mar 21, 2025

• chore: improve coordinator election logging

  #12609 opened Mar 22, 2025

• API: Follow up on adding Variant data type to implement sanitizing fo…

  #12611 opened Mar 22, 2025

• AWS: Fix Catalog URI within VendedCredentialsProvider

  #12612 opened Mar 22, 2025

• Build: Bump com.google.cloud:libraries-bom from 26.55.0 to 26.57.0

  #12619 opened Mar 23, 2025

• Build: Bump guava from 33.4.0-jre to 33.4.5-jre

  #12620 opened Mar 23, 2025

• Build: Bump software.amazon.awssdk:bom from 2.30.31 to 2.31.6

  #12621 opened Mar 23, 2025

• Doc: Update Instructions for rewrite_table_path.

  #12628 opened Mar 24, 2025

• Core: Support incremental compute for partition stats

  #12629 opened Mar 24, 2025

• Parquet: Fix column pruning for deeply nested fields

  #12634 opened Mar 25, 2025

• CORE: Inject OAuth2 Token from TableSession

  #12635 opened Mar 25, 2025

• Core, Hive: Double check commit status in case of commit conflict for NoLock

  #12637 opened Mar 25, 2025

• Flink: add snapshot expiration reset strategy

  #12639 opened Mar 25, 2025

• Flink: Support create table like and source watermark for flink sql to 1.18,1.19

  #12643 opened Mar 25, 2025

• Spec: Allow the use of `source-id` in V3

  #12644 opened Mar 25, 2025

• Spark: when doing rewrite_data_files, check for partitioning schema compatibility

  #12651 opened Mar 26, 2025

• Core: ability to create REST catalog with external AuthManager

  #12655 opened Mar 26, 2025

• Spark, API: Enhance hashing efficiency by operating on raw UTF-8 bytes

  #12657 opened Mar 26, 2025

• spec: Variant lower/upper bounds

  #12658 opened Mar 26, 2025

• AWS: Add parameter of excluding non-current fields in Glue

  #12664 opened Mar 27, 2025

• Core: Add MetricsReporter for SnapshotManager

  #12665 opened Mar 27, 2025

• Core: Cleanup TestFindFiles

  #12666 opened Mar 27, 2025

• API, Core: Geospatial bounds and spatial predicates

  #12667 opened Mar 27, 2025

• Core: Enhance remove snapshots efficiency by executing them in bulk

  #12670 opened Mar 27, 2025

• AWS: Delegate part of AWS integration tests to using mock aws services and enable tests in check task

  #12671 opened Mar 27, 2025

• Core: Support first-row-id for manifests and manifest lists

  #12672 opened Mar 28, 2025

• allow dashes in glue database and table names

  #12677 opened Mar 28, 2025

18 Issues closed by 5 people

• How to get the specific catalog config from Iceberg REST get config interface?

  #11124 closed Mar 28, 2025

• Cannot commit identity partition on datatypes time,timestamp* using 'fromPartitionString'

  #11085 closed Mar 28, 2025

• Huge amount of Aws s3 Exception "Unable to execute HTTP request: The target server failed to respond" during Iceberg v2 table merge with some DeleteFiles + DataFiles in a partition

  #8218 closed Mar 28, 2025

• Iceberg Flink: Writing full row in EQ-delete files on UPDATE BEFORE

  #12650 closed Mar 26, 2025

• support equality/positional deletes in vectorized arrow reader

  #11120 closed Mar 26, 2025

• Iceberg defaulting to URLConnectionHttpClient instead of Apache HTTP Client

  #11116 closed Mar 26, 2025

• Support partial insert in merge into command

  #8199 closed Mar 26, 2025

• Drop

  #12636 closed Mar 25, 2025

• Location Ownership

  #9133 closed Mar 25, 2025

• Multi-Column Transforms

  #9132 closed Mar 25, 2025

• Type Promotion: Int/Long to String

  #9064 closed Mar 25, 2025

• Flink: add more sink shuffling support

  #6303 closed Mar 25, 2025

• Add checkstyle rule to ensure AssertJ assertions always check for underlying exception message

  #7040 closed Mar 24, 2025

• Kafka Connect: auto create with lowercase columns

  #11091 closed Mar 24, 2025

• Spark Streaming Job with multiple queries MERGE INTO the same target table (Runtime file filtering is not possible)

  #11094 closed Mar 23, 2025

• com.esotericsoftware.kryo.KryoException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.nio.ByteBuffer

  #9738 closed Mar 23, 2025

• Iceberg Glue Concurrent Update can result in missing metadata_location

  #9411 closed Mar 23, 2025

• Improve coordinator election logging in Iceberg Kafka Connect Sink

  #12608 closed Mar 22, 2025

16 Issues opened by 16 people

• Flaky test `TestSerializedMetadata > testEmptyVariantMetadata()`

  #12676 opened Mar 28, 2025

• Multiple dialect in view

  #12675 opened Mar 28, 2025

• Flink supports clean orphan files

  #12674 opened Mar 28, 2025

• Get metadata location path of the iceberg table

  #12663 opened Mar 27, 2025

• Merge operation in spark could not find column

  #12661 opened Mar 27, 2025

• Iceberg potential metadata conflicts

  #12660 opened Mar 27, 2025

• Apache Iceberg -- Row level duplicates elimination without any data loss similar to df.dropDuplicates() method

  #12659 opened Mar 27, 2025

• Trying to access closed classloader on AWS 'getFileStatus'

  #12654 opened Mar 26, 2025

• Spark: MERGE INTO Statements with only WHEN NOT MATCHED Clauses are always executed at Snapshot Isolation

  #12653 opened Mar 26, 2025

• Exception on Subsequent Writes When Using Nessie or REST Catalog in Iceberg

  #12652 opened Mar 26, 2025

• When iceberg.tables.auto-create-enabled is set true the Iceberg Kafka Connect Sink will try to write to local database of Hive metastore instead of the bucket on object storage

  #12648 opened Mar 26, 2025

• Rest Catalog: Remove snapshots more efficiently

  #12642 opened Mar 25, 2025

• Flink and iceberg table created using AWS Athena

  #12633 opened Mar 24, 2025

• REST catalog: int64 fields should not be represented as numbers in JSON

  #12631 opened Mar 24, 2025

• Improve coordinator election logging in Iceberg Kafka Connect Sink

  #12610 opened Mar 22, 2025

• Flink TableLoader incorrectly parses a table name with a dot

  #12607 opened Mar 21, 2025

87 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• HIVE-28801 Iceberg: Refactor HMS table parameter setting to be able to reuse

  #12461 commented on Mar 27, 2025 • 50 new comments

• Core: FileRewritePlanner implementation

  #12493 commented on Mar 27, 2025 • 43 new comments

• Core: Enable row lineage for all v3 tables

  #12593 commented on Mar 26, 2025 • 34 new comments

• Core: Simplify AuthManager API

  #12555 commented on Mar 28, 2025 • 18 new comments

• CORE: Allow HTTPClient to parse headers from properties.

  #12595 commented on Mar 28, 2025 • 12 new comments

• Spec: update to reflect lineage is required

  #12580 commented on Mar 27, 2025 • 12 new comments

• Core, Spark: Add row lineage metadata columns, and surface them in SparkTable metadata columns

  #12596 commented on Mar 26, 2025 • 8 new comments

• Implementation of version metadata table for view

  #12014 commented on Mar 26, 2025 • 6 new comments

• Spark 3.5: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group

  #12120 commented on Mar 28, 2025 • 5 new comments

• Use delimited column names in CreateChangelogViewProcedure

  #12418 commented on Mar 28, 2025 • 5 new comments

• Support In and notIn operators in ParquetFilters.ConvertFilterToParquet

  #12449 commented on Mar 25, 2025 • 5 new comments

• Spec: Add details on GZIP compressed metadata files

  #12598 commented on Mar 21, 2025 • 4 new comments

• AWS: Update the aws-bundle with latest dependencies

  #12553 commented on Mar 28, 2025 • 4 new comments

• Core: Pass storage credentials from LoadTableResponse to FileIO

  #12591 commented on Mar 28, 2025 • 3 new comments

• OpenAPI: Use more clear language in recommending error responses

  #12376 commented on Mar 26, 2025 • 2 new comments

• Core: Fix numeric overflow of timestamp nano literal

  #11775 commented on Mar 26, 2025 • 1 new comment

• Flink: Add support for Flink 2.0

  #12527 commented on Mar 25, 2025 • 1 new comment

• Parquet: Add variant array reader in Parquet

  #12512 commented on Mar 26, 2025 • 1 new comment

• Proposal: IRC Events endpoint

  #12584 commented on Mar 26, 2025 • 1 new comment

• SPARK: Remove dependency on hadoop's filesystem class from remove orphan files

  #12254 commented on Mar 28, 2025 • 1 new comment

• Core: Interface based DataFile reader and writer API

  #12298 commented on Mar 28, 2025 • 0 new comments

• Add properties support for HadoopTables.load() (#12251)

  #12296 commented on Mar 22, 2025 • 0 new comments

• Docs: Add warning about `snapshot_ids` arg in `expired_snapshots` procedure

  #12291 commented on Mar 25, 2025 • 0 new comments

• S3: Disable strong integrity checksums

  #12264 commented on Mar 26, 2025 • 0 new comments

• Spark: Structured Streaming read limit support follow-up

  #12260 commented on Mar 25, 2025 • 0 new comments

• Core: Remove duplicate definitions of MAX_FILE_GROUP_SIZE_BYTES

  #12222 commented on Mar 22, 2025 • 0 new comments

• Spec additions for encryption

  #12162 commented on Mar 24, 2025 • 0 new comments

• Kafka Connect: Add kerberos authentication option

  #12119 commented on Mar 21, 2025 • 0 new comments

• Core: Select for rewriting the files belonging to old partitioning schemes

  #12083 commented on Mar 25, 2025 • 0 new comments

• Backport #11702 to FLink1.19 and 1.18

  #12080 commented on Mar 27, 2025 • 0 new comments

• Throw on `{write.folder-storage.path,write.object-storage.path}` properties

  #12315 commented on Mar 28, 2025 • 0 new comments

• WIP Parquet: Support reading/writing geometry and geography columns

  #12347 commented on Mar 23, 2025 • 0 new comments

• AWS: fix GlueCatalog name validation

  #12367 commented on Mar 25, 2025 • 0 new comments

• Build: Bump junit to 5.12.1, nessie to 0.103.2

  #12391 commented on Mar 21, 2025 • 0 new comments

• Core: Add support for Avro's timestamp-millis LogicalType in DataReader

  #12397 commented on Mar 27, 2025 • 0 new comments

• Spark: Add separate action to rewrite DVs

  #12403 commented on Mar 24, 2025 • 0 new comments

• Enable HTTP proxy support for the client used by REST Catalog

  #12406 commented on Mar 25, 2025 • 0 new comments

• Generic Serializer and DeSerializer for control topic consumers and producers

  #12583 commented on Mar 22, 2025 • 0 new comments

• Spark-3.5: Add procedure to compute partition stats

  #12451 commented on Mar 26, 2025 • 0 new comments

• Decouple Committer from Kafka and Enable Custom Coordinator Election

  #12460 commented on Mar 23, 2025 • 0 new comments

• Build: Bump com.azure:azure-sdk-bom from 1.2.31 to 1.2.32

  #12487 commented on Mar 28, 2025 • 0 new comments

• Spark: prefix SparkTable with 'iceberg' to clearly identify Iceberg table

  #12543 commented on Mar 26, 2025 • 0 new comments

• Just for running tests

  #12530 commented on Mar 24, 2025 • 0 new comments

• Build: Bump junit from 5.11.4 to 5.12.1

  #12537 commented on Mar 27, 2025 • 0 new comments

• backport #11301(rowconverter) to Flink 1.19 and 1.18

  #11826 commented on Mar 25, 2025 • 0 new comments

• Unexpected `MERGE INTO` behavior - updates applied despite conditions

  #12558 commented on Mar 26, 2025 • 0 new comments

• Provide option to specify user defined schema while reading from iceberg table

  #11217 commented on Mar 26, 2025 • 0 new comments

• Why does executing a sql "desc tableA" in hive command line report a error on a iceberg table with decimal(2,2) field type

  #11211 commented on Mar 26, 2025 • 0 new comments

• Serialization of the org.apache.iceberg.io.WriteResult class.

  #10710 commented on Mar 25, 2025 • 0 new comments

• Ingestion using Iceberg bucketing causing OOM

  #11393 commented on Mar 25, 2025 • 0 new comments

• DeleteOrphanFiles or ExpireSnapshots outofmemory

  #3703 commented on Mar 25, 2025 • 0 new comments

• Move Writer classes from kafka-connect to core

  #11207 commented on Mar 25, 2025 • 0 new comments

• Deleting metadata(expire_snapshots doesn't help...)

  #11169 commented on Mar 25, 2025 • 0 new comments

• Table has more than one bucket keys, but "show create table xxx" only displays one

  #11090 commented on Mar 25, 2025 • 0 new comments

• Flink SQL with Iceberg snapshots doesn't react if table has upsert

  #9948 commented on Mar 25, 2025 • 0 new comments

• Partition stats task tracker

  #8450 commented on Mar 24, 2025 • 0 new comments

• [Feature Request ][iceberg] use AWS IAM role with serviceAccount instated of IAM user

  #12448 commented on Mar 24, 2025 • 0 new comments

• Table corruption using lock-free Hive commits

  #11814 commented on Mar 24, 2025 • 0 new comments

• AWS: Glue ETL Job fails to create a table using lakeformation

  #11126 commented on Mar 24, 2025 • 0 new comments

• java. lang.UnsupportedOperationException: Unknown delete file content: DATA

  #11981 commented on Mar 24, 2025 • 0 new comments

• Retry logic in JDBC catalog fails with class cast exception if driver exception class does not extend SQLTransientException

  #11176 commented on Mar 24, 2025 • 0 new comments

• Improve `All` Metadata Tables with Snapshot Information

  #8856 commented on Mar 23, 2025 • 0 new comments

• Iceberg Read is not working on Iceberg Hive table

  #11168 commented on Mar 23, 2025 • 0 new comments

• Spark sort/zorder rewrite data does not apply the expected SHUFFLE_PARTITIONS for each target group

  #10716 commented on Mar 23, 2025 • 0 new comments

• Support create table `PRIMARY KEY` column via Spark sql?

  #5069 commented on Mar 23, 2025 • 0 new comments

• API: Follow up on adding Variant data type to implement sanitizing for Variant

  #11479 commented on Mar 22, 2025 • 0 new comments

• Core: Fix failure when reading files table with branch

  #11719 commented on Mar 22, 2025 • 0 new comments

• Core: Merge conflicting deletion vectors

  #11693 commented on Mar 25, 2025 • 0 new comments

• Reduce code duplication in VectorizedParquetDefinitionLevelReader

  #11661 commented on Mar 27, 2025 • 0 new comments

• Kafka Connect: Add mechanisms for routing records by topic name

  #11623 commented on Mar 26, 2025 • 0 new comments

• Materialized View Spec

  #11041 commented on Mar 24, 2025 • 0 new comments

• Use Snapshot's statistics file in SparkScan

  #11040 commented on Mar 27, 2025 • 0 new comments

• GCP: Add Iceberg Catalog for GCP BigQuery Metastore

  #11039 commented on Mar 28, 2025 • 0 new comments

• API: Define RepairManifests action interface

  #10784 commented on Mar 27, 2025 • 0 new comments

• Core: HadoopFileIO to support bulk delete through the Hadoop Filesystem APIs

  #10233 commented on Mar 26, 2025 • 0 new comments

• Manifest list encryption

  #7770 commented on Mar 28, 2025 • 0 new comments

• Unable to Roll Back to Previous Version After CREATE OR REPLACE with REST Catalog

  #11524 commented on Mar 28, 2025 • 0 new comments

• ERROR when executing UPDATE/DELETE queries in Iceberg 1.6.0: "Cannot add fieldId 1 as an identifier field"

  #11341 commented on Mar 28, 2025 • 0 new comments

• Missing size rewrite in rewrite_table_path for delete file

  #12554 commented on Mar 28, 2025 • 0 new comments

• DatasourceV2 does not prune columns after V2ScanRelationPushDown

  #9268 commented on Mar 28, 2025 • 0 new comments

• Support relative paths in Table Metadata

  #1617 commented on Mar 28, 2025 • 0 new comments

• Spark Iceberg REST Catalog refresh token

  #12363 commented on Mar 27, 2025 • 0 new comments

• Support Parquet Files with Delta Encoding and other Parquet V2 Features

  #11371 commented on Mar 27, 2025 • 0 new comments

• Subfolder with no name under /data folder

  #12065 commented on Mar 27, 2025 • 0 new comments

• Forbidden Exception creating Polaris Rest catalog with Flink 1.20

  #11836 commented on Mar 26, 2025 • 0 new comments

• Store min/max stats per column per partition

  #11083 commented on Mar 26, 2025 • 0 new comments

• Delete operation is not executed with ThreadPools#DELETE_WORKER_POOL

  #12590 commented on Mar 26, 2025 • 0 new comments