The committer and the stream writer are chained into the same operator

[uvashishtha]

Apache Iceberg version

1.10.0 (latest release)

Query engine

Flink

Please describe the bug 🐞

With an upgrade to using Flink 2.0 and iceberg 1.10, we noticed that the Iceberg Files committer stream and the writer stream were chained together in the same operator (vertex node). This was not the case with versions <= Flink 1.20 where the committer (with max parallelism 1) and writer streams were different chains (occupied different flink vertices). Is this expected?

If it is, then functionality (like autoscaling) that was dependent on per-vertex metric consumption breaks.

If there is a mistake in my understanding, please correct it.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time