Skip to content

chore: disable ORC in Iceberg Spark tests#4999

Open
hsiang-c wants to merge 5 commits into
apache:mainfrom
hsiang-c:disable_orc_in_iceberg_spark
Open

chore: disable ORC in Iceberg Spark tests#4999
hsiang-c wants to merge 5 commits into
apache:mainfrom
hsiang-c:disable_orc_in_iceberg_spark

Conversation

@hsiang-c

@hsiang-c hsiang-c commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #4586

Rationale for this change

  • Reduce CI resource consumption by skipping ORC related Iceberg Spark tests b/c Comet doesn't accelerate Iceberg ORC scans.

What changes are included in this PR?

  • Scope: Iceberg Spark 4.1 tests only
  • Summarized with the help of Claude
# Pattern What to do Applied to
1 Drop ORC rows from parameterized parameters() providers Delete the Object[] / Object[][] entries that pass FileFormat.ORC or "orc" SparkRowLevelOperationsTestBase, TestRowLevelOperationsWithLineage, TestFilteredScan, TestPartitionPruning, TestPartitionValues, TestSparkReadProjection, TestSparkMetadataColumns, TestSparkReaderDeletes, TestSparkScan, TestSparkDataWrite, TestIdentityPartitionData, TestMetadataTablesWithPartitionEvolution, TestPositionDeletesTable, TestDataFrameWriterV2Coercion, TestCompressionSettings
2 Disable whole ORC-only test classes Add import org.junit.jupiter.api.Disabled; and annotate the class with @Disabled TestOrcWrite, TestSparkOrcReader, TestSparkOrcReadMetadataColumns, TestSparkRecordOrcReaderWriter, TestVectorizedOrcDataReader, TestORCDataFrameWrite
3 Disable individual ORC test methods Replace @TestTemplate / @Test with @Disabled on the single ORC case TestRewriteTablePathsAction#testPositionDeletesOrc()
4 Remove ORC-specific assertions / setup / imports inline Strip the ORC statements and now-unused static imports from tests that mix ORC with other formats TestSparkCompressionUtil (delete testOrcCompressionRatios() + ORC branches, remove import static …FileFormat.ORC); TestSparkWriteConf (delete ORC compression cases + ORC_COMPRESSION, ORC_COMPRESSION_STRATEGY, DELETE_ORC_COMPRESSION, DELETE_ORC_COMPRESSION_STRATEGY imports); TestCompressionSettings (drop ORC_COMPRESSION/DELETE_ORC_COMPRESSION put() calls + imports); TestStructuredStreamingRead3 (remove orcFileRecords, its appendData(…, "orc"), and the ORC part of the expected-result assertion)

How are these changes tested?

Running Iceberg Spark 4.1 with the new 1.11.0.diff

@hsiang-c
hsiang-c force-pushed the disable_orc_in_iceberg_spark branch from b769a9d to af3ea84 Compare July 22, 2026 00:57
@hsiang-c
hsiang-c force-pushed the disable_orc_in_iceberg_spark branch from af3ea84 to 8791826 Compare July 22, 2026 01:07
@hsiang-c hsiang-c changed the title chore: disable orc in Iceberg Spark tests chore: disable ORC in Iceberg Spark tests Jul 22, 2026
@hsiang-c
hsiang-c marked this pull request as ready for review July 22, 2026 05:55
@mbutrovich

Copy link
Copy Markdown
Contributor

One thing we might want to add is a test to CometIcebergNativeSuite that uses ORC as the data file type, that way we at least have a self-contained test that the fallback is working correctly.

@mbutrovich

Copy link
Copy Markdown
Contributor

Interestingly, this doesn't seem to save much time in CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: Remove ORC testing from Iceberg Java tests

2 participants