Skip to content

Conversation

@voonhous
Copy link
Member

@voonhous voonhous commented Dec 20, 2025

Describe the issue this Pull Request addresses

Reference issue: #14283

Remove methods that were migrated to HoodieSchemaUtils, consolidate remaining Avro-specific utilities, update documentation. The scope here only covers:

  1. hudi-hadoop-common

NOTE: Merge this after #17599

Key Changes:

  1. Migration: Swapping out HoodieAvroUtils wherever possible.
  2. Full Qualification: For classes where migration was not immediately feasible, calls to HoodieAvroUtils static functions have been fully qualified. This explicitly marks technical debt and makes these usages easily searchable for future refactoring.

Summary and Changelog

Swap out HoodieAvroUtils to HoodieSchema equivalent.

Impact

None

Risk Level

Low

Documentation Update

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@voonhous voonhous requested a review from jonvex December 20, 2025 10:24
@github-actions github-actions bot added the size:XL PR with lines of changes > 1000 label Dec 20, 2025
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch 2 times, most recently from 2de4b54 to e0a6621 Compare December 20, 2025 10:32
@voonhous voonhous marked this pull request as draft December 20, 2025 10:37
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch from e0a6621 to 7b4aa7e Compare December 20, 2025 10:48
@voonhous voonhous changed the title feat(schema): Phase 18 - HoodieAvroUtils removal (Part 3) feat(schema): Phase 18 - HoodieAvroUtils removal (Part 2) Dec 20, 2025
@voonhous voonhous marked this pull request as ready for review December 20, 2025 10:51
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch from 7b4aa7e to c98cc5c Compare December 20, 2025 10:52
@voonhous
Copy link
Member Author

voonhous commented Dec 20, 2025

Will need some suggestions on how to rename:

  1. AvroOrcUtils
  2. TestAvroOrcUtils

These will also need to be renamed:

  1. hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroFileWriterFactory.java
  2. hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroHFileWriter.java
  3. hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroOrcReader.java
  4. hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroOrcWriter.java
  5. hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroParquetReader.java
  6. hudi-hadoop-common/src/main/java/org/apache/parquet/avro/AvroSchemaConverterWithTimestampNTZ.java
  7. hudi-hadoop-common/src/main/java/org/apache/parquet/avro/HoodieAvroParquetSchemaConverter.java
  8. hudi-hadoop-common/src/main/java/org/apache/parquet/avro/HoodieAvroReadSupport.java
  9. hudi-hadoop-common/src/main/java/org/apache/parquet/avro/NativeAvroSchemaConverter.java

I'll leave this as an exercise before actually performing the renames.

@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch 2 times, most recently from 8843e78 to d39084f Compare December 20, 2025 18:19
@voonhous
Copy link
Member Author

@hudi-bot run azure

@voonhous voonhous requested review from the-other-tim-brown and removed request for jonvex December 21, 2025 04:28
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch 2 times, most recently from 422899c to 2eda55e Compare December 21, 2025 07:26
@voonhous voonhous changed the base branch from master to phase-18-HoodieAvroUtils-removal December 24, 2025 18:44
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch from 2eda55e to f7ff4c5 Compare December 24, 2025 18:47
@voonhous
Copy link
Member Author

@the-other-tim-brown This is ready for review too, I've changed the base branch to the PR that it is stacked on top. Only the NET diff are shown now. Should be easier for you to review now.

Additional compilation error fixes
- AvroOrcUtils
- HoodieBootstrapSchemaProvider
- HoodieSparkBootstrapSchemaProvider
- TestOrcBootstrap
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal branch from 24b1721 to f93e264 Compare December 26, 2025 09:44
@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal-p3 branch from f7ff4c5 to c393e06 Compare December 26, 2025 09:47
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@voonhous voonhous force-pushed the phase-18-HoodieAvroUtils-removal branch from f93e264 to a7fd9fa Compare December 27, 2025 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL PR with lines of changes > 1000

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants