Skip to content

Conversation

@cbg-wx
Copy link

@cbg-wx cbg-wx commented Dec 25, 2025

Describe the issue this Pull Request addresses

  • This PR fixed HoodieMergedLogRecordScanner#processNextDeletedRecord, EventTimeAvroPayload#combineAndGetUpdateValue#needUpdatingPersistedRecord method, resovled merge delete record, tranform class issame before compareTo method when ordering field type is String or Decimal.
  • And add EventTimeAvroPayload#preCombine override this method to process delete record.

Summary and Changelog

This change involve the hudi-common module below method:

  • HoodieMergedLogRecordScanner#processNextDeletedRecord
  • EventTimeAvroPayload#combineAndGetUpdateValue#needUpdatingPersistedRecord
  • EventTimeAvroPayload#preCombine

Impact

  • Turn the delete record will always be chosen during merging,regardless of the ordering value into chose the record with max ordering value.
  • Tranform class issame before compareTo method when merged delete record with ordering field type is String or Decimal.

Risk Level

The logic that is not deleting records needs to be checked.

Documentation Update

none

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

is a record marked as delete record by _hoodie_is_deleted=true,the
delete record will always be chosen during merging,regardless of the
ordering value.
@cbg-wx cbg-wx closed this Dec 25, 2025
@cbg-wx cbg-wx reopened this Dec 25, 2025
@github-actions github-actions bot added the size:M PR with lines of changes in (100, 300] label Dec 25, 2025
@cbg-wx cbg-wx changed the title resolve When the ordering field of String type or Decimal type,and this fixed delete record will always be chosen during merging,regardless of the ordering value Dec 25, 2025
@cbg-wx
Copy link
Author

cbg-wx commented Dec 25, 2025

The Early PR #17668

Copy link
Collaborator

@cshuo cshuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, left one minor comment

@cshuo
Copy link
Collaborator

cshuo commented Dec 26, 2025

@cbg-wx could you also format the title according to https://hudi.apache.org/contribute/how-to-contribute/#pr-title-format. cc @danny0405 @yihua @the-other-tim-brown for final review.

@cbg-wx cbg-wx changed the title fixed delete record will always be chosen during merging,regardless of the ordering value fix: Fix the merging behavior of delete records in payload. Dec 26, 2025
@cbg-wx cbg-wx requested a review from cshuo December 26, 2025 06:47
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M PR with lines of changes in (100, 300]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants