[FLINK-35615][base] Add a workaround for timestamp binary en/decoding failure with precisions mismatch #3446
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a workaround for FLINK-35615.
Currently, MySQL source could not obtain the precise timestamp precision of data field types, and used a intuitive method to infer it by checking data record precision as follows:
Thus, the precision of timestamp might be underestimated. This would cause a binary encoding / decoding issue, since CDC will store timestamps with <= 3 precision in compact, and allocate extra memory segments for high-precision timestamps. So, timestamps with mismatched precision would cause OOB memory access.
This PR disables such low-precision timestamp storage optimization for timestamps to ensure binary records always could be correctly decoded. The cost is now each timestamp records requires 12 bytes instead of 8 bytes.
cc @Jiabao-Sun @loserwang1024