[spark] Support data evolution for bucket table #6649
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shidayang
force-pushed
the
support-data-evolution-for-bucket
branch
from
6e05843 to
8c335f2
Compare
Contributor
shidayang
changed the title
shidayang
force-pushed
the
support-data-evolution-for-bucket
branch
from
8c335f2 to
635b51e
Compare
JingsongLi
reviewed
Nov 24, 2025
| private static void validateRowTracking(TableSchema schema, CoreOptions options) { | ||
| boolean rowTrackingEnabled = options.rowTrackingEnabled(); | ||
| if (rowTrackingEnabled) { | ||
| checkArgument( |
Contributor
There was a problem hiding this comment.
remove whole checkArgument.
| ``` | ||
| Notice that: | ||
| - Row tracking is only supported for unaware append tables, not for primary key tables. Which means you can't define `bucket` and `bucket-key` for the table. | ||
| - Row tracking is only supported for unaware or hash_fixed bucket append tables, not for primary key tables. |
Contributor
There was a problem hiding this comment.
supported for append tables
Contributor
|
And maybe you can add more tests for bucketed tables. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Support data-evolution for append only fixed hash table
Tests
API and Format
Documentation