feat: introduce reconcile table procedure #6584
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
size/L
docs-not-required
WenyXu
changed the title
WenyXu
force-pushed
the
feat/reconcile-table
branch
from
e39bf15 to
5197994
Compare
Signed-off-by: WenyXu <wenymedia@gmail.com>
WenyXu
force-pushed
the
feat/reconcile-table
branch
from
5197994 to
3201a4d
Compare
Signed-off-by: WenyXu <wenymedia@gmail.com>
WenyXu
force-pushed
the
feat/reconcile-table
branch
from
3201a4d to
f792ebf
Compare
WenyXu
requested review from
waynexia,
v0y4g3r,
MichaelScofield and
a team
as code owners
WenyXu
commented
Jul 24, 2025
| @@ -140,7 +140,7 @@ etcd-client = "0.14" | |||
| fst = "0.4.7" | |||
| futures = "0.3" | |||
| futures-util = "0.3" | |||
| greptime-proto = { git = "https://github.com/GreptimeTeam/greptime-proto.git", rev = "7fcaa3e413947a7a28d9af95812af26c1939ce78" } | |||
| greptime-proto = { git = "https://github.com/GreptimeTeam/greptime-proto.git", rev = "7be0563739721b69ff78686754c51d542c8fe8db" } | |||
There was a problem hiding this comment.
wait for GreptimeTeam/greptime-proto#261
fengjiachun
reviewed
Jul 24, 2025
| let primary_key = | ||
| new_primary_keys | ||
| .get(name) | ||
| .context(InvalidRegionRequestSnafu { |
fengjiachun
reviewed
Jul 24, 2025
| /// | ||
| /// It will check the table id and table name consistency. | ||
| /// If the table id and table name are not consistent, it will return an error. | ||
| pub(crate) async fn validate_table_id_and_name( |
There was a problem hiding this comment.
Why check this? In what situations would id and name be inconsistent?
evenyag
reviewed
Jul 25, 2025
Comment on lines
+747
to
+757
| let new_primary_keys = column_metadatas | ||
| .iter() | ||
| .filter(|c| c.semantic_type == SemanticType::Tag) | ||
| .map(|c| (c.column_schema.name.as_str(), c.column_id)) | ||
| .collect::<HashMap<_, _>>(); | ||
|
|
||
| let old_primary_keys = metadata | ||
| .column_metadatas | ||
| .iter() | ||
| .filter(|c| c.semantic_type == SemanticType::Tag) | ||
| .map(|c| (c.column_schema.name.as_str(), c.column_id)); |
There was a problem hiding this comment.
Primary keys may have a different order than the original column order.
We can create a table with columns (a, b, c) but primary key (c, a).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
#6429
What's changed and what's your intention?
Introduce a procedure for reconciling table metadata between metasrv and datanode.
The reconciliation procedure implements a state machine pattern with the following states:
ReconciliationStart: Validates table info and prepares for reconciliationResolveColumnMetadata- resolve column metadata conflicts with configurable strategiesReconcileRegions- Performs region reconciliation operationsUpdateTableInfo- Update table info in metasrv if neededReconciliationEndPR Checklist
Please convert it to a draft if some of the following conditions are not met.