-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[IOTDB-1583] Raft log failed to be committed in cluster version #3832
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
91061a6
[IOTDB-1583] Raft log failed to be committed in cluster version
cigarl dda6da6
[IOTDB-1583] Raft log failed to be committed in cluster version
cigarl 88b9dd8
delete blank line
cigarl 90393fe
[IOTDB-1583] Raft log failed to be committed in cluster version
cigarl a82e276
Merge remote-tracking branch 'origin/master' into mine/master
cigarl a04a742
use the result of serialization
cigarl 313b67d
spotless apply
cigarl 1597e24
[IOTDB-1583] Raft log failed to be committed in cluster version
cigarl 18b7892
delete blank line
cigarl f2e7726
[IOTDB-1583] Raft log failed to be committed in cluster version
cigarl 75463a2
use the result of serialization
cigarl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1032,6 +1032,15 @@ TSStatus processPlanLocally(PhysicalPlan plan) { | |
| log.setCurrLogTerm(getTerm().get()); | ||
| log.setCurrLogIndex(logManager.getLastLogIndex() + 1); | ||
|
|
||
| // if a single log exceeds the threshold | ||
| // we need to return error code to the client as in server mode | ||
| if (log.serialize().capacity() + Integer.BYTES | ||
| >= ClusterDescriptor.getInstance().getConfig().getRaftLogBufferSize()) { | ||
| logger.error( | ||
| "Log cannot fit into buffer, please increase raft_log_buffer_size;" | ||
| + "or reduce the size of requests you send."); | ||
| return StatusUtils.INTERNAL_ERROR; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like EXECUTE_STATEMENT_ERROR should be a more proper status code to tell user. |
||
| } | ||
| logManager.append(log); | ||
| } | ||
| Timer.Statistic.RAFT_SENDER_APPEND_LOG.calOperationCostTimeFromStart(startTime); | ||
|
|
@@ -1077,6 +1086,14 @@ private TSStatus processPlanLocallyV2(PhysicalPlan plan) { | |
| log.setCurrLogIndex(logManager.getLastLogIndex() + 1); | ||
|
|
||
| startTime = Timer.Statistic.RAFT_SENDER_APPEND_LOG_V2.getOperationStartTime(); | ||
| // just like processPlanLocally,we need to check the size of log | ||
| if (log.serialize().capacity() + Integer.BYTES | ||
| >= ClusterDescriptor.getInstance().getConfig().getRaftLogBufferSize()) { | ||
| logger.error( | ||
| "Log cannot fit into buffer, please increase raft_log_buffer_size;" | ||
| + "or reduce the size of requests you send."); | ||
| return StatusUtils.INTERNAL_ERROR; | ||
| } | ||
| // logDispatcher will serialize log, and set log size, and we will use the size after it | ||
| logManager.append(log); | ||
| Timer.Statistic.RAFT_SENDER_APPEND_LOG_V2.calOperationCostTimeFromStart(startTime); | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When this problem occurs, the service can only be restarted. And the current modification method is consistent with the single server mode. So the operation of splitting logs involves several modules, we may need to control the plan size but not only log.
We may need to spend some time on this part in the future to make it better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this needs further discussion, is some error happened during raft log
apply, maybe only one raft group failed, it will lose some data, and cause the raft group not consistent with each other, how to deal with the exception?For now, I think it ok just throw the exception and not let the log be committed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, Thanks for submit a new issues for further discussion in jira, this pr could merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all your suggesions,I will merge the pr, further discussion can be found here #3856