HDFS-17204. EC: Reduce unnecessary log when processing excess redundancy. #6107
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
This is a follow-up of HDFS-16964. We now avoid stale replicas when dealing with redundancy. This may result in redundant replicas not being in the
nonExcess
set when we enterBlockManager#chooseExcessRedundancyStriped
(because the datanode where the redundant replicas are located has not send FBR yet, so those replicas are filtered out and not added to thenonExcess
set). A further result is that no excess storage type is selected and the log "excess types chosen for block..." is printed. When a failover occurs, a large number of datanodes become stale, which causes NameNodes to print a large number of unnecessary logs.This issue needs to be fixed, otherwise the performance after failover will be affected.