Skip to content

HDFS-17054. Erasure coding: optimize checkReplicaOnStorage method to avoid regarding all replicas on one datanode as corrupt repeatly. #5760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from
+16 −7
Open

HDFS-17054. Erasure coding: optimize checkReplicaOnStorage method to avoid regarding all replicas on one datanode as corrupt repeatly. #5760

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1b68e2b
HDFS-17054. Erasure coding: optimize checkReplicaOnStorage method to …
hfutatzhanghb Jun 19, 2023
664454c
remove unused import
hfutatzhanghb Jun 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions ...hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2550,16 +2550,18 @@ DatanodeDescriptor[] chooseSourceDatanodes(BlockInfo block,

BitSet liveBitSet = null;
BitSet decommissioningBitSet = null;
HashSet<DatanodeDescriptor> haveComputedAsCorrupted = null;
if (isStriped) {
int blockNum = ((BlockInfoStriped) block).getTotalBlockNum();
liveBitSet = new BitSet(blockNum);
decommissioningBitSet = new BitSet(blockNum);
haveComputedAsCorrupted = new HashSet<>();
}

for (DatanodeStorageInfo storage : blocksMap.getStorages(block)) {
final DatanodeDescriptor node = getDatanodeDescriptorFromStorage(storage);
final StoredReplicaState state = checkReplicaOnStorage(numReplicas, block,
storage, corruptReplicas.getNodes(block), false);
storage, corruptReplicas.getNodes(block), false, haveComputedAsCorrupted);
if (state == StoredReplicaState.LIVE) {
if (storage.getStorageType() == StorageType.PROVIDED) {
storage = new DatanodeStorageInfo(node, storage.getStorageID(),
Expand Down Expand Up @@ -4544,25 +4546,32 @@ public NumberReplicas countNodes(BlockInfo b) {
NumberReplicas countNodes(BlockInfo b, boolean inStartupSafeMode) {
NumberReplicas numberReplicas = new NumberReplicas();
Collection<DatanodeDescriptor> nodesCorrupt = corruptReplicas.getNodes(b);
HashSet<DatanodeDescriptor> haveComputedAsCorrupted = null;
if (b.isStriped()) {
haveComputedAsCorrupted = new HashSet<>();
countReplicasForStripedBlock(numberReplicas, (BlockInfoStriped) b,
nodesCorrupt, inStartupSafeMode);
nodesCorrupt, inStartupSafeMode, haveComputedAsCorrupted);
} else {
for (DatanodeStorageInfo storage : blocksMap.getStorages(b)) {
checkReplicaOnStorage(numberReplicas, b, storage, nodesCorrupt,
inStartupSafeMode);
inStartupSafeMode, haveComputedAsCorrupted);
}
}
return numberReplicas;
}

private StoredReplicaState checkReplicaOnStorage(NumberReplicas counters,
BlockInfo b, DatanodeStorageInfo storage,
Collection<DatanodeDescriptor> nodesCorrupt, boolean inStartupSafeMode) {
Collection<DatanodeDescriptor> nodesCorrupt, boolean inStartupSafeMode,
HashSet<DatanodeDescriptor> haveComputedAsCorrupted) {
final StoredReplicaState s;
if (storage.getState() == State.NORMAL) {
final DatanodeDescriptor node = storage.getDatanodeDescriptor();
if (nodesCorrupt != null && nodesCorrupt.contains(node)) {
if (nodesCorrupt != null && nodesCorrupt.contains(node) &&
(haveComputedAsCorrupted == null || !haveComputedAsCorrupted.contains(node))) {
if (haveComputedAsCorrupted != null) {
haveComputedAsCorrupted.add(node);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand your code correctly, if the same block group has two internal blocks on the same datanode, then you will only calculate one. IMO, the current implementation of CorruptReplicasMap does not record which specific internal block on the datanode was corrupt, how could you confirm that there is only one internal block corrupt?

}
s = StoredReplicaState.CORRUPT;
} else if (inStartupSafeMode) {
s = StoredReplicaState.LIVE;
Expand Down Expand Up @@ -4608,12 +4617,12 @@ private StoredReplicaState checkReplicaOnStorage(NumberReplicas counters,
*/
private void countReplicasForStripedBlock(NumberReplicas counters,
BlockInfoStriped block, Collection<DatanodeDescriptor> nodesCorrupt,
boolean inStartupSafeMode) {
boolean inStartupSafeMode, HashSet<DatanodeDescriptor> haveComputedAsCorrupted) {
BitSet liveBitSet = new BitSet(block.getTotalBlockNum());
BitSet decommissioningBitSet = new BitSet(block.getTotalBlockNum());
for (StorageAndBlockIndex si : block.getStorageAndIndexInfos()) {
StoredReplicaState state = checkReplicaOnStorage(counters, block,
si.getStorage(), nodesCorrupt, inStartupSafeMode);
si.getStorage(), nodesCorrupt, inStartupSafeMode, haveComputedAsCorrupted);
countLiveAndDecommissioningReplicas(counters, state, liveBitSet,
decommissioningBitSet, si.getBlockIndex());
}
Expand Down