HDFS-14889. Ability to check if a block has a replica on provided storage. #1573
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
goiri
reviewed
Oct 2, 2019
| public BlockInfoContiguous(short size) { | ||
| super(size); | ||
| hasProvidedStorage = false; |
There was a problem hiding this comment.
Can we just set by default as false and not do it here?
There was a problem hiding this comment.
I removed it for now.
| @@ -77,9 +85,31 @@ boolean removeStorage(DatanodeStorageInfo storage) { | |||
| setStorageInfo(dnIndex, getStorageInfo(lastNode)); | |||
| // set the last entry to null | |||
| setStorageInfo(lastNode, null); | |||
| if (storage.getStorageType() == StorageType.PROVIDED | |||
| && !hasProvidedStorages()) { | |||
There was a problem hiding this comment.
Shouldn't this already guarantee is false?
There was a problem hiding this comment.
I see is the other method which goes over each DN, nevermind.
There was a problem hiding this comment.
Can you go over the overhead added to the regular pipeline without Provided?
There was a problem hiding this comment.
I removed the check and there is no invocation in the regular pipeline now.
| return hasProvidedStorage; | ||
| } | ||
|
|
||
| private boolean hasProvidedStorages() { |
...adoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfoStriped.java
Show resolved
Hide resolved
ashvina
changed the title
|
💔 -1 overall
This message was automatically generated. |
goiri
reviewed
Oct 3, 2019
| @Override | ||
| boolean isProvided() { | ||
| int len = getCapacity(); | ||
| for(int idx = 0; idx < len; idx++) { |
goiri
reviewed
Oct 3, 2019
| int len = getCapacity(); | ||
| for(int idx = 0; idx < len; idx++) { | ||
| DatanodeStorageInfo cur = getStorageInfo(idx); | ||
| if(cur != null) { |
|
💔 -1 overall
This message was automatically generated. |
goiri
reviewed
Oct 3, 2019
...t/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockInfo.java
Show resolved
Hide resolved
ashvina
force-pushed
the
isBlockProvided
branch
from
2b84c64 to
42381be
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
goiri
approved these changes
Oct 4, 2019
asfgit
pushed a commit
that referenced
this pull request
Oct 4, 2019
…rage. Contributed by Ashvin Agrawal. (#1573)"
amahussein
pushed a commit
to amahussein/hadoop
that referenced
this pull request
Oct 29, 2019
…rage. Contributed by Ashvin Agrawal. (apache#1573)"
RogPodge
pushed a commit
to RogPodge/hadoop
that referenced
this pull request
Mar 25, 2020
…rage. Contributed by Ashvin Agrawal. (apache#1573)"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses https://issues.apache.org/jira/browse/HDFS-14889.
Adding a method in
BlockInfoto return true of the block has any replica on a Provided storage.