Skip to content

Conversation

@paulward24
Copy link
Contributor

The field timedOutItems (an ArrayList, i.e., not thread safe):

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L70

is protected by synchronization on itself (timedOutItems):

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L167-L168

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L267-L268

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L178

However, in one place:

https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReconstructionBlocks.java#L133-L135

it is (trying to be) protected by synchronized using pendingReconstructions --- but this cannot protect timedOutItems.

Synchronized on different objects does not ensure mutual exclusion with the other locations.

I.e., 2 code locations, one synchronized by pendingReconstructions and the other by timedOutItems can still executed concurrently.

This CR adds the synchronized on timedOutItems.

Note that this CR keeps the synchronized on pendingReconstructions, which is needed for a different purpose (protect pendingReconstructions)

…ted by synchronization on itself. However, in one place, the field is inside a synchronized on a different object, which does not ensure protection.
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 517 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1191 trunk passed
+1 compile 57 trunk passed
+1 checkstyle 40 trunk passed
+1 mvnsite 64 trunk passed
+1 shadedclient 737 branch has no errors when building and testing our client artifacts.
+1 javadoc 55 trunk passed
0 spotbugs 160 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 157 trunk passed
_ Patch Compile Tests _
+1 mvninstall 59 the patch passed
+1 compile 54 the patch passed
+1 javac 54 the patch passed
+1 checkstyle 38 the patch passed
+1 mvnsite 64 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 700 patch has no errors when building and testing our client artifacts.
+1 javadoc 45 the patch passed
+1 findbugs 166 the patch passed
_ Other Tests _
-1 unit 6260 hadoop-hdfs in the patch failed.
+1 asflicense 41 The patch does not generate ASF License warnings.
10347
Reason Tests
Failed junit tests hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy
hadoop.hdfs.TestErasureCodingExerciseAPIs
hadoop.hdfs.TestStateAlignmentContextWithHA
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.server.namenode.TestParallelImageWrite
hadoop.hdfs.TestReadStripedFileWithDecoding
hadoop.hdfs.TestSnapshotCommands
hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
hadoop.hdfs.TestEncryptionZonesWithKMS
hadoop.hdfs.server.namenode.TestFSEditLogLoader
hadoop.hdfs.web.TestWebHdfsTimeouts
hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
hadoop.hdfs.TestDecommission
hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
hadoop.hdfs.TestFileChecksumCompositeCrc
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1030/1/artifact/out/Dockerfile
GITHUB PR #1030
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 1660a62135ef 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 9fd3c70
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1030/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1030/1/testReport/
Max. process+thread count 4592 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1030/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@anuengineer anuengineer merged commit d203045 into apache:trunk Jun 28, 2019
shanthoosh added a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
…e#1030)

* Adding initial implementation for EventHubSystemAdmin.

* Code clean up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants