Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS-14648. DeadNodeDetector state machine model #1309

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from

Conversation

leosunli
Copy link
Contributor

Signed-off-by: sunlisheng sunlisheng@xiaomi.com

@leosunli leosunli force-pushed the deadnode branch 4 times, most recently from aa22df7 to 035fd78 Compare August 18, 2019 13:01
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 46 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 18 Maven dependency ordering for branch
+1 mvninstall 1114 trunk passed
+1 compile 189 trunk passed
+1 checkstyle 47 trunk passed
+1 mvnsite 96 trunk passed
+1 shadedclient 770 branch has no errors when building and testing our client artifacts.
+1 javadoc 75 trunk passed
0 spotbugs 155 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 274 trunk passed
_ Patch Compile Tests _
0 mvndep 10 Maven dependency ordering for patch
+1 mvninstall 99 the patch passed
+1 compile 190 the patch passed
+1 javac 190 the patch passed
-0 checkstyle 43 hadoop-hdfs-project: The patch generated 2 new + 127 unchanged - 1 fixed = 129 total (was 128)
+1 mvnsite 92 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 673 patch has no errors when building and testing our client artifacts.
+1 javadoc 72 the patch passed
+1 findbugs 278 the patch passed
_ Other Tests _
+1 unit 105 hadoop-hdfs-client in the patch passed.
-1 unit 5549 hadoop-hdfs in the patch failed.
+1 asflicense 30 The patch does not generate ASF License warnings.
9797
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestBlockRecovery
hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
hadoop.hdfs.server.datanode.TestDataNodeExit
hadoop.hdfs.server.datanode.TestBatchIbr
hadoop.hdfs.server.datanode.TestIncrementalBlockReports
hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetCache
hadoop.hdfs.server.balancer.TestBalancer
hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
hadoop.tools.TestHdfsConfigFields
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
hadoop.hdfs.server.datanode.TestDirectoryScanner
hadoop.hdfs.server.namenode.TestFsck
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/1/artifact/out/Dockerfile
GITHUB PR #1309
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 39694d5d3d0a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 6244502
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/1/artifact/out/diff-checkstyle-hadoop-hdfs-project.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/1/testReport/
Max. process+thread count 4203 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 80 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 21 Maven dependency ordering for branch
+1 mvninstall 1326 trunk passed
+1 compile 230 trunk passed
+1 checkstyle 55 trunk passed
+1 mvnsite 123 trunk passed
+1 shadedclient 876 branch has no errors when building and testing our client artifacts.
+1 javadoc 88 trunk passed
0 spotbugs 203 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 350 trunk passed
_ Patch Compile Tests _
0 mvndep 12 Maven dependency ordering for patch
+1 mvninstall 113 the patch passed
+1 compile 233 the patch passed
+1 javac 233 the patch passed
-0 checkstyle 52 hadoop-hdfs-project: The patch generated 2 new + 127 unchanged - 1 fixed = 129 total (was 128)
+1 mvnsite 113 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 2 The patch has no ill-formed XML file.
+1 shadedclient 744 patch has no errors when building and testing our client artifacts.
+1 javadoc 79 the patch passed
+1 findbugs 337 the patch passed
_ Other Tests _
+1 unit 125 hadoop-hdfs-client in the patch passed.
-1 unit 8563 hadoop-hdfs in the patch failed.
+1 asflicense 35 The patch does not generate ASF License warnings.
13582
Reason Tests
Failed junit tests hadoop.hdfs.qjournal.server.TestJournalNodeSync
hadoop.hdfs.server.namenode.TestXAttrConfigFlag
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
hadoop.hdfs.server.namenode.TestParallelImageWrite
hadoop.hdfs.TestDFSClientDetectDeadNodes
hadoop.hdfs.server.namenode.TestStorageRestore
hadoop.hdfs.server.balancer.TestBalancerService
hadoop.hdfs.server.namenode.TestEditLog
hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
hadoop.hdfs.server.namenode.TestAclConfigFlag
hadoop.hdfs.server.namenode.TestNameNodeReconfigure
hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
hadoop.hdfs.server.namenode.TestINodeFile
hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
hadoop.hdfs.server.namenode.sps.TestStoragePolicySatisfierWithStripedFile
hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
hadoop.hdfs.server.namenode.TestFSEditLogLoader
hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
hadoop.hdfs.server.namenode.TestStartup
hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
hadoop.hdfs.server.namenode.TestNamenodeStorageDirectives
hadoop.hdfs.server.namenode.TestFSDirectory
hadoop.hdfs.server.namenode.TestNameNodeXAttr
hadoop.tools.TestHdfsConfigFields
hadoop.hdfs.server.namenode.TestCheckpoint
hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
hadoop.hdfs.server.balancer.TestBalancer
hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/2/artifact/out/Dockerfile
GITHUB PR #1309
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 9a41aa5bc210 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / ee7c261
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/2/artifact/out/diff-checkstyle-hadoop-hdfs-project.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/2/testReport/
Max. process+thread count 4119 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@leosunli leosunli force-pushed the deadnode branch 2 times, most recently from a669bac to 6dc2360 Compare August 29, 2019 15:24
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 41 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 12 Maven dependency ordering for branch
+1 mvninstall 1079 trunk passed
+1 compile 198 trunk passed
+1 checkstyle 53 trunk passed
+1 mvnsite 123 trunk passed
+1 shadedclient 872 branch has no errors when building and testing our client artifacts.
+1 javadoc 95 trunk passed
0 spotbugs 173 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 294 trunk passed
_ Patch Compile Tests _
0 mvndep 13 Maven dependency ordering for patch
+1 mvninstall 110 the patch passed
+1 compile 191 the patch passed
+1 javac 191 the patch passed
-0 checkstyle 60 hadoop-hdfs-project: The patch generated 1 new + 127 unchanged - 1 fixed = 128 total (was 128)
+1 mvnsite 111 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
-1 shadedclient 238 patch has errors when building and testing our client artifacts.
+1 javadoc 86 the patch passed
+1 findbugs 303 the patch passed
_ Other Tests _
-1 unit 1025 hadoop-hdfs-client in the patch failed.
-1 unit 5340 hadoop-hdfs in the patch failed.
+1 asflicense 39 The patch does not generate ASF License warnings.
10307
Reason Tests
Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap
hadoop.hdfs.TestErasureCodingPolicies
hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy
hadoop.hdfs.security.token.block.TestBlockToken
hadoop.fs.viewfs.TestViewFileSystemLinkFallback
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash
hadoop.hdfs.TestFileCreationClient
hadoop.fs.viewfs.TestViewFileSystemHdfs
hadoop.hdfs.TestDFSClientDetectDeadNodes
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/artifact/out/Dockerfile
GITHUB PR #1309
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 3dd90e56fef5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 915cbc9
Default Java 1.8.0_222
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/artifact/out/diff-checkstyle-hadoop-hdfs-project.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/testReport/
Max. process+thread count 4203 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1309/4/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants