HDFS-14748. Make DataNodePeerMetrics#minOutlierDetectionSamples configurable #1314

leosunli · 2019-08-19T15:05:30Z

Signed-off-by: sunlisheng sunlisheng@xiaomi.com

hadoop-yetus · 2019-08-20T12:06:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	86	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1110	trunk passed
+1	compile	59	trunk passed
+1	checkstyle	46	trunk passed
+1	mvnsite	66	trunk passed
+1	shadedclient	735	branch has no errors when building and testing our client artifacts.
+1	javadoc	46	trunk passed
0	spotbugs	156	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	152	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	59	the patch passed
+1	compile	55	the patch passed
+1	javac	55	the patch passed
-0	checkstyle	43	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 606 unchanged - 0 fixed = 607 total (was 606)
+1	mvnsite	61	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	697	patch has no errors when building and testing our client artifacts.
+1	javadoc	45	the patch passed
-1	findbugs	31	hadoop-hdfs in the patch failed.
		_ Other Tests _
-1	unit	32	hadoop-hdfs in the patch failed.
+1	asflicense	25	The patch does not generate ASF License warnings.
		3404

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/artifact/out/Dockerfile
GITHUB PR	#1314
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 5f53f4a0d18f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 6244502
Default Java	1.8.0_222
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/testReport/
Max. process+thread count	414 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-08-22T12:53:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	49	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1106	trunk passed
+1	compile	55	trunk passed
+1	checkstyle	53	trunk passed
+1	mvnsite	65	trunk passed
+1	shadedclient	753	branch has no errors when building and testing our client artifacts.
+1	javadoc	51	trunk passed
0	spotbugs	160	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	158	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	57	the patch passed
+1	compile	52	the patch passed
+1	javac	52	the patch passed
-0	checkstyle	39	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 607 unchanged - 0 fixed = 608 total (was 607)
+1	mvnsite	57	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	721	patch has no errors when building and testing our client artifacts.
+1	javadoc	44	the patch passed
-1	findbugs	158	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
		_ Other Tests _
-1	unit	4939	hadoop-hdfs in the patch failed.
+1	asflicense	33	The patch does not generate ASF License warnings.
		8436

Reason	Tests
FindBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Write to static field org.apache.hadoop.hdfs.server.datanode.metrics.DataNodePeerMetrics.minOutlierDetectionSamples from instance method new org.apache.hadoop.hdfs.server.datanode.metrics.DataNodePeerMetrics(String, Configuration) At DataNodePeerMetrics.java:from instance method new org.apache.hadoop.hdfs.server.datanode.metrics.DataNodePeerMetrics(String, Configuration) At DataNodePeerMetrics.java:[line 70]
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.hdfs.server.datanode.TestDataNodeMetrics
	hadoop.hdfs.qjournal.server.TestJournalNodeSync
	hadoop.hdfs.server.balancer.TestBalancer

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/artifact/out/Dockerfile
GITHUB PR	#1314
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux e574db6f146f 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `ee7c261`
Default Java	1.8.0_222
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/artifact/out/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/testReport/
Max. process+thread count	4117 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>

hadoop-yetus · 2019-08-27T10:03:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	73	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1319	trunk passed
+1	compile	65	trunk passed
+1	checkstyle	60	trunk passed
+1	mvnsite	73	trunk passed
+1	shadedclient	793	branch has no errors when building and testing our client artifacts.
+1	javadoc	59	trunk passed
0	spotbugs	165	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	163	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	63	the patch passed
+1	compile	56	the patch passed
+1	javac	56	the patch passed
+1	checkstyle	52	the patch passed
+1	mvnsite	63	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	735	patch has no errors when building and testing our client artifacts.
+1	javadoc	52	the patch passed
+1	findbugs	162	the patch passed
		_ Other Tests _
-1	unit	6202	hadoop-hdfs in the patch failed.
+1	asflicense	41	The patch does not generate ASF License warnings.
		10082

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/3/artifact/out/Dockerfile
GITHUB PR	#1314
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux dc14634bc617 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `3329257`
Default Java	1.8.0_222
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/3/testReport/
Max. process+thread count	3571 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1314/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

jojochuang · 2019-08-29T23:59:22Z

+1

jojochuang · 2019-08-30T01:45:16Z

...p-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodePeerMetrics.java


-  public DataNodePeerMetrics(final String name) {
+  public DataNodePeerMetrics(final String name, Configuration conf) {


This one arguably breaks compatibility. But given that this class is
@InterfaceAudience.Private
@InterfaceStability.Unstable
I think this is acceptable.

…). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com> (cherry picked from commit 9152712) (cherry picked from commit b516d2e)

…). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com> (cherry picked from commit 9152712)

…che#1314). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>

…che#1314). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com> (cherry picked from commit 9152712) (cherry picked from commit b516d2e)

Make DataNodePeerMetrics#minOutlierDetectionSamples configurable

b31a0ca

Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>

leosunli force-pushed the slowpeer branch from dce04f7 to b31a0ca Compare August 24, 2019 01:50

jojochuang merged commit 9152712 into apache:trunk Aug 30, 2019

jojochuang reviewed Aug 30, 2019

View reviewed changes

asfgit pushed a commit that referenced this pull request Aug 30, 2019

Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314…

b516d2e

…). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com> (cherry picked from commit 9152712)

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019

Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (apa…

0b24949

…che#1314). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>

RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020

Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (apa…

dcaae53

…che#1314). Contributed by Lisheng Sun. Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-14748. Make DataNodePeerMetrics#minOutlierDetectionSamples configurable #1314

HDFS-14748. Make DataNodePeerMetrics#minOutlierDetectionSamples configurable #1314

Uh oh!

leosunli commented Aug 19, 2019

Uh oh!

hadoop-yetus commented Aug 20, 2019

Uh oh!

hadoop-yetus commented Aug 22, 2019

Uh oh!

hadoop-yetus commented Aug 27, 2019

Uh oh!

jojochuang commented Aug 29, 2019

Uh oh!

jojochuang Aug 30, 2019

Uh oh!

Uh oh!


		public DataNodePeerMetrics(final String name) {
		public DataNodePeerMetrics(final String name, Configuration conf) {

HDFS-14748. Make DataNodePeerMetrics#minOutlierDetectionSamples configurable #1314

HDFS-14748. Make DataNodePeerMetrics#minOutlierDetectionSamples configurable #1314

Uh oh!

Conversation

leosunli commented Aug 19, 2019

Uh oh!

hadoop-yetus commented Aug 20, 2019

Uh oh!

hadoop-yetus commented Aug 22, 2019

Uh oh!

hadoop-yetus commented Aug 27, 2019

Uh oh!

jojochuang commented Aug 29, 2019

Uh oh!

jojochuang Aug 30, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!