YARN-10041. Create tmp socket file under /tmp for CSI tests #1771

liusheng · 2019-12-19T02:28:59Z

As the issue described, it is easy to exceed the UNIX_PATH_MAX limit if
we create the CSI test socket file under the test code dicrectory.

hadoop-yetus · 2019-12-19T03:57:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 52s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	21m 31s	trunk passed
+1 💚	compile	0m 27s	trunk passed
+1 💚	checkstyle	0m 15s	trunk passed
+1 💚	mvnsite	0m 28s	trunk passed
+1 💚	shadedclient	14m 45s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	0m 54s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 52s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
-0 ⚠️	checkstyle	0m 11s	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	mvnsite	0m 23s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 15s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 19s	the patch passed
+1 💚	findbugs	0m 57s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 33s	hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 27s	The patch does not generate ASF License warnings.
		61m 11s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/1/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 99a12241e5c8 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `7b93575`
Default Java	1.8.0_222
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/1/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/1/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

ZhengZhenyu

LGTM thanks

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java

hadoop-yetus · 2019-12-20T07:55:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	21m 17s	trunk passed
+1 💚	compile	0m 25s	trunk passed
+1 💚	checkstyle	0m 19s	trunk passed
+1 💚	mvnsite	0m 29s	trunk passed
+1 💚	shadedclient	16m 40s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 25s	trunk passed
+0 🆗	spotbugs	1m 7s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	1m 5s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s	the patch passed
+1 💚	compile	0m 24s	the patch passed
+1 💚	javac	0m 24s	the patch passed
-0 ⚠️	checkstyle	0m 12s	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	mvnsite	0m 29s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	16m 48s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s	the patch passed
+1 💚	findbugs	1m 5s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 37s	hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 33s	The patch does not generate ASF License warnings.
		64m 42s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/2/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 21df018def8e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `f777cd3`
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/2/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/2/testReport/
Max. process+thread count	372 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-12-20T08:26:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 57s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	23m 3s	trunk passed
+1 💚	compile	0m 25s	trunk passed
+1 💚	checkstyle	0m 18s	trunk passed
+1 💚	mvnsite	0m 28s	trunk passed
+1 💚	shadedclient	14m 23s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	1m 2s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 59s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 27s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 14s	the patch passed
+1 💚	mvnsite	0m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	16m 55s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 23s	the patch passed
+1 💚	findbugs	1m 12s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 36s	hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 38s	The patch does not generate ASF License warnings.
		65m 1s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/3/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux c4ca9e970bae 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `f777cd3`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/3/testReport/
Max. process+thread count	371 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java

As the issue described, it is easy to exceed the UNIX_PATH_MAX limit if we create the CSI test socket file under the test code dicrectory.

hadoop-yetus · 2019-12-23T02:52:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	21m 5s	trunk passed
+1 💚	compile	0m 26s	trunk passed
+1 💚	checkstyle	0m 17s	trunk passed
+1 💚	mvnsite	0m 29s	trunk passed
+1 💚	shadedclient	14m 36s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	0m 56s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 52s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 11s	the patch passed
+1 💚	mvnsite	0m 23s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 24s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 19s	the patch passed
+1 💚	findbugs	1m 6s	the patch passed
		_ Other Tests _
-1 ❌	unit	0m 34s	hadoop-yarn-csi in the patch failed.
+1 💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		61m 16s

Reason	Tests
Failed junit tests	hadoop.yarn.csi.client.TestCsiClient

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/4/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux d1ed4947a8f8 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / bc825b9
Default Java	1.8.0_232
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/4/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/4/testReport/
Max. process+thread count	309 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/4/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

ayushtkn · 2019-12-23T04:18:31Z

The test seems failing, the file is deleted as part of test, if it is supposed to be deleted as part of test, you can’t use force delete

liusheng · 2019-12-23T10:49:19Z

@ayushtkn thanks for your review, @vinayakumarb suggested me that it is better to use java tmp dir and don't need to change more lines. :)

hadoop-yetus · 2019-12-23T12:00:08Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 32s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	23m 27s	trunk passed
+1 💚	compile	0m 26s	trunk passed
+1 💚	checkstyle	0m 17s	trunk passed
+1 💚	mvnsite	0m 28s	trunk passed
+1 💚	shadedclient	14m 40s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	0m 54s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 53s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 11s	the patch passed
+1 💚	mvnsite	0m 24s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 55s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 19s	the patch passed
+1 💚	findbugs	0m 58s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 33s	hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 29s	The patch does not generate ASF License warnings.
		63m 27s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/5/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux a777f1126b13 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `34ff7db`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/5/testReport/
Max. process+thread count	307 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/5/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java

hadoop-yetus · 2019-12-27T08:34:20Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	29m 22s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	20m 39s	trunk passed
+1 💚	compile	0m 25s	trunk passed
+1 💚	checkstyle	0m 19s	trunk passed
+1 💚	mvnsite	0m 29s	trunk passed
+1 💚	shadedclient	14m 29s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	0m 54s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 53s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 11s	the patch passed
+1 💚	mvnsite	0m 24s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 48s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 19s	the patch passed
+1 💚	findbugs	0m 56s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 32s	hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		86m 54s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/6/artifact/out/Dockerfile
GITHUB PR	#1771
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux ee92b62eb028 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `8730a7b`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/6/testReport/
Max. process+thread count	368 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1771/6/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

ayushtkn

+1

vinayakumarb

LGTM

…pache#1771)

ZhengZhenyu reviewed Dec 19, 2019

View reviewed changes

vinayakumarb requested changes Dec 20, 2019

View reviewed changes

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java Outdated Show resolved Hide resolved

liusheng requested a review from vinayakumarb December 20, 2019 09:28

ayushtkn reviewed Dec 22, 2019

View reviewed changes

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java Outdated Show resolved Hide resolved

liusheng added 3 commits December 23, 2019 09:49

YARN-10041. Create tmp socket file under /tmp for CSI tests

6a4876d

As the issue described, it is easy to exceed the UNIX_PATH_MAX limit if we create the CSI test socket file under the test code dicrectory.

Delete socket file in cleanup

b263256

change the tearDown

350090c

use Files.createTempDir for creating testRoot

c0d1979

vinayakumarb reviewed Dec 27, 2019

View reviewed changes

...doop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java Outdated Show resolved Hide resolved

correct the Files class import

c9a9ca1

liusheng closed this Dec 27, 2019

liusheng reopened this Dec 27, 2019

ayushtkn approved these changes Dec 27, 2019

View reviewed changes

vinayakumarb approved these changes Dec 27, 2019

View reviewed changes

vinayakumarb merged commit 0fed874 into apache:trunk Dec 27, 2019

RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020

YARN-10041. Should not use AbstractPath to create unix domain socket (a…

98a4784

…pache#1771)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

YARN-10041. Create tmp socket file under /tmp for CSI tests #1771

YARN-10041. Create tmp socket file under /tmp for CSI tests #1771

Uh oh!

liusheng commented Dec 19, 2019

Uh oh!

hadoop-yetus commented Dec 19, 2019

Uh oh!

ZhengZhenyu left a comment

Uh oh!

Uh oh!

hadoop-yetus commented Dec 20, 2019

Uh oh!

hadoop-yetus commented Dec 20, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Dec 23, 2019

Uh oh!

ayushtkn commented Dec 23, 2019

Uh oh!

liusheng commented Dec 23, 2019

Uh oh!

hadoop-yetus commented Dec 23, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Dec 27, 2019

Uh oh!

ayushtkn left a comment

Uh oh!

vinayakumarb left a comment

Uh oh!

Uh oh!

YARN-10041. Create tmp socket file under /tmp for CSI tests #1771

YARN-10041. Create tmp socket file under /tmp for CSI tests #1771

Uh oh!

Conversation

liusheng commented Dec 19, 2019

Uh oh!

hadoop-yetus commented Dec 19, 2019

Uh oh!

ZhengZhenyu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hadoop-yetus commented Dec 20, 2019

Uh oh!

hadoop-yetus commented Dec 20, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Dec 23, 2019

Uh oh!

ayushtkn commented Dec 23, 2019

Uh oh!

liusheng commented Dec 23, 2019

Uh oh!

hadoop-yetus commented Dec 23, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Dec 27, 2019

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

vinayakumarb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!