MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception #2775

lichaojacobs · 2021-03-15T06:52:45Z

jira: https://issues.apache.org/jira/browse/MAPREDUCE-7329

Hadoop Pipes Ping implement has a bug. Recently, we upgrade linux kernel version from 3.x to 4.x. And we find hadoop pipe task exit with connect timeout which is implemented by PingThread in HadoopPipes.cc.

After a deep research, we finally find that current ping server won't accept ping client created socket, which may cause critical problem:

it will cause tcp accept queue full(default 50)
when client close socket, server socket won't call close method, which will leave too many CLOSE_WAIT socket fd existed(default 2h), and accept queue never cleared.
Even worse, in 4.x linux kernel version, it will cause tcp drop packet directly which makes ping client connect time out. While In 3.x linux kernel version, when accept queue full, client can also make half connection till sync queue full (default 2048), so from client side, ping will aslo work till sync queue full. And after 3 hours, task will also exit with connect timeout exception.

To fix this bug, we introduced a PingSocketCleaner thread, which will continuously accept ping socket connection from ping client. When socket close from client, cleaner thread will detecte closed inputStream through read method returns -1, then finally close socket from sever side.

Refrenced by linux kernel patch: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5ea8ea2cb7

hadoop-yetus · 2021-03-15T08:15:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 9s		trunk passed
+1 💚	compile	0m 44s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 35s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 34s		trunk passed
+1 💚	mvnsite	0m 39s		trunk passed
+1 💚	javadoc	0m 22s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 20s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 19s		trunk passed
+1 💚	shadedclient	14m 50s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 32s		the patch passed
+1 💚	compile	0m 34s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 34s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 27s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7)
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 17s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 20s		the patch passed
+1 💚	shadedclient	14m 28s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	7m 12s		hadoop-mapreduce-client-core in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		81m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/1/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux bc05a6db4771 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 393b71c6182645f660d87b74eb6e2d745200f5b5
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/1/testReport/
Max. process+thread count	1612 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

lichaojacobs · 2021-03-16T01:13:02Z

@liuml07 @steveloughran Could you please help review the patch? It aims to fix Hadoop Pipes bug. Thanks.

liuml07 · 2021-03-16T03:01:38Z

I did not touch this pipe class or package. I can take a look later this week, but @aajisaka and @wangdatan will have more context. Thanks,

lichaojacobs · 2021-03-16T06:21:12Z

I did not touch this pipe class or package. I can take a look later this week, but @aajisaka and @wangdatan will have more context. Thanks,

ok, thanks any way !

lichaojacobs · 2021-03-16T17:44:06Z

@aajisaka Could you please help review the patch? thanks.

steveloughran · 2021-03-17T11:26:31Z

Never seen this code before so I'm not really in a position to review. Just trying to revise my sockets API knowledge, which dates from when I was writing Windows 3.1 code and hasn't been refreshed much, not since HTTP came along

lichaojacobs · 2021-03-28T08:49:33Z

@liuml07 @steveloughran hi, how is it going now? can you review this patch?

steveloughran

Like I said "I have never looked at this code", so am not in a position to it in. It wouldn't be safe

...t/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/pipes/Application.java

hadoop-yetus · 2021-03-30T04:54:31Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 40s		trunk passed
+1 💚	compile	0m 41s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 44s		trunk passed
+1 💚	javadoc	0m 24s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 21s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 20s		trunk passed
+1 💚	shadedclient	14m 32s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 35s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 35s		the patch passed
+1 💚	compile	0m 27s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 26s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7)
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 19s		the patch passed
+1 💚	shadedclient	14m 23s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 15s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	asflicense	0m 32s		The patch does not generate ASF License warnings.
		80m 27s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/2/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 2bf8a2b04881 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 5dcb6055edccc1f12c2531002b7b66a611d9761f
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/2/testReport/
Max. process+thread count	1557 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-03-30T05:08:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 9s		trunk passed
+1 💚	compile	0m 38s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 34s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 34s		trunk passed
+1 💚	mvnsite	0m 40s		trunk passed
+1 💚	javadoc	0m 23s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 20s		trunk passed
+1 💚	shadedclient	14m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 32s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 25s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7)
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 15s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 24s		the patch passed
+1 💚	shadedclient	14m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 30s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		79m 48s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/3/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux ac69e3c3a4b2 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c4afa6873a5d9ff8927228ff5d1bb5c0b7692581
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/3/testReport/
Max. process+thread count	1581 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-01T04:44:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 12s		trunk passed
+1 💚	compile	0m 39s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 26s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 24s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 15s		trunk passed
+1 💚	shadedclient	14m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	0m 31s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 26s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7)
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 16s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 16s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 17s		the patch passed
+1 💚	shadedclient	14m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 29s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		79m 48s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/4/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux b26c9dc0913a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 9fce5d15c8457e30c02c31422458ee7d8debbe13
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/4/testReport/
Max. process+thread count	1588 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-04-07T06:06:52Z

Can we create a regression test for this issue?

...t/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/pipes/Application.java

aajisaka · 2021-04-07T06:37:40Z

Can we set a shorter timeout than the default (2h) in the server socket to clear the queue?

lichaojacobs · 2021-04-07T06:45:25Z

Can we set a shorter timeout than the default (2h) in the server socket to clear the queue?

if do this, we need to modify kernel parameter net.ipv4.tcp_keepalive_time which i think is out of application scope and also dangerous

aajisaka · 2021-04-07T06:54:09Z

Can ServerSocket#setSoTimeout(int) be used instead of modifying the kernel parameter?

lichaojacobs · 2021-04-07T07:06:58Z

Can ServerSocket#setSoTimeout(int) be used instead of modifying the kernel parameter?

Of course, but need to accept client socket first? In my cleaner implement, it will close socket correctly, and thus accept queue will also be cleared.
And there is another problem, if we set so timeout, then which value is suitable? If too short, data stream may block due to gc or other reasons which will cause task failed.

aajisaka · 2021-04-07T07:43:37Z

Of course, but need to accept client socket first?

No, this method must be called before accepting client sockets.

if we set so timeout, then which value is suitable?

The default value is 60 seconds in Hadoop IPC client (ipc.ping.interval). Is it too long for Hadoop Pipes application?

lichaojacobs · 2021-04-07T08:06:02Z

Of course, but need to accept client socket first?

No, this method must be called before accepting client sockets.

if we set so timeout, then which value is suitable?

The default value is 60 seconds in Hadoop IPC client (ipc.ping.interval). Is it too long for Hadoop Pipes application?

The value sounds reasonable. And I think we should set so timeout on client socket? cause ServerSocket#setSoTimeout(int) just affect accept

And we should do like this:

aajisaka · 2021-04-07T09:49:33Z

And we should do like this:

LGTM

I thought only setting so_timeout fixes this problem. However, it is not true. The problem is that the Application does not call serverSocket.accept() for ping requests to drain the tcp accept queue. Agreed to implement the ping socket cleaner. Thanks.

lichaojacobs · 2021-04-07T10:08:08Z

And we should do like this:

LGTM

I thought only setting so_timeout fixes this problem. However, it is not true. The problem is that the Application does not call serverSocket.accept() for ping requests to drain the tcp accept queue. Agreed to implement the ping socket cleaner. Thanks.

Yes, this is exactly what i am trying to fix. I will try to update with a regression test soon. Thanks!

hadoop-yetus · 2021-04-07T15:27:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 2s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 43s		trunk passed
+1 💚	compile	2m 36s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 11s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	1m 3s		trunk passed
+1 💚	mvnsite	1m 17s		trunk passed
+1 💚	javadoc	0m 48s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 42s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 12s		trunk passed
+1 💚	shadedclient	16m 5s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 3s		the patch passed
+1 💚	compile	2m 26s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 26s		the patch passed
+1 💚	compile	2m 4s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	2m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 54s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt	hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 6 new + 7 unchanged - 0 fixed = 13 total (was 7)
+1 💚	mvnsite	1m 6s		the patch passed
+1 💚	javadoc	0m 34s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 33s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 20s		the patch passed
+1 💚	shadedclient	16m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 50s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	134m 28s		hadoop-mapreduce-client-jobclient in the patch passed.
-1 ❌	asflicense	0m 38s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		236m 48s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/5/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 604906062d2e 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 459bdcdfc42dc481e5796fbe0d9896a244382c45
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/5/testReport/
Max. process+thread count	1559 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-07T15:46:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 48s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 0s		trunk passed
+1 💚	compile	2m 59s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 28s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	1m 1s		trunk passed
+1 💚	mvnsite	1m 18s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 42s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 19s		trunk passed
+1 💚	shadedclient	16m 46s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 2s		the patch passed
+1 💚	compile	2m 33s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 33s		the patch passed
+1 💚	compile	2m 4s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	2m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 52s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt	hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 11 new + 7 unchanged - 0 fixed = 18 total (was 7)
+1 💚	mvnsite	1m 1s		the patch passed
+1 💚	javadoc	0m 32s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 30s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 13s		the patch passed
+1 💚	shadedclient	16m 10s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 39s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	130m 25s		hadoop-mapreduce-client-jobclient in the patch passed.
-1 ❌	asflicense	0m 35s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		236m 43s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/6/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux cb7456b0eb4c 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 2012f3038f5ef44bb1db62cd239927503b1c4bf1
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/6/testReport/
Max. process+thread count	1176 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-07T20:14:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 4s		trunk passed
+1 💚	compile	2m 30s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 8s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 59s		trunk passed
+1 💚	mvnsite	1m 15s		trunk passed
+1 💚	javadoc	0m 49s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 42s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 4s		trunk passed
+1 💚	shadedclient	14m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 21s		the patch passed
+1 💚	compile	2m 28s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 28s		the patch passed
+1 💚	compile	2m 0s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	2m 0s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 50s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt	hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 9 new + 39 unchanged - 0 fixed = 48 total (was 39)
+1 💚	mvnsite	1m 3s		the patch passed
+1 💚	javadoc	0m 36s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 30s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 12s		the patch passed
+1 💚	shadedclient	13m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 43s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	130m 50s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		227m 21s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/7/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 757c8e1b496b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f43e450fede35d6eef0f36c6aaa9dc64a7a80d6b
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/7/testReport/
Max. process+thread count	1581 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-07T20:19:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 31s		trunk passed
+1 💚	compile	2m 49s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 5s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 55s		trunk passed
+1 💚	mvnsite	1m 10s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 39s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 57s		trunk passed
+1 💚	shadedclient	16m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 59s		the patch passed
+1 💚	compile	2m 25s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 25s		the patch passed
+1 💚	compile	2m 3s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	2m 3s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 51s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt	hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 9 new + 39 unchanged - 0 fixed = 48 total (was 39)
+1 💚	mvnsite	0m 58s		the patch passed
+1 💚	javadoc	0m 31s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 29s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 9s		the patch passed
+1 💚	shadedclient	16m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 40s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	130m 30s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		231m 14s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/8/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux c2e92af0fe23 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f43e450fede35d6eef0f36c6aaa9dc64a7a80d6b
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/8/testReport/
Max. process+thread count	1197 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-08T07:08:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 29s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 1s		trunk passed
+1 💚	compile	2m 24s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 3s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 58s		trunk passed
+1 💚	mvnsite	1m 17s		trunk passed
+1 💚	javadoc	0m 53s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 48s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 1s		trunk passed
+1 💚	shadedclient	13m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	2m 17s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 17s		the patch passed
+1 💚	compile	1m 56s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	1m 56s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 49s		the patch passed
+1 💚	mvnsite	1m 1s		the patch passed
+1 💚	javadoc	0m 36s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 34s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 6s		the patch passed
+1 💚	shadedclient	13m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 28s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	129m 48s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		223m 48s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/9/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux eb1df8066109 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 9d4443f5502d5ff82fedcdbc99571980404d1478
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/9/testReport/
Max. process+thread count	1607 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/9/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

lichaojacobs · 2021-04-08T07:12:27Z

@aajisaka Hi, would you please help to review some changes about yesterday's advice again ? And i don't know why there is 💔 -1 overall cause i have not changed that part code.

hadoop-yetus · 2021-04-08T07:22:57Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 11s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 42s		trunk passed
+1 💚	compile	2m 34s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 6s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 56s		trunk passed
+1 💚	mvnsite	1m 10s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 39s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 56s		trunk passed
+1 💚	shadedclient	16m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 58s		the patch passed
+1 💚	compile	2m 24s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 24s		the patch passed
+1 💚	compile	2m 2s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	2m 2s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 50s		the patch passed
+1 💚	mvnsite	0m 57s		the patch passed
+1 💚	javadoc	0m 31s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 29s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 7s		the patch passed
+1 💚	shadedclient	16m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 42s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
-1 ❌	unit	135m 17s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt	hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 34s		The patch does not generate ASF License warnings.
		236m 19s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier
	hadoop.mapreduce.TestMRJobClient

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/10/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 8a43060978c9 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 9d4443f5502d5ff82fedcdbc99571980404d1478
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/10/testReport/
Max. process+thread count	1192 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/10/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-04-08T08:10:45Z

The test failures look unrelated to the patch.

TestJobEndNotifier is failing on trunk. Filed MAPREDUCE-7334.
TestMRJobClient seems flaky. FIled MAPREDUCE-7335

...educe-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java

aajisaka

The regression test looks good to me. I'm +1 if my comments are addressed.

…hange from 3.x to 4.x

hadoop-yetus · 2021-04-08T12:06:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 15s		Maven dependency ordering for branch
-1 ❌	mvninstall	6m 4s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	2m 24s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	2m 0s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	0m 55s		trunk passed
+1 💚	mvnsite	1m 15s		trunk passed
+1 💚	javadoc	0m 50s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 42s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	1m 54s		trunk passed
+1 💚	shadedclient	13m 49s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	2m 17s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	2m 17s		the patch passed
+1 💚	compile	1m 57s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	1m 57s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 49s		the patch passed
+1 💚	mvnsite	1m 1s		the patch passed
+1 💚	javadoc	0m 37s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	0m 34s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 7s		the patch passed
+1 💚	shadedclient	13m 38s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	7m 27s	/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt	hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	131m 12s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		210m 9s

Reason	Tests
Failed junit tests	hadoop.mapred.TestJobEndNotifier

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/11/artifact/out/Dockerfile
GITHUB PR	#2775
JIRA Issue	MAPREDUCE-7329
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 9228899b8f35 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `08790b1`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/11/testReport/
Max. process+thread count	1639 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2775/11/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

lichaojacobs · 2021-04-08T12:13:59Z

It seems like all comments are addressed, and the -1 ❌ | mvninstall is also not related with my patch. FYI @aajisaka

aajisaka · 2021-04-09T02:59:05Z

Merged. Thank you @lichaojacobs and @steveloughran

…hange from 3.x to 4.x (#2775) (cherry picked from commit 663ca14)

…hange from 3.x to 4.x (apache#2775)

lichaojacobs changed the title ~~MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version change from 3.x to 4.x~~ MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version upgrade from 3.x to 4.x Mar 16, 2021

lichaojacobs changed the title ~~MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version upgrade from 3.x to 4.x~~ MAPREDUCE-7329: HadoopPipes task may fail when linux kernel upgrade from 3.x to 4.x Mar 16, 2021

lichaojacobs changed the title ~~MAPREDUCE-7329: HadoopPipes task may fail when linux kernel upgrade from 3.x to 4.x~~ MAPREDUCE-7329: HadoopPipes task failed as a result of ping timeout exception Mar 21, 2021

lichaojacobs changed the title ~~MAPREDUCE-7329: HadoopPipes task failed as a result of ping timeout exception~~ MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception Mar 21, 2021

steveloughran reviewed Mar 29, 2021

View reviewed changes

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch 2 times, most recently from 5dcb605 to c4afa68 Compare March 30, 2021 03:47

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch from c4afa68 to 9fce5d1 Compare April 1, 2021 03:23

aajisaka reviewed Apr 7, 2021

View reviewed changes

...t/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/pipes/Application.java Show resolved Hide resolved

...t/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/pipes/Application.java Outdated Show resolved Hide resolved

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch 2 times, most recently from 459bdcd to 2012f30 Compare April 7, 2021 11:48

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch 2 times, most recently from 92a8f1e to f43e450 Compare April 7, 2021 16:27

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch 2 times, most recently from cebcd85 to 9d4443f Compare April 8, 2021 03:25

aajisaka reviewed Apr 8, 2021

View reviewed changes

...educe-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java Outdated Show resolved Hide resolved

aajisaka reviewed Apr 8, 2021

View reviewed changes

...educe-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java Outdated Show resolved Hide resolved

aajisaka reviewed Apr 8, 2021

View reviewed changes

MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version c…

08790b1

…hange from 3.x to 4.x

lichaojacobs force-pushed the lc_fix_hadoop_pipe_timeout branch from 9d4443f to 08790b1 Compare April 8, 2021 08:34

aajisaka approved these changes Apr 9, 2021

View reviewed changes

aajisaka merged commit 663ca14 into apache:trunk Apr 9, 2021

aajisaka pushed a commit that referenced this pull request Apr 9, 2021

MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version c…

068f114

…hange from 3.x to 4.x (#2775) (cherry picked from commit 663ca14)

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

MAPREDUCE-7329: HadoopPipes task may fail when linux kernel version c…

e8021e6

…hange from 3.x to 4.x (apache#2775)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception #2775

MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception #2775

lichaojacobs commented Mar 15, 2021 •

edited

Loading

hadoop-yetus commented Mar 15, 2021

lichaojacobs commented Mar 16, 2021 •

edited

Loading

liuml07 commented Mar 16, 2021

lichaojacobs commented Mar 16, 2021

lichaojacobs commented Mar 16, 2021

steveloughran commented Mar 17, 2021

lichaojacobs commented Mar 28, 2021

steveloughran left a comment

hadoop-yetus commented Mar 30, 2021

hadoop-yetus commented Mar 30, 2021

hadoop-yetus commented Apr 1, 2021

aajisaka commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 8, 2021

lichaojacobs commented Apr 8, 2021

hadoop-yetus commented Apr 8, 2021

aajisaka commented Apr 8, 2021

aajisaka left a comment

hadoop-yetus commented Apr 8, 2021

lichaojacobs commented Apr 8, 2021 •

edited

Loading

aajisaka commented Apr 9, 2021

MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception #2775

MAPREDUCE-7329: HadoopPipes task has failed because of the ping timeout exception #2775

Conversation

lichaojacobs commented Mar 15, 2021 • edited Loading

hadoop-yetus commented Mar 15, 2021

lichaojacobs commented Mar 16, 2021 • edited Loading

liuml07 commented Mar 16, 2021

lichaojacobs commented Mar 16, 2021

lichaojacobs commented Mar 16, 2021

steveloughran commented Mar 17, 2021

lichaojacobs commented Mar 28, 2021

steveloughran left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Mar 30, 2021

hadoop-yetus commented Mar 30, 2021

hadoop-yetus commented Apr 1, 2021

aajisaka commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

aajisaka commented Apr 7, 2021

lichaojacobs commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 7, 2021

hadoop-yetus commented Apr 8, 2021

lichaojacobs commented Apr 8, 2021

hadoop-yetus commented Apr 8, 2021

aajisaka commented Apr 8, 2021

aajisaka left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Apr 8, 2021

lichaojacobs commented Apr 8, 2021 • edited Loading

aajisaka commented Apr 9, 2021

lichaojacobs commented Mar 15, 2021 •

edited

Loading

lichaojacobs commented Mar 16, 2021 •

edited

Loading

lichaojacobs commented Apr 8, 2021 •

edited

Loading