HADOOP-16888. [JDK11] Support JDK11 in the precommit job #2012

aajisaka · 2020-05-12T17:03:39Z

JIRA: https://issues.apache.org/jira/browse/HADOOP-16888

Install JDK 11 in the build image.
Run mvninstall, compile, javac, and javadoc with JDK 8 and JDK 11. (Now javadoc with JDK 11 is broken.)
Run the unit tests with JDK 8 only. This is because it takes too long to run unit tests with both JDK 8 and JDK 11.

After that, I'll prepare daily qbt jobs for JDK 11. The job will run full unit tests with JDK 11.

hadoop-yetus · 2020-05-12T17:06:07Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/1/console in case of problems.

hadoop-yetus · 2020-05-12T18:15:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	29m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
-1 ❌	shadedclient	19m 24s	branch has errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 2s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
-1 ❌	shadedclient	19m 14s	patch has errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		70m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/1/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux 22f82500c8bf 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `936bf09`
Max. process+thread count	315 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/1/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

aajisaka · 2020-05-12T19:21:36Z

shaded client test is failing. Probably this is because JAVA_HOME is set to Java 11.

https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/1/console

02:35:43  WARNING: Setting /usr/lib/jvm/java-11-openjdk-amd64 as the JAVA_HOME.

hadoop-yetus · 2020-05-12T19:26:40Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/2/console in case of problems.

hadoop-yetus · 2020-05-12T20:01:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 44s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	15m 59s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 3s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 48s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		35m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/2/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux c38e30b0faf0 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `d60496e`
Max. process+thread count	309 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/2/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

aajisaka · 2020-05-13T02:01:37Z

The build timeout in #2013.

hadoop-yetus · 2020-05-13T02:16:36Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/3/console in case of problems.

hadoop-yetus · 2020-05-13T02:50:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	15m 27s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 2s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 40s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 29s	The patch does not generate ASF License warnings.
		34m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/3/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux bd7715e4f38d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `d60496e`
Max. process+thread count	333 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/3/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-05-18T08:25:07Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/4/console in case of problems.

hadoop-yetus · 2020-05-18T09:30:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	29m 13s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	18m 1s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 4s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	17m 3s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 36s	The patch does not generate ASF License warnings.
		67m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/4/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux 65d0f23ad560 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `b65815d`
Max. process+thread count	414 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/4/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-05-19T01:34:11Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/5/console in case of problems.

hadoop-yetus · 2020-05-19T02:32:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	26m 4s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	16m 0s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 3s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 12s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 31s	The patch does not generate ASF License warnings.
		59m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/5/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux 7066bf7bb680 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `7bb902b`
Max. process+thread count	340 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/5/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-05-20T07:14:36Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/6/console in case of problems.

hadoop-yetus · 2020-05-20T08:06:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	22m 46s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	14m 18s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 3s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 46s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 35s	The patch does not generate ASF License warnings.
		53m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/6/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux 2842cc4f4a40 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `cef0756`
Max. process+thread count	414 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/6/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

aajisaka · 2020-05-24T08:08:43Z

Full tests ran and it took 906 minutes. It seems that the timeout should be increased > 15 hours.
#2013 (comment)

hadoop-yetus · 2020-05-24T08:15:21Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/7/console in case of problems.

aajisaka · 2020-05-24T08:22:17Z

Now I think the PR is ready for merge.

hadoop-yetus · 2020-05-24T09:15:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	27m 15s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ trunk Compile Tests _
+1 💚	shadedclient	15m 58s	branch has no errors when building and testing our client artifacts.
		_ Patch Compile Tests _
+1 💚	hadolint	0m 3s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 42s	patch has no errors when building and testing our client artifacts.
		_ Other Tests _
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		61m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/7/artifact/out/Dockerfile
GITHUB PR	#2012
Optional Tests	dupname asflicense shellcheck shelldocs hadolint
uname	Linux 048c2aa08103 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6e04b00`
Max. process+thread count	309 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/7/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 hadolint=1.11.1-0-g0e692dd
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn

Thanx @aajisaka for the efforts here.
changes LGTM +1

I suppose the timeout is for testing purpose only and you will be reverting it, before pushing

aajisaka · 2020-06-19T04:26:14Z

Thank you @ayushtkn
Reverted the timeout change. I'll merge this.

hadoop-yetus · 2020-06-19T04:27:28Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/8/console in case of problems.

hadoop-yetus · 2020-06-19T04:55:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	28m 0s	Docker mode activated.
-1 ❌	patch	0m 5s	#2012 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/8/artifact/out/Dockerfile
GITHUB PR	#2012
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2012/8/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

aajisaka added 3 commits May 13, 2020 01:53

Refactor Dockerfile

1cad9ef

Add Java 11 test in GitHub PR

fc7d9e6

Correct the order

1862abd

aajisaka mentioned this pull request May 12, 2020

[DO NOT MERGE] Testing HADOOP-16888 #2013

Closed

Set Java home to JDK 8 explicitly

2752cdb

Increase the timeout

11558af

Increase the timeout to 24 hours.

c6de145

Stop running unit tests in Java 11.

9a3526c

Increase timeout

cb289d0

Set timeout to 20 hours.

7a4aba6

aajisaka requested review from ayushtkn and steveloughran June 11, 2020 04:34

aajisaka requested review from vinayakumarb and kihwal June 11, 2020 04:34

ayushtkn approved these changes Jun 11, 2020

View reviewed changes

Revert the timeout change

8cb30d1

aajisaka merged commit 9821b94 into apache:trunk Jun 19, 2020

aajisaka deleted the HADOOP-16888 branch June 19, 2020 04:29

HADOOP-16888. [JDK11] Support JDK11 in the precommit job #2012

HADOOP-16888. [JDK11] Support JDK11 in the precommit job #2012

Uh oh!

Conversation

aajisaka commented May 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented May 12, 2020

Uh oh!

hadoop-yetus commented May 12, 2020

Uh oh!

aajisaka commented May 12, 2020

Uh oh!

hadoop-yetus commented May 12, 2020

Uh oh!

hadoop-yetus commented May 12, 2020

Uh oh!

aajisaka commented May 13, 2020

Uh oh!

hadoop-yetus commented May 13, 2020

Uh oh!

hadoop-yetus commented May 13, 2020

Uh oh!

hadoop-yetus commented May 18, 2020

Uh oh!

hadoop-yetus commented May 18, 2020

Uh oh!

hadoop-yetus commented May 19, 2020

Uh oh!

hadoop-yetus commented May 19, 2020

Uh oh!

hadoop-yetus commented May 20, 2020

Uh oh!

hadoop-yetus commented May 20, 2020

Uh oh!

aajisaka commented May 24, 2020

Uh oh!

hadoop-yetus commented May 24, 2020

Uh oh!

aajisaka commented May 24, 2020

Uh oh!

hadoop-yetus commented May 24, 2020

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

aajisaka commented Jun 19, 2020

Uh oh!

hadoop-yetus commented Jun 19, 2020

Uh oh!

hadoop-yetus commented Jun 19, 2020

Uh oh!

Uh oh!

aajisaka commented May 12, 2020 •

edited

Loading