HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes #2026

vinayakumarb · 2020-05-18T06:40:33Z

After upgrade/shade of protobuf to 3.7 version, existing Hadoop-Rpc client-server implementations using ProtobufRpcEngine will not work.

So, this Jira proposes to keep existing ProtobuRpcEngine as-is (without shading and with protobuf-2.5.0 implementation) to support downstream implementations.

Use new ProtobufRpcEngine2 to use shaded protobuf classes within Hadoop and later projects who wish to upgrade their protobufs to 3.x.

steveloughran · 2020-05-19T16:21:18Z

way beyond my area of expertise, i'm afraid

hadoop-yetus · 2020-05-19T19:40:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	shelldocs	0m 2s	Shelldocs was not available.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 22 new or modified test files.
		_ trunk Compile Tests _
+0 🆗	mvndep	1m 0s	Maven dependency ordering for branch
+1 💚	mvninstall	21m 52s	trunk passed
+1 💚	compile	18m 11s	trunk passed
+1 💚	checkstyle	3m 23s	trunk passed
+1 💚	mvnsite	18m 4s	trunk passed
+1 💚	shadedclient	15m 22s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	5m 33s	trunk passed
+0 🆗	spotbugs	32m 24s	Used deprecated FindBugs config; considering switching to SpotBugs.
-1 ❌	findbugs	1m 40s	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common in trunk has 1 extant findbugs warnings.
-1 ❌	findbugs	32m 19s	root in trunk has 4 extant findbugs warnings.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s	Maven dependency ordering for patch
+1 💚	mvninstall	28m 51s	the patch passed
+1 💚	compile	17m 40s	the patch passed
-1 ❌	cc	17m 40s	root generated 25 new + 137 unchanged - 25 fixed = 162 total (was 162)
+1 💚	javac	17m 40s	the patch passed
-0 ⚠️	checkstyle	3m 19s	root: The patch generated 39 new + 1370 unchanged - 12 fixed = 1409 total (was 1382)
+1 💚	mvnsite	18m 6s	the patch passed
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
-1 ❌	whitespace	0m 0s	The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+1 💚	shadedclient	15m 26s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	5m 37s	the patch passed
-1 ❌	findbugs	2m 12s	hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌	findbugs	32m 35s	root generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4)
		_ Other Tests _
-1 ❌	unit	612m 34s	root in the patch passed.
-1 ❌	asflicense	1m 21s	The patch generated 1 ASF License warnings.
		894m 3s

Reason	Tests
FindBugs	module:hadoop-common-project/hadoop-common
	Useless control flow in org.apache.hadoop.ipc.protobuf.ProtobufRpcEngine2Protos$RequestHeaderProto$Builder.maybeForceBuilderInitialization() At ProtobufRpcEngine2Protos.java: At ProtobufRpcEngine2Protos.java:[line 643]
FindBugs	module:root
	Useless control flow in org.apache.hadoop.ipc.protobuf.ProtobufRpcEngine2Protos$RequestHeaderProto$Builder.maybeForceBuilderInitialization() At ProtobufRpcEngine2Protos.java: At ProtobufRpcEngine2Protos.java:[line 643]
Failed junit tests	hadoop.yarn.applications.distributedshell.TestDistributedShell
	hadoop.security.TestRaceWhenRelogin
	hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
	hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
	hadoop.hdfs.TestRollingUpgrade
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
	hadoop.yarn.sls.TestSLSRunner

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/Dockerfile
GITHUB PR	#2026
Optional Tests	dupname asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool
uname	Linux bd63306be0e9 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `bdbd59c`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-warnings.html
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/branch-findbugs-root-warnings.html
cc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/diff-compile-cc-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/diff-checkstyle-root.txt
whitespace	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/whitespace-eol.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/new-findbugs-root.html
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/patch-unit-root.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/testReport/
asflicense	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	2861 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java

...mon-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngineCallback.java

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java

vinayakumarb · 2020-06-03T08:55:24Z

Updated the PR with review comment fixes. + below changes
ProtobufRpcEngineProto.java is generated using protobuf 2.5.0, which does not available for arm. So I have added the generated ProtobufRpcEngineProto.java in a separare source directory in hadoop-common\src\main\arm-java and adding this to sources in case of arm. But Re-generating using protobuf-maven-plugin in case of x86 platform as before.

hadoop-yetus · 2020-06-03T23:24:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 44s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	shelldocs	0m 1s	Shelldocs was not available.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 23 new or modified test files.
		_ trunk Compile Tests _
+0 🆗	mvndep	0m 34s	Maven dependency ordering for branch
+1 💚	mvninstall	20m 19s	trunk passed
+1 💚	compile	17m 5s	trunk passed
+1 💚	checkstyle	3m 19s	trunk passed
+1 💚	mvnsite	17m 19s	trunk passed
+1 💚	shadedclient	14m 1s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	5m 39s	trunk passed
+0 🆗	spotbugs	29m 21s	Used deprecated FindBugs config; considering switching to SpotBugs.
-1 ❌	findbugs	29m 16s	root in trunk has 3 extant findbugs warnings.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s	Maven dependency ordering for patch
+1 💚	mvninstall	27m 40s	the patch passed
+1 💚	compile	16m 41s	the patch passed
-1 ❌	cc	16m 41s	root generated 31 new + 131 unchanged - 31 fixed = 162 total (was 162)
-1 ❌	javac	16m 41s	root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857)
-0 ⚠️	checkstyle	3m 26s	root: The patch generated 12 new + 1400 unchanged - 13 fixed = 1412 total (was 1413)
+1 💚	mvnsite	17m 29s	the patch passed
+1 💚	shellcheck	0m 1s	There were no new shellcheck issues.
-1 ❌	whitespace	0m 1s	The patch has 103 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	xml	0m 4s	The patch has no ill-formed XML file.
+1 💚	shadedclient	14m 13s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	5m 38s	the patch passed
+1 💚	findbugs	48m 51s	the patch passed
		_ Other Tests _
-1 ❌	unit	604m 55s	root in the patch passed.
-1 ❌	asflicense	1m 52s	The patch generated 1 ASF License warnings.
		872m 53s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.hdfs.TestGetFileChecksum
	hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR
	hadoop.hdfs.server.datanode.TestBPOfferService
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.tools.TestDFSAdminWithHA
	hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
	hadoop.tools.TestDistCpSync
	hadoop.tools.TestDistCpSystem
	hadoop.yarn.applications.distributedshell.TestDistributedShell
	hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/Dockerfile
GITHUB PR	#2026
Optional Tests	dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool
uname	Linux fe8dfdd17c03 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `e8cb2ae`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/branch-findbugs-root-warnings.html
cc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-compile-cc-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-compile-javac-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-checkstyle-root.txt
whitespace	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/whitespace-eol.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/patch-unit-root.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/testReport/
asflicense	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	3839 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hadoop-common-project/hadoop-common/pom.xml

hadoop-yetus · 2020-06-08T10:44:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	27m 53s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 23 new or modified test files.
		_ trunk Compile Tests _
+0 🆗	mvndep	1m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	24m 20s	trunk passed
+1 💚	compile	21m 36s	trunk passed
+1 💚	checkstyle	3m 39s	trunk passed
+1 💚	mvnsite	21m 12s	trunk passed
+1 💚	shadedclient	16m 25s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	6m 52s	trunk passed
+0 🆗	spotbugs	39m 7s	Used deprecated FindBugs config; considering switching to SpotBugs.
-1 ❌	findbugs	39m 1s	root in trunk has 3 extant findbugs warnings.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s	Maven dependency ordering for patch
+1 💚	mvninstall	33m 23s	the patch passed
+1 💚	compile	20m 42s	the patch passed
-1 ❌	cc	20m 42s	root generated 4 new + 158 unchanged - 4 fixed = 162 total (was 162)
-1 ❌	javac	20m 42s	root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857)
-0 ⚠️	checkstyle	3m 36s	root: The patch generated 2 new + 1401 unchanged - 13 fixed = 1403 total (was 1414)
+1 💚	mvnsite	21m 55s	the patch passed
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	shelldocs	0m 12s	There were no new shelldocs issues.
-1 ❌	whitespace	0m 0s	The patch has 103 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	xml	0m 4s	The patch has no ill-formed XML file.
+1 💚	shadedclient	16m 56s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	6m 40s	the patch passed
+1 💚	findbugs	63m 44s	the patch passed
		_ Other Tests _
-1 ❌	unit	19m 32s	root in the patch failed.
-1 ❌	asflicense	1m 3s	The patch generated 1 ASF License warnings.
		377m 0s

Reason	Tests
Failed junit tests	hadoop.security.TestRaceWhenRelogin
	hadoop.util.TestDataChecksum
	hadoop.util.curator.TestChildReaper

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/Dockerfile
GITHUB PR	#2026
Optional Tests	dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool
uname	Linux 4437dd23e314 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `a8610c1`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/branch-findbugs-root-warnings.html
cc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-compile-cc-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-compile-javac-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-checkstyle-root.txt
whitespace	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/whitespace-eol.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/patch-unit-root.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/testReport/
asflicense	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	3204 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

...educe-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java

aajisaka · 2020-06-09T10:21:58Z

Now I'm +1. Hi @jojochuang could you review this?

vinayakumarb · 2020-06-10T04:37:45Z

I think, by mistake I had closed this PR, while replying to comment. Not able to re-open.
Will raise a new PR for this. Please review.

vinayakumarb · 2020-06-10T04:43:46Z

I think, by mistake I had closed this PR, while replying to comment. Not able to re-open.
Will raise a new PR for this. Please review.
Had pushed trunk itself without mychanges to source branch, so PR got closed.
Now its back.

ayushtkn

Thanx @vinayakumarb for the fix.
Jenkins has complains regarding whitespace, give a check if that can be fixed.
Apart LGTM +1

hadoop-yetus · 2020-06-10T21:16:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 22 new or modified test files.
		_ trunk Compile Tests _
+0 🆗	mvndep	1m 8s	Maven dependency ordering for branch
+1 💚	mvninstall	21m 37s	trunk passed
+1 💚	compile	18m 18s	trunk passed
+1 💚	checkstyle	3m 19s	trunk passed
+1 💚	mvnsite	20m 19s	trunk passed
+1 💚	shadedclient	16m 35s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	6m 42s	trunk passed
+0 🆗	spotbugs	39m 52s	Used deprecated FindBugs config; considering switching to SpotBugs.
-1 ❌	findbugs	39m 40s	root in trunk has 3 extant findbugs warnings.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s	Maven dependency ordering for patch
+1 💚	mvninstall	34m 0s	the patch passed
+1 💚	compile	21m 5s	the patch passed
-1 ❌	cc	21m 5s	root generated 20 new + 142 unchanged - 20 fixed = 162 total (was 162)
-1 ❌	javac	21m 5s	root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857)
-0 ⚠️	checkstyle	3m 51s	root: The patch generated 2 new + 1369 unchanged - 13 fixed = 1371 total (was 1382)
+1 💚	mvnsite	20m 54s	the patch passed
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	shelldocs	0m 15s	There were no new shelldocs issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 4s	The patch has no ill-formed XML file.
+1 💚	shadedclient	16m 32s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	6m 32s	the patch passed
+1 💚	findbugs	62m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	670m 34s	root in the patch passed.
-1 ❌	asflicense	1m 47s	The patch generated 1 ASF License warnings.
		992m 57s

Reason	Tests
Failed junit tests	hadoop.mapreduce.v2.app.rm.TestRMCommunicator
	hadoop.yarn.server.timelineservice.storage.TestTimelineWriterHBaseDown
	hadoop.yarn.applications.distributedshell.TestDistributedShell
	hadoop.security.TestRaceWhenRelogin
	hadoop.hdfs.server.namenode.TestINodeFile
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
	hadoop.hdfs.TestDecommissionWithStriped
	hadoop.hdfs.server.namenode.TestAddStripedBlocks
	hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.server.datanode.TestDirectoryScanner
	hadoop.hdfs.server.balancer.TestBalancerRPCDelay
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
	hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/Dockerfile
GITHUB PR	#2026
Optional Tests	dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool
uname	Linux 80092813a2cb 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `56247db`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/branch-findbugs-root-warnings.html
cc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-compile-cc-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-compile-javac-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-checkstyle-root.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/patch-unit-root.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/testReport/
asflicense	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	3086 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/console
versions	git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…s using non-shaded protobuf classes (apache#2026)

hadoop-yetus · 2020-06-12T17:48:06Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 5s	#2026 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.

Subsystem	Report/Notes
GITHUB PR	#2026
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/6/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…s using non-shaded protobuf classes (#2026)

…s using non-shaded protobuf classes (apache#2026)

vinayakumarb requested review from steveloughran and jojochuang May 18, 2020 06:40

vinayakumarb force-pushed the bugs/HADOOP-17046 branch from bd8509e to 397300e Compare May 19, 2020 04:45

aajisaka reviewed May 28, 2020

View reviewed changes

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java Show resolved Hide resolved

aajisaka reviewed May 28, 2020

View reviewed changes

...mon-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngineCallback.java Show resolved Hide resolved

aajisaka reviewed May 28, 2020

View reviewed changes

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java Show resolved Hide resolved

vinayakumarb force-pushed the bugs/HADOOP-17046 branch from 397300e to 23384ec Compare June 3, 2020 08:50

aajisaka reviewed Jun 4, 2020

View reviewed changes

hadoop-common-project/hadoop-common/pom.xml Outdated Show resolved Hide resolved

aajisaka reviewed Jun 4, 2020

View reviewed changes

hadoop-common-project/hadoop-common/pom.xml Outdated Show resolved Hide resolved

vinayakumarb force-pushed the bugs/HADOOP-17046 branch from 23384ec to 75f47ac Compare June 8, 2020 04:26

aajisaka reviewed Jun 9, 2020

View reviewed changes

...educe-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java Outdated Show resolved Hide resolved

steveloughran removed their request for review June 9, 2020 13:45

vinayakumarb closed this Jun 10, 2020

vinayakumarb force-pushed the bugs/HADOOP-17046 branch from 75f47ac to 56247db Compare June 10, 2020 04:28

vinayakumarb reopened this Jun 10, 2020

ayushtkn approved these changes Jun 10, 2020

View reviewed changes

aajisaka self-requested a review June 12, 2020 06:52

aajisaka approved these changes Jun 12, 2020

View reviewed changes

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementation…

8ff3758

…s using non-shaded protobuf classes (apache#2026)

vinayakumarb force-pushed the bugs/HADOOP-17046 branch from f4f0051 to 8ff3758 Compare June 12, 2020 17:45

vinayakumarb merged commit e154084 into apache:trunk Jun 12, 2020

asfgit pushed a commit that referenced this pull request Jun 12, 2020

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementation…

e6887bd

…s using non-shaded protobuf classes (#2026)

asfgit pushed a commit that referenced this pull request Jun 12, 2020

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementation…

534b15c

…s using non-shaded protobuf classes (#2026)

steveloughran mentioned this pull request Oct 12, 2022

HADOOP-18487. protobuf 2.5.0 marked as provided. #4996

Merged

4 tasks

zhangxiping1 pushed a commit to zhangxiping1/hadoop that referenced this pull request Dec 13, 2022

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementation…

69c8e04

…s using non-shaded protobuf classes (apache#2026)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes #2026

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes #2026

Uh oh!

vinayakumarb commented May 18, 2020

Uh oh!

steveloughran commented May 19, 2020

Uh oh!

hadoop-yetus commented May 19, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vinayakumarb commented Jun 3, 2020

Uh oh!

hadoop-yetus commented Jun 3, 2020

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jun 8, 2020

Uh oh!

Uh oh!

aajisaka commented Jun 9, 2020

Uh oh!

vinayakumarb commented Jun 10, 2020

Uh oh!

vinayakumarb commented Jun 10, 2020

Uh oh!

ayushtkn left a comment

Uh oh!

hadoop-yetus commented Jun 10, 2020

Uh oh!

hadoop-yetus commented Jun 12, 2020

Uh oh!

Uh oh!

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes #2026

HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes #2026

Uh oh!

Conversation

vinayakumarb commented May 18, 2020

Uh oh!

steveloughran commented May 19, 2020

Uh oh!

hadoop-yetus commented May 19, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vinayakumarb commented Jun 3, 2020

Uh oh!

hadoop-yetus commented Jun 3, 2020

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jun 8, 2020

Uh oh!

Uh oh!

aajisaka commented Jun 9, 2020

Uh oh!

vinayakumarb commented Jun 10, 2020

Uh oh!

vinayakumarb commented Jun 10, 2020

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jun 10, 2020

Uh oh!

hadoop-yetus commented Jun 12, 2020

Uh oh!

Uh oh!