HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… #4249

comnetwork · 2022-03-20T12:26:04Z

…ntext is null

Apache-HBase · 2022-03-20T13:21:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 54s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 24s	master passed
+1 💚	compile	3m 5s	master passed
+1 💚	checkstyle	0m 53s	master passed
+1 💚	spotbugs	2m 3s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 19s	the patch passed
+1 💚	compile	3m 1s	the patch passed
+1 💚	javac	3m 1s	the patch passed
-0 ⚠️	checkstyle	0m 41s	hbase-server: The patch generated 2 new + 165 unchanged - 0 fixed = 167 total (was 165)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 28s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 7s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 17s	The patch does not generate ASF License warnings.
		40m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4249
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 6db4fa50a634 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `53a7ca7`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-20T16:13:53Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 18s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 21s	master passed
+1 💚	compile	0m 54s	master passed
+1 💚	shadedjars	3m 41s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	0m 54s	the patch passed
+1 💚	javac	0m 54s	the patch passed
+1 💚	shadedjars	3m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 29s	hbase-common in the patch passed.
+1 💚	unit	194m 56s	hbase-server in the patch passed.
		213m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4249
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a1b2d92506c3 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `53a7ca7`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/testReport/
Max. process+thread count	2816 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-21T03:45:13Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 18s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 44s	master passed
+1 💚	compile	3m 31s	master passed
+1 💚	checkstyle	1m 0s	master passed
+1 💚	spotbugs	2m 17s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 45s	the patch passed
+1 💚	compile	3m 34s	the patch passed
+1 💚	javac	3m 34s	the patch passed
+1 💚	checkstyle	1m 1s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	14m 35s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 19s	The patch does not generate ASF License warnings.
		42m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4249
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 53df18ad624d 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0d7638d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-21T06:21:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 19s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 13s	master passed
+1 💚	compile	0m 55s	master passed
+1 💚	shadedjars	3m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 15s	the patch passed
+1 💚	compile	0m 55s	the patch passed
+1 💚	javac	0m 55s	the patch passed
+1 💚	shadedjars	3m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 29s	hbase-common in the patch passed.
-1 ❌	unit	183m 6s	hbase-server in the patch failed.
		201m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4249
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 36183669a566 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0d7638d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/testReport/
Max. process+thread count	2246 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-21T09:14:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 15s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 18s	Maven dependency ordering for branch
+1 💚	mvninstall	5m 33s	master passed
+1 💚	compile	2m 20s	master passed
+1 💚	shadedjars	6m 32s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 11s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 30s	the patch passed
+1 💚	compile	1m 52s	the patch passed
+1 💚	javac	1m 52s	the patch passed
+1 💚	shadedjars	6m 27s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 14s	the patch passed
		_ Other Tests _
+1 💚	unit	3m 40s	hbase-common in the patch passed.
-1 ❌	unit	337m 36s	hbase-server in the patch failed.
		374m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4249
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 13730a65ff56 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0d7638d`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/testReport/
Max. process+thread count	2481 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

sunhelly · 2022-03-23T02:09:43Z

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java

-        results.add(cell instanceof ByteBufferExtendedCell ?
-            KeyValueUtil.copyToNewKeyValue(cell) : cell);
+        results.add(
+          CellUtil.cloneIfNecessary(cell));


one line is ok

sunhelly · 2022-03-23T02:20:33Z

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java

-          addScannerLeaseBack(lease);
+          // If context is null,we call rsh.shippedCallback directly to release the
+          // internal resources in rsh.
+          runShippedCallback(rsh);


The issue here is scanner closed too early then the result bytebuffer might be overwritten. I think the scanner resources are released and lease is added back too, so is it necessary to change codes here?And if there is no rpc call context here, is it a little strange to run a kind of RpcCallBack?

Oh, it also fixed the scanner leak problem caused by the null context and partial results. LGTM.

sunhelly · 2022-03-23T02:22:20Z

Please fix the check style issue.

comnetwork added 2 commits March 20, 2022 20:13

HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo…

075aa79

…ntext is null

fix test

1e085d7

fix checkstyle

3a60e96

comnetwork requested a review from Apache9 March 21, 2022 06:52

lhofhansl approved these changes Mar 21, 2022

View reviewed changes

sunhelly reviewed Mar 23, 2022

View reviewed changes

sunhelly approved these changes Mar 23, 2022

View reviewed changes

add more comments

97aa0ec

comnetwork merged commit ab4351a into apache:master Mar 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… #4249

HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… #4249

Uh oh!

comnetwork commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

sunhelly Mar 23, 2022

Uh oh!

sunhelly Mar 23, 2022

Uh oh!

sunhelly Mar 23, 2022

Uh oh!

sunhelly commented Mar 23, 2022

Uh oh!

Uh oh!

HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… #4249

HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… #4249

Uh oh!

Conversation

comnetwork commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 20, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

Apache-HBase commented Mar 21, 2022

Uh oh!

sunhelly Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

sunhelly Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

sunhelly Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

sunhelly commented Mar 23, 2022

Uh oh!

Uh oh!