HBASE-29397 Deadlock in BufferedMuratorOverAsyncBufferedMutator

[Apache9]

No description provided.

[Copilot]

Pull Request Overview

This pull request addresses the deadlock issue in BufferedMutatorOverAsyncBufferedMutator and improves test coverage for buffered mutator behavior. Key changes include new multi-threaded tests in TestBufferedMutator.java, modifications to test setup/teardown routines, and refactoring of the asynchronous mutation listener in BufferedMutatorOverAsyncBufferedMutator.java.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutator.java	Added multi-thread tests and improved test lifecycle management to better simulate concurrent mutation scenarios.
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorOverAsyncBufferedMutator.java	Removed the synchronized block in the asynchronous mutation listener; however, key operations (bufferedSize decrement and future completion) appear to be omitted.

Comments suppressed due to low confidence (2)

hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorOverAsyncBufferedMutator.java:137

• The removal of the synchronized block also removed critical operations: decrementing bufferedSize, handling errors, and completing the future. These operations are essential to ensuring that mutation processing and internal flush behavior work correctly. Please revert or rework the changes to include these steps.

addListener(fs.get(i), (r, e) -> {

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutator.java:86

• [nitpick] The test method name 'test' is ambiguous. Consider renaming it to a more descriptive name (e.g., testBufferedMutatorSimple) to better reflect its purpose.

@Test public void test() throws Exception {

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 11s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 14s		master passed
+1 💚	compile	4m 3s		master passed
+1 💚	checkstyle	0m 53s		master passed
+1 💚	spotbugs	2m 13s		master passed
+1 💚	spotless	0m 48s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 8s		the patch passed
+1 💚	compile	4m 6s		the patch passed
+1 💚	javac	4m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 52s		the patch passed
+1 💚	spotbugs	2m 26s		the patch passed
+1 💚	hadoopcheck	11m 59s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 44s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 18s		The patch does not generate ASF License warnings.
		43m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7107/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7107
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux b5180be6c6ca 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 62490ff
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7107/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
-0 ⚠️	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 10s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 18s		master passed
+1 💚	compile	1m 19s		master passed
+1 💚	javadoc	0m 45s		master passed
+1 💚	shadedjars	6m 6s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 9s		the patch passed
+1 💚	compile	1m 20s		the patch passed
+1 💚	javac	1m 20s		the patch passed
+1 💚	javadoc	0m 45s		the patch passed
+1 💚	shadedjars	6m 3s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 35s		hbase-client in the patch passed.
+1 💚	unit	233m 33s		hbase-server in the patch passed.
		264m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7107/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7107
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux adabafcbb327 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 62490ff
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7107/1/testReport/
Max. process+thread count	4431 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7107/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

We need to get this in immediately since it the bug affects bunch of tests under hbase-mapreduce...

With this patch in place, I've successfully finished a 10B ITBLL run. Please HBASE-27919.

Thanks.

[ndimiduk]

Seems alright to me. Nice avoidance of synchronized block.

[Apache9]

Seems alright to me. Nice avoidance of synchronized block.

The synchronized block was added recently in HBASE-29394, which is incorrect...

Thanks Nick. Let me merge...