HBASE-28965 Make the approach in HBASE-28955 can work together with h… #6450

Apache9 · 2024-11-07T16:00:28Z

…adoop 2.x

Apache9 · 2024-11-07T16:00:40Z

Let's have a try.

stoty · 2024-11-07T18:33:32Z

...e-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutput.java

+
+  private DFSOutputStream tryCreateDummyDFSOutputStream(EnumSet<CreateFlag> createFlags) {
+    try {
+      return (DFSOutputStream) Class.forName(DUMMY_DFS_OUTPUT_STREAM_CLASS).getConstructors()[0]


Push this down to the helper, and check (and store the resolved class and constructor handle) if the class exists when initializing the helper just like we do for the other Hadoop-dependent cases ?

Should be faster.

...e-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutput.java

hbase-asyncfs/pom.xml

…adoop 2.x

Apache-HBase · 2024-11-13T02:57:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
-0 ⚠️	yetus	0m 4s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	2m 29s		branch-2 passed
+1 💚	compile	0m 36s		branch-2 passed
+1 💚	javadoc	0m 33s		branch-2 passed
+1 💚	shadedjars	4m 49s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 25s		the patch passed
+1 💚	compile	0m 36s		the patch passed
+1 💚	javac	0m 36s		the patch passed
+1 💚	javadoc	0m 33s		the patch passed
+1 💚	shadedjars	4m 48s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 57s		hbase-asyncfs in the patch passed.
+1 💚	unit	0m 10s		hbase-shaded-check-invariants in the patch passed.
+1 💚	unit	0m 11s		hbase-shaded-with-hadoop-check-invariants in the patch passed.
		21m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6450
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 66d19721dbfa 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `d6aef55`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/testReport/
Max. process+thread count	457 (vs. ulimit of 30000)
modules	C: hbase-asyncfs hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-11-13T03:00:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
-0 ⚠️	yetus	0m 4s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+0 🆗	mvndep	0m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	2m 53s		branch-2 passed
+1 💚	compile	0m 40s		branch-2 passed
+1 💚	javadoc	0m 37s		branch-2 passed
+1 💚	shadedjars	5m 35s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 5s		the patch passed
+1 💚	compile	0m 39s		the patch passed
+1 💚	javac	0m 39s		the patch passed
+1 💚	javadoc	0m 37s		the patch passed
+1 💚	shadedjars	5m 36s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 34s		hbase-asyncfs in the patch passed.
+1 💚	unit	0m 11s		hbase-shaded-check-invariants in the patch passed.
+1 💚	unit	0m 11s		hbase-shaded-with-hadoop-check-invariants in the patch passed.
		24m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6450
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 3c040cdbe2c3 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `d6aef55`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/testReport/
Max. process+thread count	784 (vs. ulimit of 30000)
modules	C: hbase-asyncfs hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-11-13T03:00:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 17s		branch-2 passed
+1 💚	compile	0m 33s		branch-2 passed
+1 💚	javadoc	0m 31s		branch-2 passed
+1 💚	shadedjars	6m 9s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 13s		the patch passed
+1 💚	compile	0m 33s		the patch passed
+1 💚	javac	0m 33s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed
+1 💚	shadedjars	6m 8s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 4s		hbase-asyncfs in the patch passed.
+1 💚	unit	0m 9s		hbase-shaded-check-invariants in the patch passed.
+1 💚	unit	0m 9s		hbase-shaded-with-hadoop-check-invariants in the patch passed.
		24m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6450
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 60ab27dc79fc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `d6aef55`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/testReport/
Max. process+thread count	607 (vs. ulimit of 30000)
modules	C: hbase-asyncfs hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-11-13T03:08:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+0 🆗	mvndep	0m 52s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 33s		branch-2 passed
+1 💚	compile	0m 41s		branch-2 passed
+1 💚	checkstyle	0m 23s		branch-2 passed
+1 💚	spotbugs	0m 28s		branch-2 passed
+1 💚	spotless	0m 48s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 4s		the patch passed
+1 💚	compile	0m 38s		the patch passed
-0 ⚠️	javac	0m 19s	/results-compile-javac-hbase-asyncfs.txt	hbase-asyncfs generated 2 new + 18 unchanged - 0 fixed = 20 total (was 18)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 23s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	spotbugs	0m 33s		the patch passed
+1 💚	hadoopcheck	17m 4s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 26s		The patch does not generate ASF License warnings.
		32m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6450
Optional Tests	dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti shellcheck shelldocs
uname	Linux c00ee9c2c91c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `d6aef55`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-asyncfs hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6450/5/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

stoty

+1 LGTM

NihalJain · 2024-11-15T06:52:30Z

hbase-asyncfs/pom.xml

+                <exclude>**/org/apache/hadoop/hdfs/**</exclude>
+              </excludes>
+              <testExcludes>
+                <testExclude>**/org/apache/hadoop/hbase/io/asyncfs/TestLeaseRenewal**</testExclude>


Very nice. I could use this approach to add feature HBASE-27693 Support for Hadoop's LDAP Authentication mechanism (Web UI only) i.e. #5680 to branch-2!

…adoop 2.x (#6450) Signed-off-by: Istvan Toth <stoty@apache.org> (cherry picked from commit e4f5d55)

Apache9 self-assigned this Nov 7, 2024

This comment has been minimized.

Sign in to view

stoty reviewed Nov 7, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

Apache9 force-pushed the HBASE-28965-branch-2 branch from 010f564 to a925d4c Compare November 12, 2024 12:18

This comment has been minimized.

Sign in to view

HBASE-28965 Make the approach in HBASE-28955 can work together with h…

d6aef55

…adoop 2.x

Apache9 force-pushed the HBASE-28965-branch-2 branch from e0cd899 to d6aef55 Compare November 13, 2024 02:27

Apache9 requested a review from stoty November 13, 2024 02:27

stoty approved these changes Nov 15, 2024

View reviewed changes

NihalJain reviewed Nov 15, 2024

View reviewed changes

Apache9 merged commit e4f5d55 into apache:branch-2 Nov 15, 2024
1 check passed

Apache9 added a commit that referenced this pull request Nov 15, 2024

HBASE-28965 Make the approach in HBASE-28955 can work together with h…

ea765ef

…adoop 2.x (#6450) Signed-off-by: Istvan Toth <stoty@apache.org> (cherry picked from commit e4f5d55)

Apache9 added a commit that referenced this pull request Nov 15, 2024

HBASE-28965 Make the approach in HBASE-28955 can work together with h…

baea7c0

…adoop 2.x (#6450) Signed-off-by: Istvan Toth <stoty@apache.org> (cherry picked from commit e4f5d55)

HBASE-28965 Make the approach in HBASE-28955 can work together with h… #6450

HBASE-28965 Make the approach in HBASE-28955 can work together with h… #6450

Uh oh!

Conversation

Apache9 commented Nov 7, 2024

Uh oh!

Apache9 commented Nov 7, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

stoty Nov 7, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Nov 13, 2024

Uh oh!

Apache-HBase commented Nov 13, 2024

Uh oh!

Apache-HBase commented Nov 13, 2024

Uh oh!

Apache-HBase commented Nov 13, 2024

Uh oh!

stoty left a comment

Choose a reason for hiding this comment

Uh oh!

NihalJain Nov 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

NihalJain Nov 15, 2024 •

edited

Loading