HBASE-27095 HbckChore should produce a report #4498

ndimiduk · 2022-06-07T13:56:23Z

In #4470 for HBASE-26192, it was noted that the HbckChore is kind of a pain to use and test because it maintains a bunch of local state. By contract, the CatalogJanitorChore makes a nice self-contained report. Let's update HbckChore to do the same.

apurtell

Nice refactor.

virajjasani · 2022-06-08T20:04:29Z

Re-triggered the build.
Thanks @ndimiduk for the changes!

ndimiduk · 2022-06-09T11:13:39Z

Shadedjars failure looks like a build environment hiccup:

[INFO] Apache HBase Shaded Packaging Invariants ........... SUCCESS [ 13.049 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:36 min
[INFO] Finished at: 2022-06-08T20:40:42Z
[INFO] ------------------------------------------------------------------------
[ERROR] An internal error occurred during: "Periodic workspace save.".
java.lang.IllegalStateException: Job manager has been shut down.
    at org.eclipse.core.internal.jobs.JobManager.schedule (JobManager.java:1295)
    at org.eclipse.core.internal.jobs.InternalJob.schedule (InternalJob.java:385)
    at org.eclipse.core.runtime.jobs.Job.schedule (Job.java:684)
    at org.eclipse.core.internal.events.AutoBuildJob.build (AutoBuildJob.java:110)
    at org.eclipse.core.internal.events.BuildManager.endTopLevel (BuildManager.java:604)
    at org.eclipse.core.internal.resources.Workspace.endOperation (Workspace.java:1518)
    at org.eclipse.core.internal.resources.SaveManager.save (SaveManager.java:1246)
    at org.eclipse.core.internal.resources.SaveManager.save (SaveManager.java:1143)
    at org.eclipse.core.internal.resources.DelayedSnapshotJob.run (DelayedSnapshotJob.java:55)
    at org.eclipse.core.internal.jobs.Worker.run (Worker.java:63)

Done because we want to make HbckChore also generate a report object, and the results of both chores are consumed in a shared context (hbck.jsp). Give the classes different names so that they can be used in the same scope. Done as a separate commit to make things easier to review.

In apache#4470 for HBASE-26192, it was noted that the HbckChore is kind of a pain to use and test because it maintains a bunch of local state. By contract, the CatalogJanitorChore makes a nice self-contained report. Let's update HbckChore to do the same. Signed-off-by: Andrew Purtell <apurtell@apache.org>

Apache-HBase · 2022-06-09T12:00:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 5s	master passed
+1 💚	compile	2m 13s	master passed
+1 💚	checkstyle	0m 29s	master passed
+1 💚	spotless	0m 38s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 14s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 28s	the patch passed
+1 💚	compile	2m 13s	the patch passed
+1 💚	javac	2m 13s	the patch passed
+1 💚	checkstyle	0m 28s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	13m 35s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotless	0m 39s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 19s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		34m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4498
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 030974c98d54 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `414cfb3`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	68 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-09T14:59:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 3s	master passed
+1 💚	compile	0m 45s	master passed
+1 💚	shadedjars	4m 37s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 8s	the patch passed
+1 💚	compile	0m 45s	the patch passed
+1 💚	javac	0m 45s	the patch passed
+1 💚	shadedjars	4m 35s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	193m 55s	hbase-server in the patch passed.
		213m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4498
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 873f363dcf9c 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `414cfb3`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/testReport/
Max. process+thread count	2516 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-09T15:34:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 54s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	4m 12s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 41s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	4m 12s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	231m 15s	hbase-server in the patch failed.
		249m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4498
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 84e304e4fdc2 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `414cfb3`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/testReport/
Max. process+thread count	2203 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4498/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk requested review from saintstack, apurtell, infraio, Apache9 and virajjasani June 7, 2022 13:56

ndimiduk mentioned this pull request Jun 7, 2022

HBASE-26192 Master UI hbck should provide a JSON formatted output option #4470

Closed

This comment was marked as outdated.

Sign in to view

apurtell approved these changes Jun 7, 2022

View reviewed changes

This comment was marked as outdated.

Sign in to view

ndimiduk added 2 commits June 9, 2022 13:14

ndimiduk force-pushed the 27095-hbckchore-report branch from 3e0717e to 4fc1f37 Compare June 9, 2022 11:16

This was referenced Jun 9, 2022

Backport "HBASE-27095 HbckChore should produce a report" to branch-2 #4509

Merged

Backport "HBASE-27095 HbckChore should produce a report" to branch-2.5 #4510

Merged

Backport "HBASE-27095 HbckChore should produce a report" to branch-2.4 #4511

Merged

ndimiduk merged commit e191fe9 into apache:master Jun 10, 2022

ndimiduk deleted the 27095-hbckchore-report branch June 10, 2022 13:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-27095 HbckChore should produce a report #4498

HBASE-27095 HbckChore should produce a report #4498

Uh oh!

ndimiduk commented Jun 7, 2022

Uh oh!

This comment was marked as outdated.

apurtell left a comment

Uh oh!

This comment was marked as outdated.

virajjasani commented Jun 8, 2022

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

ndimiduk commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Uh oh!

HBASE-27095 HbckChore should produce a report #4498

HBASE-27095 HbckChore should produce a report #4498

Uh oh!

Conversation

ndimiduk commented Jun 7, 2022

Uh oh!

This comment was marked as outdated.

apurtell left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

virajjasani commented Jun 8, 2022

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

ndimiduk commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Apache-HBase commented Jun 9, 2022

Uh oh!

Uh oh!