Skip to content

HBASE-25830 HBaseCluster support CompactionServer for UTs (addendum) #3464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 14, 2021
Merged

HBASE-25830 HBaseCluster support CompactionServer for UTs (addendum) #3464

merged 2 commits into from
Jul 14, 2021

Conversation

nyl3532016
Copy link
Contributor

Support create/shutdown compactionserver method in HBaseMinicluster
Fix weakness of TestCompactionServer

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HBASE-25714 Compile Tests _
+1 💚 mvninstall 4m 0s HBASE-25714 passed
+1 💚 compile 3m 11s HBASE-25714 passed
+1 💚 checkstyle 1m 6s HBASE-25714 passed
+1 💚 spotbugs 2m 5s HBASE-25714 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 36s the patch passed
+1 💚 compile 3m 11s the patch passed
+1 💚 javac 3m 11s the patch passed
-0 ⚠️ checkstyle 1m 4s hbase-server: The patch generated 4 new + 58 unchanged - 1 fixed = 62 total (was 59)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 18m 4s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚 spotbugs 2m 15s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 15s The patch does not generate ASF License warnings.
47m 9s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #3464
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux 6c95bd1e78b8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-25714 / 7be5879
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 96 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 10s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚 mvninstall 4m 21s HBASE-25714 passed
+1 💚 compile 1m 4s HBASE-25714 passed
+1 💚 shadedjars 8m 29s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 37s HBASE-25714 passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 0s the patch passed
+1 💚 compile 1m 4s the patch passed
+1 💚 javac 1m 4s the patch passed
+1 💚 shadedjars 8m 20s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 35s the patch passed
_ Other Tests _
+1 💚 unit 215m 2s hbase-server in the patch passed.
246m 40s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #3464
Optional Tests javac javadoc unit shadedjars compile
uname Linux 6a355265d23f 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-25714 / 7be5879
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/testReport/
Max. process+thread count 3350 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ HBASE-25714 Compile Tests _
+1 💚 mvninstall 4m 0s HBASE-25714 passed
+1 💚 compile 3m 9s HBASE-25714 passed
+1 💚 checkstyle 1m 7s HBASE-25714 passed
+1 💚 spotbugs 2m 7s HBASE-25714 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 36s the patch passed
+1 💚 compile 3m 13s the patch passed
+1 💚 javac 3m 13s the patch passed
-0 ⚠️ checkstyle 1m 5s hbase-server: The patch generated 4 new + 58 unchanged - 1 fixed = 62 total (was 59)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 18m 9s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚 spotbugs 2m 19s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 16s The patch does not generate ASF License warnings.
47m 19s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #3464
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux f048d50a3387 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-25714 / 7be5879
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 96 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 10s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚 mvninstall 3m 56s HBASE-25714 passed
+1 💚 compile 1m 3s HBASE-25714 passed
+1 💚 shadedjars 8m 27s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 37s HBASE-25714 passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 6s the patch passed
+1 💚 compile 1m 4s the patch passed
+1 💚 javac 1m 4s the patch passed
+1 💚 shadedjars 8m 31s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 37s the patch passed
_ Other Tests _
+1 💚 unit 214m 23s hbase-server in the patch passed.
245m 48s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #3464
Optional Tests javac javadoc unit shadedjars compile
uname Linux 600de0d519a1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-25714 / 7be5879
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/testReport/
Max. process+thread count 3645 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/2/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@nyl3532016 nyl3532016 requested a review from Apache9 July 8, 2021 01:52
Apache9
Apache9 reviewed Jul 11, 2021
View reviewed changes
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java Outdated
/**
* Starts a compaction server thread running
*
* @throws IOException
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove empty throws doc

hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java Outdated
/**
* Wait for the specified compaction server to stop. Removes this thread from list
* of running threads.
* @param serverNumber
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: remove empty param doc

hbase-server/src/test/java/org/apache/hadoop/hbase/compactionserver/TestCompactionServer.java Outdated
TEST_UTIL.getAdmin().modifyTable(htd);
TEST_UTIL.waitUntilAllRegionsAssigned(TABLENAME);
// invoke compact
TEST_UTIL.compact(TABLENAME, false);
TEST_UTIL.waitFor(6000, () -> COMPACTION_SERVER.requestCount.sum() > 0);
long requestCount = COMPACTION_SERVER.requestCount.sum();
Thread.sleep(1000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind explaining a bit here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK,this sleep is no need. My Initially thought is compact request is async, this sleep will make the check below effective, After view code I find the increment of request count on compaction server is sync in compaction request

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 9s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ HBASE-25714 Compile Tests _
+1 💚 mvninstall 4m 10s HBASE-25714 passed
+1 💚 compile 1m 3s HBASE-25714 passed
+1 💚 shadedjars 8m 28s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 37s HBASE-25714 passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 7s the patch passed
+1 💚 compile 1m 4s the patch passed
+1 💚 javac 1m 4s the patch passed
+1 💚 shadedjars 8m 25s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s the patch passed
_ Other Tests _
+1 💚 unit 214m 33s hbase-server in the patch passed.
245m 57s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #3464
Optional Tests javac javadoc unit shadedjars compile
uname Linux 0c298f49e568 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-25714 / 7be5879
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/testReport/
Max. process+thread count 3415 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3464/3/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@nyl3532016 nyl3532016 requested a review from Apache9 July 14, 2021 10:32
@nyl3532016 nyl3532016 merged commit 27ec80d into apache:HBASE-25714 Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Apache9 Apache9 Apache9 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nyl3532016 @Apache-HBase @Apache9