Skip to content

HBASE-27335 HBase shell hang for a minute when quiting #4737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 31, 2022

Conversation

2005hithlj
Copy link
Contributor

@2005hithlj
Copy link
Contributor Author

@Apache9 sir. Could you take a look? Thanks.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 51s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 2m 25s branch-2 passed
+1 💚 compile 0m 16s branch-2 passed
+1 💚 shadedjars 3m 52s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 15s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 3s the patch passed
+1 💚 compile 0m 17s the patch passed
+1 💚 javac 0m 17s the patch passed
+1 💚 shadedjars 3m 56s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 14s the patch passed
_ Other Tests _
+1 💚 unit 2m 31s hbase-client in the patch passed.
17m 49s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #4737
Optional Tests javac javadoc unit shadedjars compile
uname Linux 8ca260d2edcf 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 935b0d2
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/testReport/
Max. process+thread count 173 (vs. ulimit of 30000)
modules C: hbase-client U: hbase-client
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 56s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 3m 1s branch-2 passed
+1 💚 compile 0m 18s branch-2 passed
+1 💚 shadedjars 4m 8s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 18s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 28s the patch passed
+1 💚 compile 0m 18s the patch passed
+1 💚 javac 0m 18s the patch passed
+1 💚 shadedjars 4m 3s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 16s the patch passed
_ Other Tests _
+1 💚 unit 2m 44s hbase-client in the patch passed.
19m 41s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #4737
Optional Tests javac javadoc unit shadedjars compile
uname Linux 4d6610485f8d 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 935b0d2
Default Java AdoptOpenJDK-11.0.10+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/testReport/
Max. process+thread count 198 (vs. ulimit of 30000)
modules C: hbase-client U: hbase-client
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 3m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚 mvninstall 2m 34s branch-2 passed
+1 💚 compile 0m 44s branch-2 passed
+1 💚 checkstyle 0m 16s branch-2 passed
+1 💚 spotless 0m 41s branch has no errors when running spotless:check.
+1 💚 spotbugs 0m 49s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 17s the patch passed
+1 💚 compile 0m 42s the patch passed
+1 💚 javac 0m 42s the patch passed
+1 💚 checkstyle 0m 14s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 8m 41s Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚 spotless 0m 39s patch has no errors when running spotless:check.
+1 💚 spotbugs 0m 51s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 9s The patch does not generate ASF License warnings.
27m 52s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #4737
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux 2cc15911cf2d 5.4.0-1083-aws #90~18.04.1-Ubuntu SMP Fri Aug 5 08:12:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 935b0d2
Default Java AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count 64 (vs. ulimit of 30000)
modules C: hbase-client U: hbase-client
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4737/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Copy link
Contributor

Apache9 commented Aug 28, 2022

When quiting, we will not close this connection?

@2005hithlj
Copy link
Contributor Author

When quiting, we will not close this connection?

@Apache9 sir. Thanks for your review.
I'm not sure if the connection will be closed when exiting from the hbase shell, because I can't find the relevant information in the ruby script. At the same time, no relevant information was found in the master and branch-1 branches. But I will confirm further.

@Apache9
Copy link
Contributor

Apache9 commented Aug 28, 2022

I prefer we first find out why the daemon thread does not quit, because we do not close the connection? Or we have closed the connection but haven't shutdown the thread pool?

@Apache9
Copy link
Contributor

Apache9 commented Aug 28, 2022

I think the problem is that, we defined a shutdown method in hbase.rb where we close the connection in this method, but we never call it in our code base...

I'm not a ruby expert so I'm not sure how to call shutdown when we see 'exit', as seems there is no 'exit' command defined in our code base...

@Apache9
Copy link
Contributor

Apache9 commented Aug 28, 2022

diff --git a/hbase-shell/src/main/ruby/shell.rb b/hbase-shell/src/main/ruby/shell.rb
index f6b3efd945..4bffdaacab 100644
--- a/hbase-shell/src/main/ruby/shell.rb
+++ b/hbase-shell/src/main/ruby/shell.rb
@@ -108,6 +108,9 @@ module Shell
     # exit the interactive shell and save that this
     # happend via a call to exit
     def exit(ret = 0)
+      if self.hbase
+        self.hbase.shutdown
+      end
       @exit_code = ret
       IRB.irb_exit(IRB.CurrentContext.irb, ret)
     end

Apply this to branch-2.5 can make the shell quit immediately when typing 'exit' command, but when calling 'quit' command it will still hang there... I'm not sure how we do we deal with the 'quit' command...

@2005hithlj
Copy link
Contributor Author

Yes, the 'quit' command is really a difficult thing to deal with. I searched the ruby IRB for info about the 'quit' command, and there is not much info to refer to.

@Apache9
Copy link
Contributor

Apache9 commented Aug 31, 2022

Anyway, let's apply this PR to fix the problem first.

Please open another issue to track the 'releasing resources when quiting IRB' problem @2005hithlj .

Thanks.

@2005hithlj
Copy link
Contributor Author

@Apache9 OK, sir.

@2005hithlj
Copy link
Contributor Author

I have created an issue to track this issue:
https://issues.apache.org/jira/browse/HBASE-27348

@Apache9 Apache9 merged commit 6a7a91a into apache:branch-2 Aug 31, 2022
Apache9 pushed a commit that referenced this pull request Aug 31, 2022
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6a7a91a)
Apache9 pushed a commit that referenced this pull request Aug 31, 2022
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6a7a91a)
bbeaudreault pushed a commit to HubSpot/hbase that referenced this pull request Sep 23, 2022
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6a7a91a)
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6a7a91a)
(cherry picked from commit fd27d68)
Change-Id: I6e94a26f5ac63396f5f898fdb5b380b9adc06c96
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants