Skip to content

HBASE-23304: RPCs needed for client meta information lookup (apache#904) #1098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 29, 2020

Conversation

bharathv
Copy link
Contributor

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell apurtell@apache.org

This patch implements a simple cache that all the masters
can lookup to serve cluster ID to clients. Active HMaster
is still responsible for creating it but all the masters
will read it from fs to serve clients.

RPCs exposing it will come in a separate patch as a part of
HBASE-18095.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
(cherry picked from commit c2e01f2)
…ache#812)

Currently we just track whether an active master exists.
It helps to also track the address of the active master in
all the masters to help serve the client RPC requests to
know which master is active.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit efebb84)
* HBASE-23281: Track meta region changes on masters

This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.

This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 8571d38)
* HBASE-23304: RPCs needed for client meta information lookup

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 4f8fbba)
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 prototool 0m 0s prototool was not available.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 19 new or modified test files.
_ branch-2 Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for branch
+1 💚 mvninstall 5m 22s branch-2 passed
+1 💚 compile 2m 25s branch-2 passed
+1 💚 checkstyle 2m 15s branch-2 passed
+1 💚 shadedjars 4m 16s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 34s branch-2 passed
+0 🆗 spotbugs 0m 34s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 7m 51s branch-2 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 15s Maven dependency ordering for patch
+1 💚 mvninstall 4m 55s the patch passed
+1 💚 compile 2m 24s the patch passed
+1 💚 cc 2m 24s the patch passed
+1 💚 javac 2m 24s the patch passed
+1 💚 checkstyle 0m 12s The patch passed checkstyle in hbase-protocol-shaded
+1 💚 checkstyle 0m 37s hbase-client: The patch generated 0 new + 194 unchanged - 1 fixed = 194 total (was 195)
+1 💚 checkstyle 0m 13s The patch passed checkstyle in hbase-zookeeper
+1 💚 checkstyle 1m 12s hbase-server: The patch generated 0 new + 119 unchanged - 13 fixed = 119 total (was 132)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 4m 19s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 15m 39s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚 hbaseprotoc 2m 17s the patch passed
+1 💚 javadoc 1m 33s the patch passed
+1 💚 findbugs 8m 20s the patch passed
_ Other Tests _
+1 💚 unit 0m 42s hbase-protocol-shaded in the patch passed.
+1 💚 unit 3m 45s hbase-client in the patch passed.
+1 💚 unit 0m 53s hbase-zookeeper in the patch passed.
+1 💚 unit 153m 34s hbase-server in the patch passed.
+1 💚 asflicense 2m 12s The patch does not generate ASF License warnings.
235m 0s
Subsystem Report/Notes
Docker Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/artifact/out/Dockerfile
GITHUB PR #1098
JIRA Issue HBASE-23304
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
uname Linux 1aaa31531bb2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1098/out/precommit/personality/provided.sh
git revision branch-2 / e0f9133
Default Java 1.8.0_181
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/testReport/
Max. process+thread count 6066 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-zookeeper hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@wchevreuil wchevreuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ndimiduk
Copy link
Member

ndimiduk commented Feb 3, 2020

@bharathv @apurtell What exactly was this commit? The message looks like it contains everything from the feature branch, but the changed files is just HBASE-23304.

Anyway, we haven't merged the feature branch to master yet; we're not quite ready to back port to branch-2. I think this should be reverted.

ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Feb 3, 2020
@bharathv bharathv deleted the HBASE-23304-branch-2 branch February 3, 2020 22:47
@bharathv
Copy link
Contributor Author

bharathv commented Feb 3, 2020

Ya, so this is a squashed commit up until HBASE-23304. I'm ok with reverting it. I can submit a single PR with all the commits later, once the feature branch merges with master.

ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Feb 3, 2020
@apurtell
Copy link
Contributor

apurtell commented Feb 4, 2020

Copying my comment from the JIRA:

There were multiple PRs opened for the same set of changes. I merged the one @bharathv indicated.

If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth.

I think it better one PR/merge for the whole backport per branch, with commit message incorporating all relevant JIRA references. These changes for the most part aren't useful in isolation.

@ndimiduk
Copy link
Member

ndimiduk commented Feb 4, 2020

I see your reasoning. Squishing all the commits makes accounting more difficult, IMHO. The larger issue is that we're still stabilizing the tests vs. master; none of these commits have landed on master yet.

@ndimiduk
Copy link
Member

ndimiduk commented Feb 4, 2020

I should say -- I propose one PR per branch containing all the commits, but land it using "rebase and merge" instead of "squash and merge" so that the individual commits are preserved. That make the accounting I refer to simpler. Agreed there's no need for one PR per commit.

ndimiduk added a commit that referenced this pull request Feb 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants