HBASE-28484 Allow replication to different target table #5819

hgromer · 2024-04-10T12:39:40Z

No description provided.

hgromer · 2024-04-10T12:42:06Z

hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java

@@ -117,6 +121,10 @@ public Map<TableName, List<String>> getExcludeTableCFsMap() {
    return (Map<TableName, List<String>>) excludeTableCFsMap;
  }



I'm not sure what the best practice here is, but would it make sense to add a deprecated setter to ReplicationPeerConfig?

ReplicationPeerConfig is designed to be immutable. So you'd better add a setter in the ReplicationPeerConfigBuilder.

I added this method to the builder in the initial commit. I asked this question because I noticed ReplicationPeerConfig has some deprecated setter methods. I wasn't sure if it would make sense to introduce a deprecated setter for this new field to follow the existing code, or if I should leave it out for this new field.

Apache-HBase · 2024-04-10T13:56:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 40s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for branch
+1 💚	mvninstall	5m 0s	master passed
+1 💚	compile	6m 26s	master passed
+1 💚	checkstyle	1m 32s	master passed
+1 💚	spotless	1m 4s	branch has no errors when running spotless:check.
+1 💚	spotbugs	7m 30s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 37s	the patch passed
+1 💚	compile	6m 44s	the patch passed
+1 💚	cc	6m 44s	the patch passed
+1 💚	javac	6m 44s	the patch passed
+1 💚	checkstyle	1m 26s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	7m 47s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	hbaseprotoc	2m 3s	the patch passed
+1 💚	spotless	1m 8s	patch has no errors when running spotless:check.
+1 💚	spotbugs	8m 5s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 42s	The patch does not generate ASF License warnings.
		66m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5819
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool
uname	Linux 2b9ea8acb78c 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adc79a0`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	82 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-04-10T16:53:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 13s	master passed
+1 💚	compile	1m 48s	master passed
+1 💚	shadedjars	5m 46s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 56s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 0s	the patch passed
+1 💚	compile	1m 49s	the patch passed
+1 💚	javac	1m 49s	the patch passed
+1 💚	shadedjars	5m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 49s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 33s	hbase-protocol-shaded in the patch passed.
+1 💚	unit	1m 32s	hbase-client in the patch passed.
+1 💚	unit	212m 2s	hbase-server in the patch passed.
		242m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5819
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 859cb28184a3 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adc79a0`
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/testReport/
Max. process+thread count	5110 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-04-10T17:16:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 38s	master passed
+1 💚	compile	1m 34s	master passed
+1 💚	shadedjars	5m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 52s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 28s	the patch passed
+1 💚	compile	1m 32s	the patch passed
+1 💚	javac	1m 32s	the patch passed
+1 💚	shadedjars	5m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 51s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 32s	hbase-protocol-shaded in the patch passed.
+1 💚	unit	1m 16s	hbase-client in the patch passed.
+1 💚	unit	237m 11s	hbase-server in the patch passed.
		265m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5819
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c7e9eb69499f 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adc79a0`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/testReport/
Max. process+thread count	5107 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-04-10T18:21:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 49s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 48s	master passed
+1 💚	compile	1m 58s	master passed
+1 💚	shadedjars	6m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 13s	the patch passed
+1 💚	compile	2m 16s	the patch passed
+1 💚	javac	2m 16s	the patch passed
+1 💚	shadedjars	6m 22s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 0s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 46s	hbase-protocol-shaded in the patch passed.
+1 💚	unit	1m 54s	hbase-client in the patch passed.
+1 💚	unit	294m 39s	hbase-server in the patch passed.
		330m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5819
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 13bbb20f3af2 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `adc79a0`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/testReport/
Max. process+thread count	4614 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5819/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2024-04-24T02:57:45Z

I think we may have more patterns when replication...

For example, we want to replicate a whole namespace and change the target namespace name, we want to replicate all the tables on the cluster but adding a prefix to all the table names, etc.

I do not mean we should support them all in the first place, but we'd better list all the requirements first, and then see whether there is a general enough solution to support them all. So in the future, if we want to add more supports, we do not need to change bunch of existing code, as it is a big pain if we want to change existing data structures...

Thanks.

Hernan Gelaf-Romer and others added 2 commits April 4, 2024 15:13

Supports intercluster replication to a different target table

6366577

Merge branch 'apache:master' into HBASE-28484

575574d

hgromer commented Apr 10, 2024

View reviewed changes

rmdmattingly mentioned this pull request Nov 24, 2024

HBASE-28484: Add ability to replicate to a different tableName #6484

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-28484 Allow replication to different target table #5819

HBASE-28484 Allow replication to different target table #5819

hgromer commented Apr 10, 2024

Uh oh!

hgromer Apr 10, 2024

Uh oh!

Apache9 Apr 24, 2024

Uh oh!

hgromer Apr 26, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache9 commented Apr 24, 2024

Uh oh!

Uh oh!

		@@ -117,6 +121,10 @@ public Map<TableName, List<String>> getExcludeTableCFsMap() {
		return (Map<TableName, List<String>>) excludeTableCFsMap;
		}

HBASE-28484 Allow replication to different target table #5819

Are you sure you want to change the base?

HBASE-28484 Allow replication to different target table #5819

Conversation

hgromer commented Apr 10, 2024

Uh oh!

hgromer Apr 10, 2024

Choose a reason for hiding this comment

Uh oh!

Apache9 Apr 24, 2024

Choose a reason for hiding this comment

Uh oh!

hgromer Apr 26, 2024

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache-HBase commented Apr 10, 2024

Uh oh!

Apache9 commented Apr 24, 2024

Uh oh!

Uh oh!