Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GET ERROR: Moving source lab_to_dr with empty password to preserve replication stream on new master #699

Open
peterpan104 opened this issue Jul 8, 2024 · 11 comments

Comments

@peterpan104
Copy link

peterpan104 commented Jul 8, 2024

I have two clusters in repman

## cluster 1
Master01
Slave01

## cluster 2
Master02
Slave02

Master02 also acts as a slave to Master01 (using replication source lab_to_dr).

However, when Master02 failed, I received an error message indicating that Slave02 entered an empty password during the 'change master' process.

I was wondering replication manager supports this kind of architecture?
I'm happy to provide more information if needed.

image

@svaroqui
Copy link
Collaborator

svaroqui commented Jul 8, 2024

Yes it does as long as you declare your 2 clusters , each cluster it's source name and domain id add cluster-head = "cluster-haproxy-source1"


[cluster-haproxy-source1] 
title = "cluster-haproxy-source1"

db-servers-hosts = "127.0.0.1:3310,127.0.0.1:3311"
db-servers-prefered-master = "127.0.0.1:3310"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source1"
replication-credential = "root:mariadb"
force-slave-readonly = false
haproxy = true
haproxy-write-port = 3303
haproxy-read-port = 3302

[cluster-haproxy-source2]
title = "cluster-haproxy-source2"

cluster-head = "cluster-haproxy-source1"
db-servers-hosts = "127.0.0.1:3312,127.0.0.1:3313"
db-servers-prefered-master = "127.0.0.1:3312"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source2"
replication-credential = "root:mariadb"
force-slave-readonly = false
prov-db-domain = "2"

@svaroqui
Copy link
Collaborator

svaroqui commented Jul 9, 2024

Just we have not been able to make such topology work with replication filters so far

@peterpan104
Copy link
Author

peterpan104 commented Jul 9, 2024

Yes it does as long as you declare your 2 clusters , each cluster it's source name and domain id add cluster-head = "cluster-haproxy-source1"


[cluster-haproxy-source1] 
title = "cluster-haproxy-source1"

db-servers-hosts = "127.0.0.1:3310,127.0.0.1:3311"
db-servers-prefered-master = "127.0.0.1:3310"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source1"
replication-credential = "root:mariadb"
force-slave-readonly = false
haproxy = true
haproxy-write-port = 3303
haproxy-read-port = 3302

[cluster-haproxy-source2]
title = "cluster-haproxy-source2"

cluster-head = "cluster-haproxy-source1"
db-servers-hosts = "127.0.0.1:3312,127.0.0.1:3313"
db-servers-prefered-master = "127.0.0.1:3312"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source2"
replication-credential = "root:mariadb"
force-slave-readonly = false
prov-db-domain = "2"

Hi @svaroqui
I don't want both clusters to use replication for the channel.
Instead, I only want it used between master02 and master01 to ensure that master01 data can be written to master02.

Alternatively, is it possible to prevent slave02 from executing 'change master to channel lab_to_dr' when master02 failed?
Or have other possible solutions?

It's just like:

## cluster 1
master01
	--slave01
	## cluser 2
	--master02 (using replication for channel lab_to_dr)
		--slave02

thanks for your reply.

@svaroqui
Copy link
Collaborator

svaroqui commented Jul 9, 2024

multi source replication does not work the way you describe it. We should no build on unsafe topology , just consider one cluster channel01 made with master01<-slave01 and <-master02 on domain 01, and an other cluster channel02 master02 <- slave02 on domain 02, if this not mariadb please consider 8.3 the new tagged GTID , ( but we do not yet support tags )

@svaroqui
Copy link
Collaborator

svaroqui commented Jul 9, 2024

I think you could try not to name the source of the parent cluster and so the replication that glue the 2 clusters only naming the cluster02

@svaroqui svaroqui added the bug label Jul 9, 2024
@svaroqui
Copy link
Collaborator

svaroqui commented Jul 9, 2024

Could you tag the issue with mysql or mariadb ?

@svaroqui
Copy link
Collaborator

svaroqui commented Jul 9, 2024

Alternatively, is it possible to prevent slave02 from executing 'change master to channel lab_to_dr' when master02 failed?

I guess that we could fixe it as i already notice this issue

@peterpan104
Copy link
Author

Hi @svaroqui
I'm testing with mariadb 10.4.

I tried the above solution

## cluster1 (not use 'replication-source-name' )
master01
	slave01
	master02


## cluster2 (replication-source-name = "test01")
master02
	slave02

But when master02 failed,slave02 is selected as master, but I still get the error Moving source with empty password to preserve replication stream on new master.

When master02 recovers, it will not become a slave of slave02.

image

I also hope that this problem will be solved.
I'm happy to provide more information if needed.
Thank you for your suggestions.

@caffeinated92
Copy link
Collaborator

caffeinated92 commented Jul 18, 2024

The problem is not about moving sources. your replication-manager clearly stated mysqlbinlog to rejoin using binary log. You have to specify using other method if you're not using binary logs to rejoin or specify your mysqlbinlog path installation

@peterpan104
Copy link
Author

Hi @caffeinated92

Under the same parameter file,
When master02 is not using replication-source, if master02 fails and recovers, it will automatically rejoin. Is there any difference between these scenarios?

When using both replication-source and regular replication simultaneously, is it possible to control replication-source so that it won't be transferred?

Thank you for your reply.

@svaroqui
Copy link
Collaborator

Can you provide us with the version your are using please also send your configs and logs in plain text, we have multiple paying user using that topology an not only for migration , but things have do be done correctly for that to work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants