Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在多partition场景下实现ddl同步 #2842

Open
roamerway opened this issue Jun 29, 2020 · 0 comments
Open

在多partition场景下实现ddl同步 #2842

roamerway opened this issue Jun 29, 2020 · 0 comments

Comments

@roamerway
Copy link

roamerway commented Jun 29, 2020

在高并发数据同步场景下,送kafka必须采用多partition并行模式。与此同时,ddl同步又必须保证按顺序执行,前后不能有一条sql的顺序错乱。否则很可能会引起目标库中执行错误进而导致不一致数据产生。

建议扩展MQProducer类的send方法,通过zk辅助进行停顿操作,从而实现ddl有序同步。
具体步骤为:

  1. 在识别出ddl以后,将ddl之前的dml照常发送,
  2. 停止发送后续dml,在zk中当前instance节点下创建一个ddl节点,用来做ddl同步期间的状态控制。
  3. 监听等待zk中ddl节点状态改变事件,然后向kafka发送ddl同步消息
    4.一旦ddl节点状态变为已完成,则说明ddl同步成功,删除该节点并继续后续message处理。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant