Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dm: add a configuration to adjust safe mode duration when dm task abnormal reboot #10555

Merged
merged 7 commits into from
Sep 15, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dm/dm-safe-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ REPLACE INTO dummydb.dummytbl (id, int_value, str_value) VALUES (999, 888888, 'a

第二种情况下,DM 并不知道哪些 checkpoint 之后的 binlog 已经被执行过。为了保险起见,只要没有在 checkpoint 找到 `safemode_exit_point` 信息,DM 就会在前 2 个 checkpoint 间隔中都开启安全模式,确保这段时间里的 binlog 被重复执行不会引发问题。默认的 checkpoint 间隔是 30 秒,也就是说,一个正常的增量同步任务开始时,前 2 * 30 = 60 秒会自动强制开启安全模式。

你可以配置 syncer 的配置项 `checkpoint-flush-interval` 来调整 checkpoint 的间隔,从而影响增量同步任务开始时安全模式的持续时间。一般情况下不建议调整,如有需要,应优先使用[手动开启](#手动开启)
你可以配置 syncer 的配置项 `checkpoint-flush-interval` 来调整 checkpoint 的间隔,从而影响增量同步任务开始时安全模式的持续时间。一般情况下不建议调整,如有需要,也可以配置 syncer 的配置项 `safe-mode-duration` 来直接调整安全模式的持续时间
okJiang marked this conversation as resolved.
Show resolved Hide resolved

### 手动开启

Expand Down
5 changes: 5 additions & 0 deletions dm/task-configuration-file-full.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ syncers: # sync 处理单元的运行配置参数

# 设置为 true,则将来自上游的 `INSERT` 改写为 `REPLACE`,将 `UPDATE` 改写为 `DELETE` 与 `REPLACE`,保证在表结构中存在主键或唯一索引的条件下迁移数据时可以重复导入 DML。
safe-mode: false
# 安全模式的持续时间
ran-huang marked this conversation as resolved.
Show resolved Hide resolved
# 如不设置或者设置为 "",则默认为 `checkpoint-flush-interval`(默认为 30s)的两倍,即 60s。
# 如设置为 "0s",则在 DM 自动进入安全模式的时候报错。
# 如设置为正常值,例如 "1m30s",则在该任务异常暂停、记录 safemode_exit_point 失败、或是 DM 进程异常退出时,把安全模式持续时间调整为 1 分 30 秒。详情可见 https://docs.pingcap.com/zh/tidb/stable/dm-safe-mode#自动开启。
ran-huang marked this conversation as resolved.
Show resolved Hide resolved
safe-mode-duration: "60s"
# 设置为 true,DM 会在不增加延迟的情况下,尽可能地将上游对同一条数据的多次操作压缩成一次操作。
# 如 INSERT INTO tb(a,b) VALUES(1,1); UPDATE tb SET b=11 WHERE a=1; 会被压缩成 INSERT INTO tb(a,b) VALUES(1,11); 其中 a 为主键
# 如 UPDATE tb SET b=1 WHERE a=1; UPDATE tb(a,b) SET b=2 WHERE a=1; 会被压缩成 UPDATE tb(a,b) SET b=2 WHERE a=1; 其中 a 为主键
Expand Down