DM: gh-ost --cut-over=two-step compatibility

[chenmin1992]

What did you do?

Setup a dm cluster and online schema change by gh-ost --cut-over=two-step
gh-ost
--user="root"
--password="root"
--host=10.10.10.101
--port=3336
--database="test"
--table="test"
--alter="add column c3 varchar(32)"
--switch-to-rbr
--cut-over=two-step
--cut-over-lock-timeout-seconds=1
--initially-drop-old-table
--initially-drop-ghost-table
--initially-drop-socket-file
--ok-to-drop-table
--approve-renamed-columns
--max-load='Threads_running=100,Threads_connected=500'
--default-retries=3600
--allow-on-master
--execute

gh-ost \
--user="root" \
--password="123456" \
--host=127.0.0.1 \
--port=3306 \
--database="test" \
--table="test" \
--alter="add column c3 varchar(32)" \
--switch-to-rbr \
--cut-over=two-step \
--cut-over-lock-timeout-seconds=1 \
--initially-drop-old-table \
--initially-drop-ghost-table \
--initially-drop-socket-file \
--ok-to-drop-table \
--approve-renamed-columns \
--max-load='Threads_running=100,Threads_connected=500' \
--default-retries=3600 \
--allow-on-master \
--execute

What did you expect to see?

Downstream TiDB can alter table via online DDL successfully

What did you see instead?

Origin table has been renamed to _test_del, which is gh-ost trash table.

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

Starting component `dm`: /root/.tiup/components/dm/v1.10.2/tiup-dm display dm-test
Cluster type:       dm
Cluster name:       dm-test
Cluster version:    v6.1.0
Deploy user:        tidb
SSH type:           builtin

Upstream MySQL/MariaDB server version:

Server version:		5.7.39-log MySQL Community Server (GPL)

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

mysql> SELECT tidb_version();
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                                                                       |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v6.1.0
Edition: Community
Git Commit Hash: 1a89decdb192cbdce6a7b0020d71128bc964d30f
Git Branch: heads/refs/tags/v6.1.0
UTC Build Time: 2022-06-05 05:15:11
GoVersion: go1.18.2
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

How did you deploy DM: tiup or manually?

tiup

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mysql-01",
                "worker": "dm-10.10.10.102-8262",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "test",
                    "stage": "Running",
                    "unit": "Sync",
                    "result": null,
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "110",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(mysql-bin.000041, 408806)",
                        "masterBinlogGtid": "",
                        "syncerBinlog": "(mysql-bin.000041, 408412)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false,
                        "binlogType": "remote",
                        "secondsBehindMaster": "0"
                    }
                }
            ]
        }
    ]
}

[chenmin1992]

[lance6716]

reproduced

[lance6716]

dm-worker.log

you provided wrong log file 😅

[lance6716]

currently --cut-over=two-step will generate two ALTER TABLE DDL, but DM can only recognize one RENAME DDL. Will support it later.

/cc @niubell