dm: add a dm directory (#873)

README.md

@@ -123,30 +123,29 @@
   - [mydumper](tools/mydumper.md)
   - [Loader](tools/loader.md)
   + Data Migration
-    - [Overview](tools/data-migration-overview.md)
-    - [Restrictions](tools/data-migration-overview.md#usage-restrictions)
+    - [Overview](/tools/dm/overview.md)
+    - [Restrictions](/tools/dm/overview.md#usage-restrictions)
+    - [Deploy](/tools/dm/practice.md)
     + Features
-      - [Table Routing](tools/dm-data-synchronization-features.md#table-routing)
-      - [Black and White Lists](tools/dm-data-synchronization-features.md#black-and-white-table-lists)
-      - [Binlog Event Filtering](tools/dm-data-synchronization-features.md#binlog-event-filtering)
-      - [Column Mapping](tools/dm-data-synchronization-features.md#column-mapping)
-      - [Synchronization Delay Monitoring](tools/dm-data-synchronization-features.md#synchronization-delay-monitoring)
+      - [Table Routing](/tools/dm/data-synchronization-features.md#table-routing)
+      - [Black and White Lists](/tools/dm/data-synchronization-features.md#black-and-white-table-lists)
+      - [Binlog Event Filtering](/tools/dm/data-synchronization-features.md#binlog-event-filtering)
+      - [Column Mapping](/tools/dm/data-synchronization-features.md#column-mapping)
+      - [Synchronization Delay Monitoring](/tools/dm/data-synchronization-features.md#synchronization-delay-monitoring)
       + Sharding Support
-        - [Introduction](tools/dm-sharding-solution.md)
-        - [Restrictions](tools/dm-sharding-solution.md#sharding-ddl-usage-restrictions)
-        - [Troubleshoot](tools/troubleshooting-sharding-ddl-locks.md)
+        - [Introduction](/tools/dm/sharding-solution.md)
+        - [Restrictions](/tools/dm/sharding-solution.md#sharding-ddl-usage-restrictions)
+        - [Troubleshoot](/tools/dm/troubleshooting-sharding-ddl-locks.md)
     + Usage Scenarios
-      - [Simple Scenario](tools/dm-simple-synchronization-scenario.md)
-      - [Shard Merge Scenario](tools/dm-shard-merge-scenario.md)
-    - [Deploy](tools/data-migration-practice.md)
+      - [Simple Scenario](/tools/dm/simple-synchronization-scenario.md)
+      - [Shard Merge Scenario](/tools/dm/shard-merge-scenario.md)
     + Configure
-      - [Overview](tools/dm-configuration-file-overview.md)
-      - [Task Configuration](tools/dm-task-configuration-file-intro.md)
-    - [Monitor](tools/dm-monitor.md)
-    - [Manage the Task](tools/data-migration-manage-task.md)
-    - [Cluster Operations](tools/data-migration-cluster-operations.md)
-    - [Upgrade Loader or Syncer to DM](tools/upgrade-loader-or-syncer-to-dm.md)
-    - [Troubleshoot](tools/data-migration-troubleshooting.md)
+      - [Overview](/tools/dm/dm-configuration-file-overview.md)
+      - [Task Configuration](/tools/dm/task-configuration-file-intro.md)
+    - [Monitor](/tools/dm/monitor.md)
+    - [Manage the Task](/tools/dm/manage-task.md)
+    - [Cluster Operations](/tools/dm/cluster-operations.md)
+    - [Troubleshoot](/tools/dm/troubleshooting.md)
   + TiDB-Lightning
     - [Overview](tools/lightning/overview-architecture.md)
     - [Deployment](tools/lightning/deployment.md)

tools/data-migration-cluster-operations.md → tools/dm/cluster-operations.md

@@ -40,7 +40,7 @@ This sections describes the considerations that you need to know when you restar
 
 **In the process of full data loading:**
 
-For the SQL files during full data import, DM uses the downstream database to record the checkpoint information. When DM-worker is restarted, it checks the checkpoint information and you can use the [`start-task` command](../tools/data-migration-practice.md#step-4-start-the-data-synchronization-task) to recover the data synchronization task automatically.
+For the SQL files during full data import, DM uses the downstream database to record the checkpoint information. When DM-worker is restarted, it checks the checkpoint information and you can use the [`start-task` command](/tools/dm/practice.md#step-4-start-the-data-synchronization-task) to recover the data synchronization task automatically.
 
 **In the process of incremental data synchronization:**
 
@@ -60,7 +60,7 @@ For the binlog during incremental data import, DM uses the downstream database t
 
         At this time, DM tries again to synchronize these DDL statements that are not skipped. However, the restarted DM-worker instances will be blocked at the position of the binlog event corresponding to the DDL binlog event, because the DM-worker instance that is not restarted has executed to the place after this DDL binlog event.
 
-        To resolve this issue, follow the steps described in [Troubleshooting Sharding DDL Locks](../tools/troubleshooting-sharding-ddl-locks.md#condition-two-a-dm-worker-restarts-or-is-unreachable-temporarily)
+        To resolve this issue, follow the steps described in [Troubleshooting Sharding DDL Locks](/tools/dm/troubleshooting-sharding-ddl-locks.md#condition-two-a-dm-worker-restarts-or-is-unreachable-temporarily)
 
 #### Restarting DM-master considerations
 
@@ -71,7 +71,7 @@ The information maintained by DM-master includes the following two major types,
 
 When DM-master is restarted, it automatically requests the task information from each DM-worker instance and rebuilds the corresponding relationship between the task and DM-worker. However, at this time, DM-worker does not resend the sharding DDL information, so it might occur that the sharding DDL lock synchronization cannot be finished automatically because of the lost lock information.
 
-To resolve this issue, follow the steps described in [Troubleshooting Sharding DDL Locks](../tools/troubleshooting-sharding-ddl-locks.md#condition-three-dm-master-restarts).
+To resolve this issue, follow the steps described in [Troubleshooting Sharding DDL Locks](/tools/dm/troubleshooting-sharding-ddl-locks.md#condition-three-dm-master-restarts).
 
 #### Restarting dmctl considerations
 
@@ -173,7 +173,7 @@ Assuming that you want to add a DM-worker instance on the `172.16.10.74` machine
 
 1. Configure the SSH mutual trust and sudo rules on the Control Machine.
 
-    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](../tools/data-migration-deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account and add `172.16.10.74` to the `[servers]` section of the `hosts.ini` file.
+    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](/tools/dm/deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account and add `172.16.10.74` to the `[servers]` section of the `hosts.ini` file.
 
         ```
         $ cd /home/tidb/dm-ansible
@@ -267,7 +267,7 @@ Assuming that the `172.16.10.71` machine needs to be maintained or this machine
 
 1. Configure the SSH mutual trust and sudo rules on the Control machine.
 
-    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](../tools/data-migration-deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account, and add `172.16.10.80` to the `[servers]` section of the `hosts.ini` file.
+    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](/tools/dm/deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account, and add `172.16.10.80` to the `[servers]` section of the `hosts.ini` file.
 
         ```
         $ cd /home/tidb/dm-ansible
@@ -327,7 +327,7 @@ Assuming that the `172.16.10.72` machine needs to be maintained or this machine
 
 1. Configure the SSH mutual trust and sudo rules on the Control Machine. 
 
-    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](../tools/data-migration-deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account, and add `172.16.10.75` to the `[servers]` section of the `hosts.ini` file.
+    1. Refer to [Configure the SSH mutual trust and sudo rules on the Control Machine](/tools/dm/deployment.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine), log in to the Control Machine using the `tidb` user account, and add `172.16.10.75` to the `[servers]` section of the `hosts.ini` file.
 
         ```
         $ cd /home/tidb/dm-ansible

tools/dm-data-synchronization-features.md → tools/dm/data-synchronization-features.md

@@ -33,7 +33,7 @@ routes:
 
 ### Parameter explanation
 
-DM synchronizes the upstream MySQL or MariaDB instance table that matches the [`schema-pattern`/`table-pattern` rule provided by Table selector](../tools/dm-table-selector.md) to the downstream `target-schema`/`target-table`.
+DM synchronizes the upstream MySQL or MariaDB instance table that matches the [`schema-pattern`/`table-pattern` rule provided by Table selector](/tools/dm/table-selector.md) to the downstream `target-schema`/`target-table`.
 
 ### Usage examples
 
@@ -221,7 +221,7 @@ filters:
 
 ### Parameter explanation
 
-- [`schema-pattern`/`table-pattern`](../tools/dm-table-selector.md): the binlog events or DDL SQL statements of upstream MySQL or MariaDB instance tables that match `schema-pattern`/`table-pattern` are filtered by the rules below.
+- [`schema-pattern`/`table-pattern`](/tools/dm/table-selector.md): the binlog events or DDL SQL statements of upstream MySQL or MariaDB instance tables that match `schema-pattern`/`table-pattern` are filtered by the rules below.
 
 - `events`: the binlog event array.
 
@@ -349,7 +349,7 @@ column-mappings:
 
 ### Parameter explanation
 
-- [`schema-pattern`/`table-pattern`](../tools/dm-table-selector.md): to execute column value modifying operations on the upstream MySQL or MariaDB instance tables that match the `schema-pattern`/`table-pattern` filtering rule.
+- [`schema-pattern`/`table-pattern`](/tools/dm/table-selector.md): to execute column value modifying operations on the upstream MySQL or MariaDB instance tables that match the `schema-pattern`/`table-pattern` filtering rule.
 - `source-column`, `target-column`: to modify the value of the `source-column` column according to specified `expression` and assign the new value to `target-column`.
 - `expression`: the expression used to modify data. Currently, only the `partition id` built-in expression is supported.
 
@@ -451,4 +451,4 @@ enable-heartbeat: true
 - DM-worker updates the `TS_slave_task` synchronization time after each synchronization task obtains the `dm_heartbeat`.`heartbeat` binlog.
 - DM-worker queries the current `TS_master` timestamp in the corresponding upstream MySQL or MariaDB `dm_heartbeat`.`heartbeat` tables every 10 seconds, and calculates `task_lag` = `TS_master` - `TS_slave_task` for each task.
 
-See the `replicate lag` in the [binlog replication](../tools/dm-monitor.md#binlog-replication) processing unit of DM monitoring metrics.
+See the `replicate lag` in the [binlog replication](/tools/dm/monitor.md#binlog-replication) processing unit of DM monitoring metrics.

tools/data-migration-deployment.md → tools/dm/deployment.md

@@ -181,7 +181,7 @@ You can choose one of the following two types of cluster topology according to y
 
 - [The cluster topology of a single DM-worker instance on each node](#option-1-use-the-cluster-topology-of-a-single-dm-worker-instance-on-each-node)
 
-    Generally, it is recommended to deploy one DM-worker instance on each node. Howerver, if the CPU and memory of your machine is much better than the required in [Hardware and Software Requirements](../op-guide/recommendation.md), and you have more than 2 disks in one node or the capacity of one SSD is larger than 2 TB, you can deploy no more than 2 DM-worker instances on a single node.
+    Generally, it is recommended to deploy one DM-worker instance on each node. Howerver, if the CPU and memory of your machine is much better than the required in [Hardware and Software Requirements](/op-guide/recommendation.md), and you have more than 2 disks in one node or the capacity of one SSD is larger than 2 TB, you can deploy no more than 2 DM-worker instances on a single node.
 
 - [The cluster topology of multiple DM-worker instances on each node](#option-2-use-the-cluster-topology-of-multiple-dm-worker-instances-on-each-node)
 

tools/dm-configuration-file-overview.md → tools/dm/dm-configuration-file-overview.md

@@ -10,7 +10,7 @@ This document gives an overview of configuration files of DM (Data Migration).
 
 ## DM process configuration files
 
-- `inventory.ini`: The configuration file of deploying DM using DM-Ansible. You need to edit it based on your machine topology. For details, see [Edit the `inventory.ini` file to orchestrate the DM cluster](../tools/data-migration-deployment.md#step-7-edit-the-inventoryini-file-to-orchestrate-the-dm-cluster).
+- `inventory.ini`: The configuration file of deploying DM using DM-Ansible. You need to edit it based on your machine topology. For details, see [Edit the `inventory.ini` file to orchestrate the DM cluster](/tools/dm/deployment.md#step-7-edit-the-inventoryini-file-to-orchestrate-the-dm-cluster).
 - `dm-master.toml`: The configuration file of running the DM-master process, including the topology information of the DM cluster and the corresponding relationship between the MySQL instance and DM-worker (must be one-to-one relationship). When you use DM-Ansible to deploy DM, `dm-master.toml` is generated automatically.
 - `dm-worker.toml`: The configuration file of running the DM-worker process, including the configuration information of upstream MySQL instance. When you use DM-Ansible to deploy DM, `dm-worker.toml` is generated automatically.
 
@@ -20,14 +20,14 @@ This document gives an overview of configuration files of DM (Data Migration).
 
 When you use DM-Ansible to deploy DM, you can find the following task configuration file template in `<path-to-dm-ansible>/conf`:
 
-- `task.yaml.exmaple`: The standard configuration file of the data synchronization task (a specific task corresponds to a `task.yaml`). For the introduction of the configuration file, see [Task Configuration File](../tools/dm-task-configuration-file-intro.md).
+- `task.yaml.exmaple`: The standard configuration file of the data synchronization task (a specific task corresponds to a `task.yaml`). For the introduction of the configuration file, see [Task Configuration File](/tools/dm/task-configuration-file-intro.md).
 
 ### Data synchronization task creation
 
 You can perform the following steps to create a data synchronization task based on `task.yaml.example`:
 
 1. Copy `task.yaml.example` as `your_task.yaml`.
-2. Refer to the description in the [Task Configuration File](../tools/dm-task-configuration-file-intro.md) and modify the configuration in `your_task.yaml`.
+2. Refer to the description in the [Task Configuration File](/tools/dm/task-configuration-file-intro.md) and modify the configuration in `your_task.yaml`.
 3. Create your data synchronization task using dmctl.
 
 ### Important concepts

tools/data-migration-manage-task.md → tools/dm/manage-task.md

@@ -6,7 +6,7 @@ category: tools
 
 # Manage the Data Synchronization Task
 
-This document describes how to manage and maintain the data synchronization task using the [dmctl](../tools/data-migration-overview.md#dmctl) component. For the Data Migration cluster deployed using DM-Ansible, the dmctl binary file is in `dm-ansible/dmctl`.
+This document describes how to manage and maintain the data synchronization task using the [dmctl](/tools/dm/overview.md#dmctl) component. For the Data Migration cluster deployed using DM-Ansible, the dmctl binary file is in `dm-ansible/dmctl`.
 
 ## dmctl basic usage
 
@@ -113,7 +113,7 @@ The user of the upstream and downstream databases must have the corresponding re
     TiDB differs from MySQL in compatibility in the following aspects:
 
     - Does not support the foreign key
-    - [Character set compatibility differs](../sql/character-set-support.md)
+    - [Character set compatibility differs](/sql/character-set-support.md)
 
 + The consistency check on the upstream MySQL multiple-instance shards
 
@@ -555,7 +555,7 @@ update-task [-w "127.0.0.1:10181"] ./task.yaml
 
 ## Manage the DDL locks
 
-See [Troubleshooting Sharding DDL Locks](../tools/troubleshooting-sharding-ddl-locks.md).
+See [Troubleshooting Sharding DDL Locks](/tools/dm/troubleshooting-sharding-ddl-locks.md).
 
 ## Refresh worker tasks
 

tools/dm-monitor.md → tools/dm/monitor.md

@@ -6,7 +6,7 @@ category: operations
 
 # Data Migration Monitoring Metrics
 
-If your DM cluster is deployed using DM-Ansible, the [monitoring system](../tools/data-migration-practice.md#step-7-monitor-the-task-and-check-logs) is also deployed at the same time. This document describes the monitoring metrics provided by DM-worker.
+If your DM cluster is deployed using DM-Ansible, the [monitoring system](/tools/dm/practice.md#step-7-monitor-the-task-and-check-logs) is also deployed at the same time. This document describes the monitoring metrics provided by DM-worker.
 
 > **Note:** Currently, DM-master does not provide monitoring metrics yet.