cherry pick #3083 to release-4.0 (#3298)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Liqi Geng <gengliqiii@gmail.com>

tikv-control.md

@@ -7,9 +7,33 @@ aliases: ['/docs/stable/tikv-control/','/docs/v4.0/tikv-control/','/docs/stable/
 
 # TiKV Control User Guide
 
-TiKV Control (`tikv-ctl`) is a command line tool of TiKV, used to manage the cluster.
+TiKV Control (`tikv-ctl`) is a command line tool of TiKV, used to manage the cluster. Its installation directory is as follows:
 
-When you compile TiKV, the `tikv-ctl` command is also compiled at the same time. If the cluster is deployed using TiDB Ansible, the `tikv-ctl` binary file exists in the corresponding `tidb-ansible/resources/bin` directory. If the cluster is deployed using the binary, the `tikv-ctl` file is in the `bin` directory together with other files such as `tidb-server`, `pd-server`, `tikv-server`, etc.
+* If the cluster is deployed using TiDB Ansible, `tikv-ctl` directory is in the `resources/bin` subdirectory under the `ansible` directory.
+* If the cluster is deployed using TiUP, `tikv-ctl` directory is in the in `~/.tiup/components/ctl/{VERSION}/` directory.
+
+[TiUP](https://github.com/pingcap-incubator/tiuptiup) is a deployment tool introduced later than TiDB Ansible, and its usage is simpler. `tikv-ctl` is also integrated in the `tiup` command. Execute the following command to call the `tikv-ctl` tool:
+
+{{< copyable "shell-regular" >}}
+
+```bash
+tiup ctl tikv
+```
+
+```
+Starting component `ctl`: ~/.tiup/components/ctl/v4.0.0-rc.2/ctl tikv
+TiKV Control (tikv-ctl)
+Release Version:   4.0.0-rc.2
+Edition:           Community
+Git Commit Hash:   2fdb2804bf8ffaab4b18c4996970e19906296497
+Git Commit Branch: heads/refs/tags/v4.0.0-rc.2
+UTC Build Time:    2020-05-15 11:58:49
+Rust Version:      rustc 1.42.0-nightly (0de96d37f 2019-12-19)
+Enable Features:   jemalloc portable sse protobuf-codec
+Profile:           dist_release
+```
+
+You can add corresponding parameters and subcommands after `tiup ctl tikv`.
 
 ## General options
 
@@ -149,7 +173,11 @@ The properties can be used to check whether the Region is healthy or not. If not
 
 ### Compact data of each TiKV manually
 
-Use the `compact` command to manually compact data of each TiKV. If you specify the `--from` and `--to` options, then their flags are also in the form of escaped raw key. You can use the `--db` option to specify the RocksDB that you need to compact. The optional values are `kv` and `raft`. Also, the `--threads` option allows you to specify the concurrency that you compact and its default value is 8. Generally, a higher concurrency comes with a faster compact speed, which might yet affect the service. You need to choose an appropriate concurrency based on the scenario.
+Use the `compact` command to manually compact data of each TiKV. If you specify the `--from` and `--to` options, then their flags are also in the form of escaped raw key. 
+
+- Use the `--host` option to specify the TiKV that you need to compact.
+- Use the `-d` option to specify the RocksDB that you need to compact. The optional values are `kv` and `raft`.
+- Use the `--threads` option allows you to specify the concurrency that you compact and its default value is 8. Generally, a higher concurrency comes with a faster compact speed, which might yet affect the service. You need to choose an appropriate concurrency based on the scenario.
 
 ```bash
 $ tikv-ctl --db /path/to/tikv/db compact -d kv
@@ -164,29 +192,39 @@ Use the `compact-cluster` command to manually compact data of the whole TiKV clu
 
 The `tombstone` command is usually used in circumstances where the sync-log is not enabled, and some data written in the Raft state machine is lost caused by power down.
 
-In a TiKV instance, you can use this command to set the status of some Regions to Tombstone. Then when you restart the instance, those Regions are skipped. Those Regions need to have enough healthy replicas in other TiKV instances to be able to continue writing and reading through the Raft mechanism.
+In a TiKV instance, you can use this command to set the status of some Regions to tombstone. Then when you restart the instance, those Regions are skipped to avoid the restart failure caused by damaged Raft state machines of those Regions. Those Regions need to have enough healthy replicas in other TiKV instances to be able to continue the reads and writes through the Raft mechanism.
 
-Follow the two steps to set a Region to Tombstone:
+In general cases, you can remove the corresponding Peer of this Region using the `remove-peer` command:
 
-1. Remove the corresponding Peer of this Region on the machine in `pd-ctl`:
+{{< copyable "shell-regular" >}}
 
-    {{< copyable "shell-regular" >}}
+```shell
+pd-ctl operator add remove-peer <region_id> <store_id>
+```
 
-    ```shell
-    pd-ctl operator add remove-peer <region_id> <store_id>
-    ```
+Then use the `tikv-ctl` tool to set a Region to tombstone on the corresponding TiKV instance to skip the health check for this Region at startup:
+
+{{< copyable "shell-regular" >}}
 
-2. Use the `tombstone` command to set a Region to Tombstone:
+```shell
+tikv-ctl --db /path/to/tikv/db tombstone -p 127.0.0.1:2379 -r <region_id>
+```
 
-    {{< copyable "shell-regular" >}}
+```
+success!
+```
 
-    ```shell
-    tikv-ctl --db /path/to/tikv/db tombstone -p 127.0.0.1:2379 -r <region_id>
-    ```
+However, in some cases, you cannot easily remove this Peer of this Region from PD, so you can specify the `--force` option in `tikv-ctl` to forcibly set the Peer to tombstone:
 
-    ```
-    success!
-    ```
+{{< copyable "shell-regular" >}}
+
+```shell
+tikv-ctl --db /path/to/tikv/db tombstone -p 127.0.0.1:2379 -r <region_id>,<region_id> --force
+```
+
+```
+success!
+```
 
 > **Note:**
 >
@@ -277,7 +315,8 @@ success!
 
 Use the `recover-mvcc` command in circumstances where TiKV cannot run normally caused by MVCC data corruption. It cross-checks 3 CFs ("default", "write", "lock") to recover from various kinds of inconsistency.
 
-Use the `-r` option to specify involved Regions by `region_id`. Use the `-p` option to specify PD endpoints.
+- Use the `-r` option to specify involved Regions by `region_id`. 
+- Use the `-p` option to specify PD endpoints.
 
 ```bash
 $ tikv-ctl --db /path/to/tikv/db recover-mvcc -r 1001,1002 -p 127.0.0.1:2379
@@ -292,7 +331,7 @@ success!
 
 ### Ldb Command
 
-The ldb command line tool offers multiple data access and database administration commands. Some examples are listed below.
+The `ldb` command line tool offers multiple data access and database administration commands. Some examples are listed below.
 For more information, refer to the help message displayed when running `tikv-ctl ldb` or check the documents from RocksDB.
 
 Examples of data access sequence: