diff --git a/TOC.md b/TOC.md index 1cc78c49c69d9..2b31ed3295e88 100644 --- a/TOC.md +++ b/TOC.md @@ -11,9 +11,6 @@ + [TiDB 4.0 Experimental Features](/experimental-features-4.0.md) + [Basic Features](/basic-features.md) + Benchmarks - + [v4.0 Sysbench Performance Test](/benchmark/benchmark-sysbench-v4-vs-v3.md) - + [v4.0 TPC-H Performance Test](/benchmark/v4.0-performance-benchmarking-with-tpch.md) - + [v4.0 TPC-C Performance Test](/benchmark/v4.0-performance-benchmarking-with-tpcc.md) + [Interaction Test on Online Workloads and `ADD INDEX`](/benchmark/online-workloads-and-add-index-operations.md) + [MySQL Compatibility](/mysql-compatibility.md) + [TiDB Limitations](/tidb-limitations.md) @@ -39,8 +36,6 @@ + [Use TiUP (Recommended)](/production-deployment-using-tiup.md) + [Use TiUP Offline (Recommended)](/production-offline-deployment-using-tiup.md) + [Deploy in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/stable) - + [Use TiDB Ansible](/online-deployment-using-ansible.md) - + [Use TiDB Ansible Offline](/offline-deployment-using-ansible.md) + [Verify Cluster Status](/post-installation-check.md) + Migrate + [Overview](/migration-overview.md) @@ -56,10 +51,8 @@ + [Use TiUP (Recommended)](/upgrade-tidb-using-tiup.md) + [Use TiUP Offline (Recommended)](/upgrade-tidb-using-tiup-offline.md) + [Use TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/upgrade-a-tidb-cluster) - + [Use TiDB Ansible](/upgrade-tidb-using-ansible.md) + Scale + [Use TiUP (Recommended)](/scale-tidb-using-tiup.md) - + [Use TiDB Ansible](/scale-tidb-using-ansible.md) + [Use TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/scale-a-tidb-cluster) + Backup and Restore + Use BR Tool (Recommended) @@ -73,7 +66,6 @@ + [Daily Checklist](/daily-check.md) + [Maintain TiFlash](/tiflash/maintain-tiflash.md) + [Maintain TiDB Using TiUP](/maintain-tidb-using-tiup.md) - + [Maintain TiDB Using Ansible](/maintain-tidb-using-ansible.md) + [Modify Configuration Online](/dynamic-config.md) + Monitor and Alert + [Monitoring Framework Overview](/tidb-monitoring-framework.md) diff --git a/best-practices/grafana-monitor-best-practices.md b/best-practices/grafana-monitor-best-practices.md index 3683898b9ffdb..5fb569a29d538 100644 --- a/best-practices/grafana-monitor-best-practices.md +++ b/best-practices/grafana-monitor-best-practices.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/best-practices/grafana-monitor-best-practices/','/docs/dev/ # Best Practices for Monitoring TiDB Using Grafana -When you [deploy a TiDB cluster using TiDB Ansible](/online-deployment-using-ansible.md), a set of [Grafana + Prometheus monitoring platform](/tidb-monitoring-framework.md) is deployed simultaneously to collect and display metrics for various components and machines in the TiDB cluster. This document describes best practices for monitoring TiDB using Grafana. It aims to help you use metrics to analyze the status of the TiDB cluster and diagnose problems. +When you [deploy a TiDB cluster using TiUP](/production-deployment-using-tiup.md) and have added Grafana and Prometheus in the topology configuration, a set of [Grafana + Prometheus monitoring platform](/tidb-monitoring-framework.md) is deployed simultaneously to collect and display metrics for various components and machines in the TiDB cluster. This document describes best practices for monitoring TiDB using Grafana. It aims to help you use metrics to analyze the status of the TiDB cluster and diagnose problems. ## Monitoring architecture @@ -17,7 +17,7 @@ When you [deploy a TiDB cluster using TiDB Ansible](/online-deployment-using-ans For TiDB 2.1.3 or later versions, TiDB monitoring supports the pull method. It is a good adjustment with the following benefits: - There is no need to restart the entire TiDB cluster if you need to migrate Prometheus. Before adjustment, migrating Prometheus requires restarting the entire cluster because the target address needs to be updated. -- You can deploy 2 separate sets of Grafana + Prometheus monitoring platforms (not highly available) to prevent a single point of monitoring. To do this, execute the deployment command of TiDB ansible twice with different IP addresses. +- You can deploy 2 separate sets of Grafana + Prometheus monitoring platforms (not highly available) to prevent a single point of monitoring. - The Pushgateway which might become a single point of failure is removed. ## Source and display of monitoring data @@ -203,4 +203,4 @@ curl -u user:pass 'http://__grafana_ip__:3000/api/datasources/proxy/1/api/v1/que ## Summary -The Grafana + Prometheus monitoring platform is a very powerful tool. Making good use of it can improve efficiency, saving you a lot of time on analyzing the status of the TiDB cluster. More importantly, it can help you diagnose problems. This tool is very useful in the operation and maintenance of TiDB clusters, especially when there is a large amount of data. \ No newline at end of file +The Grafana + Prometheus monitoring platform is a very powerful tool. Making good use of it can improve efficiency, saving you a lot of time on analyzing the status of the TiDB cluster. More importantly, it can help you diagnose problems. This tool is very useful in the operation and maintenance of TiDB clusters, especially when there is a large amount of data. diff --git a/dashboard/dashboard-diagnostics-access.md b/dashboard/dashboard-diagnostics-access.md index 97315586f6f94..ce064392e9679 100644 --- a/dashboard/dashboard-diagnostics-access.md +++ b/dashboard/dashboard-diagnostics-access.md @@ -14,7 +14,7 @@ The cluster diagnostics feature in TiDB Dashboard diagnoses the problems that mi > **Note:** > -> The cluster diagnostics feature depends on Prometheus deployed in the cluster. For details about how to deploy this monitoring component, see [TiUP](/tiup/tiup-overview.md) or [TiDB Ansible](/online-deployment-using-ansible.md) deployment document. If no monitoring component is deployed in the cluster, the generated diagnostic report will indicate a failure. +> The cluster diagnostics feature depends on Prometheus deployed in the cluster. For details about how to deploy this monitoring component, see the [TiUP](/tiup/tiup-overview.md) deployment document. If no monitoring component is deployed in the cluster, the generated diagnostic report will indicate a failure. ## Access the page diff --git a/dashboard/dashboard-faq.md b/dashboard/dashboard-faq.md index fca8279c95d81..8dac0ef972f23 100644 --- a/dashboard/dashboard-faq.md +++ b/dashboard/dashboard-faq.md @@ -30,7 +30,7 @@ If you have deployed TiDB using the `tiup cluster` or `tiup playground` command, The **QPS** and **Latency** sections on the **Overview** page require a cluster with Prometheus deployed. Otherwise, the error is shown. You can solve this problem by deploying a Prometheus instance in the cluster. -If you still encounter this problem when the Prometheus instance has been deployed, the possible reason is that your deployment tool is out of date (TiUP, TiDB Operator, or TiDB Ansible), and your tool does not automatically report metrics addresses, which makes TiDB Dashboard unable to query metrics. You can upgrade you deployment tool to the latest version and try again. +If you still encounter this problem when the Prometheus instance has been deployed, the possible reason is that your deployment tool is out of date (TiUP or TiDB Operator), and your tool does not automatically report metrics addresses, which makes TiDB Dashboard unable to query metrics. You can upgrade you deployment tool to the latest version and try again. If your deployment tool is TiUP, take the following steps to solve this problem. For other deployment tools, refer to the corresponding documents of those tools. diff --git a/dashboard/dashboard-overview.md b/dashboard/dashboard-overview.md index cac38b5ca9b7f..baa49107e5ba8 100644 --- a/dashboard/dashboard-overview.md +++ b/dashboard/dashboard-overview.md @@ -65,7 +65,7 @@ The content displayed in this area is consistent with the more detailed [Slow Qu > **Note:** > -> This feature is available only in the cluster with slow query logs enabled. By default, slow query logs are enabled in the cluster deployed using TiUP or TiDB Ansible. +> This feature is available only in the cluster with slow query logs enabled. By default, slow query logs are enabled in the cluster deployed using TiUP. ## Instances diff --git a/faq/deploy-and-maintain-faq.md b/faq/deploy-and-maintain-faq.md index 561cbd189c798..c417317a31b4a 100644 --- a/faq/deploy-and-maintain-faq.md +++ b/faq/deploy-and-maintain-faq.md @@ -89,7 +89,7 @@ Check the time difference between the machine time of the monitor and the time w | Variable | Description | | ---- | ------- | | `cluster_name` | the name of a cluster, adjustable | -| `tidb_version` | the version of TiDB, configured by default in TiDB Ansible branches | +| `tidb_version` | the version of TiDB | | `deployment_method` | the method of deployment, binary by default, Docker optional | | `process_supervision` | the supervision way of processes, systemd by default, supervise optional | | `timezone` | the timezone of the managed node, adjustable, `Asia/Shanghai` by default, used with the `set_timezone` variable | @@ -104,25 +104,13 @@ Check the time difference between the machine time of the monitor and the time w | `enable_slow_query_log` | to record the slow query log of TiDB into a single file: ({{ deploy_dir }}/log/tidb_slow_query.log). False by default, to record it into the TiDB log | | `deploy_without_tidb` | the Key-Value mode, deploy only PD, TiKV and the monitoring service, not TiDB; set the IP of the tidb_servers host group to null in the `inventory.ini` file | -### Deploy TiDB offline using TiDB Ansible（not recommended since TiDB v4.0) - -> **Warning:** -> -> It is not recommended to deploy TiDB using TiDB Ansible since TiDB v4.0. [Use TiUP to deploy TiDB](/production-deployment-using-tiup.md) instead. - -If the central control machine cannot access the Internet, you can [deploy TiDB offline using TiDB Ansible](https://docs.pingcap.com/tidb/stable/offline-deployment-using-ansible). - ### How to deploy TiDB quickly using Docker Compose on a single machine? You can use Docker Compose to build a TiDB cluster locally, including the cluster monitoring components. You can also customize the version and number of instances for each component. The configuration file can also be customized. You can only use this deployment method for testing and development environment. For details, see [TiDB Docker Compose Deployment](/deploy-test-cluster-using-docker-compose.md). ### How to separately record the slow query log in TiDB? How to locate the slow query SQL statement? -1. The slow query definition for TiDB is in the `conf/tidb.yml` configuration file of `tidb-ansible`. The `slow-threshold: 300` parameter is used to configure the threshold value of the slow query (unit: millisecond). - - The slow query log is recorded in `tidb.log` by default. If you want to generate a slow query log file separately, set `enable_slow_query_log` in the `inventory.ini` configuration file to `True`. - - Then run `ansible-playbook rolling_update.yml --tags=tidb` to perform a rolling update on the `tidb-server` instance. After the update is finished, the `tidb-server` instance will record the slow query log in `tidb_slow_query.log`. +1. The slow query definition for TiDB is in the TiDB configuration file. The `slow-threshold: 300` parameter is used to configure the threshold value of the slow query (unit: millisecond). 2. If a slow query occurs, you can locate the `tidb-server` instance where the slow query is and the slow query time point using Grafana and find the SQL statement information recorded in the log on the corresponding node. @@ -156,30 +144,10 @@ The Direct mode wraps the Write request into the I/O command and sends this comm ./fio -ioengine=psync -bs=32k -fdatasync=1 -thread -rw=randrw -percentage_random=100,0 -size=10G -filename=fio_randread_write_test.txt -name='fio mixed randread and sequential write test' -iodepth=4 -runtime=60 -numjobs=4 -group_reporting --output-format=json --output=fio_randread_write_test.json ``` -### Error `UNREACHABLE! "msg": "Failed to connect to the host via ssh: "` when deploying TiDB using TiDB Ansible - -Two possible reasons and solutions: - -- The SSH mutual trust is not configured as required. It’s recommended to follow [the steps described in the official document](/online-deployment-using-ansible.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine) and check whether it is successfully configured using `ansible -i inventory.ini all -m shell -a 'whoami' -b`. -- If it involves the scenario where a single server is assigned multiple roles, for example, the mixed deployment of multiple components or multiple TiKV instances are deployed on a single server, this error might be caused by the SSH reuse mechanism. You can use the option of `ansible … -f 1` to avoid this error. - ## Cluster management ### Daily management -#### What are the common operations of TiDB Ansible? - -| Job | Playbook | -|:----------------------------------|:-----------------------------------------| -| Start the cluster | `ansible-playbook start.yml` | -| Stop the cluster | `ansible-playbook stop.yml` | -| Destroy the cluster | `ansible-playbook unsafe_cleanup.yml` (If the deployment directory is a mount point, an error will be reported, but implementation results will remain unaffected) | -| Clean data (for test) | `ansible-playbook unsafe_cleanup_data.yml` | -| Apply rolling updates | `ansible-playbook rolling_update.yml` | -| Apply rolling updates to TiKV | `ansible-playbook rolling_update.yml --tags=tikv` | -| Apply rolling updates to components except PD | `ansible-playbook rolling_update.yml --skip-tags=pd` | -| Apply rolling updates to the monitoring components | `ansible-playbook rolling_update_monitor.yml` | - #### How to log into TiDB? You can log into TiDB like logging into MySQL. For example: @@ -206,7 +174,7 @@ By default, TiDB/PD/TiKV outputs standard error in the logs. If a log file is sp #### How to safely stop TiDB? -If the cluster is deployed using TiDB Ansible, you can use the `ansible-playbook stop.yml` command to stop the TiDB cluster. If the cluster is not deployed using TiDB Ansible, `kill` all the services directly. The components of TiDB will do `graceful shutdown`. +Kill all the services using `kill` directly. The components of TiDB will do `graceful shutdown`. #### Can `kill` be executed in TiDB? @@ -227,11 +195,11 @@ Take `Release Version: v1.0.3-1-ga80e796` as an example of version number descri - `-1` indicates the current version has one commit. - `ga80e796` indicates the version `git-hash`. -#### What's the difference between various TiDB master versions? How to avoid using the wrong TiDB Ansible version? +#### What's the difference between various TiDB master versions? The TiDB community is highly active. After the 1.0 GA release, the engineers have been keeping optimizing and fixing bugs. Therefore, the TiDB version is updated quite fast. If you want to keep informed of the latest version, see [TiDB Weekly update](https://pingcap.com/weekly/). -It is not recommended to deploy the TiDB cluster using TiDB Ansible. [Deploy TiDB using TiUP](/production-deployment-using-tiup.md) instead. TiDB has a unified management of the version number after the 1.0 GA release. You can view the version number using the following two methods: +It is recommeneded to [deploy TiDB using TiUP](/production-deployment-using-tiup.md). TiDB has a unified management of the version number after the 1.0 GA release. You can view the version number using the following two methods: - `select tidb_version()` - `tidb-server -V` @@ -316,11 +284,6 @@ The offline node usually indicates the TiKV node. You can determine whether the 1. Manually stop the relevant services on the offline node. 2. Delete the `node_exporter` data of the corresponding node from the Prometheus configuration file. -3. Delete the data of the corresponding node from Ansible `inventory.ini`. - -#### Why couldn't I connect to the PD server using `127.0.0.1` when I was using the PD Control? - -If your TiDB cluster is deployed using TiDB Ansible, the PD external service port is not bound to `127.0.0.1`, so PD Control does not recognize `127.0.0.1` and you can only connect to it using the local IP address. ### TiDB server management diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index 501f5780e4c42..10271f0ea584f 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -76,18 +76,6 @@ See [Parsing TiDB online data synchronization tool Syncer](https://pingcap.com/b See [Syncer User Guide](/syncer-overview.md). -#### How to configure to monitor Syncer status? - -Download and import [Syncer Json](https://github.com/pingcap/docs/blob/master/etc/Syncer.json) to Grafana. Edit the Prometheus configuration file and add the following content: - -``` -- job_name: 'syncer_ops' // task name - static_configs: - - targets: [’10.10.1.1:10096’] // Syncer monitoring address and port, informing Prometheus to pull the data of Syncer -``` - -Restart Prometheus. - #### Is there a current solution to replicating data from TiDB to other databases like HBase and Elasticsearch? No. Currently, the data replication depends on the application itself. diff --git a/get-started-with-tispark.md b/get-started-with-tispark.md index 2ed6276c87b39..363b235b23814 100644 --- a/get-started-with-tispark.md +++ b/get-started-with-tispark.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/get-started-with-tispark/','/docs/dev/how-to/get-started/ti # TiSpark Quick Start Guide -To make it easy to [try TiSpark](/tispark-overview.md), the TiDB cluster installed using TiDB Ansible integrates Spark, TiSpark jar package and TiSpark sample data by default. +To make it easy to [try TiSpark](/tispark-overview.md), the TiDB cluster installed using TiUP integrates Spark and TiSpark jar package by default. ## Deployment information diff --git a/grafana-overview-dashboard.md b/grafana-overview-dashboard.md index 4cd8bc9467cef..f173eca3a1eb2 100644 --- a/grafana-overview-dashboard.md +++ b/grafana-overview-dashboard.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/grafana-overview-dashboard/','/docs/dev/reference/key-monit # Key Metrics -If you use TiDB Ansible or TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [TiDB Monitoring Framework Overview](/tidb-monitoring-framework.md). +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [TiDB Monitoring Framework Overview](/tidb-monitoring-framework.md). The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and so on. A lot of metrics are there to help you diagnose. diff --git a/grafana-pd-dashboard.md b/grafana-pd-dashboard.md index aa36392d5f261..ba35780dc11a3 100644 --- a/grafana-pd-dashboard.md +++ b/grafana-pd-dashboard.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/grafana-pd-dashboard/','/docs/dev/reference/key-monitoring- # Key Monitoring Metrics of PD -If you use TiUP or TiDB Ansible to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and so on. A lot of metrics are there to help you diagnose. diff --git a/grafana-tidb-dashboard.md b/grafana-tidb-dashboard.md index a7305847c34ff..a39b7779aed06 100644 --- a/grafana-tidb-dashboard.md +++ b/grafana-tidb-dashboard.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/grafana-tidb-dashboard/','/docs/dev/reference/key-monitorin # TiDB Monitoring Metrics -If you use TiDB Ansible or TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For the monitoring architecture, see [TiDB Monitoring Framework Overview](/tidb-monitoring-framework.md). +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For the monitoring architecture, see [TiDB Monitoring Framework Overview](/tidb-monitoring-framework.md). The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and so on. The TiDB dashboard consists of the TiDB panel and the TiDB Summary panel. The differences between the two panels are different in the following aspects: diff --git a/grafana-tikv-dashboard.md b/grafana-tikv-dashboard.md index 9c3eb3b7dea21..4b39d0b6f647d 100644 --- a/grafana-tikv-dashboard.md +++ b/grafana-tikv-dashboard.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/grafana-tikv-dashboard/','/docs/dev/reference/key-monitorin # Key Monitoring Metrics of TiKV -If you use TiUP or TiDB Ansible to deploy the TiDB cluster, the monitoring system (Prometheus/Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus/Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, and so on. A lot of metrics are there to help you diagnose. diff --git a/maintain-tidb-using-ansible.md b/maintain-tidb-using-ansible.md deleted file mode 100644 index 1f5c2234dce35..0000000000000 --- a/maintain-tidb-using-ansible.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -title: TiDB Ansible Common Operations -summary: Learn some common operations when using TiDB Ansible to administer a TiDB cluster. -aliases: ['/docs/dev/maintain-tidb-using-ansible/','/docs/dev/how-to/deploy/orchestrated/ansible-operations/'] ---- - -# TiDB Ansible Common Operations - -This guide describes the common operations when you administer a TiDB cluster using TiDB Ansible. - -## Start a cluster - -```bash -$ ansible-playbook start.yml -``` - -This operation starts all the components in the entire TiDB cluster in order, which include PD, TiDB, TiKV, and the monitoring components. - -## Stop a cluster - -```bash -$ ansible-playbook stop.yml -``` - -This operation stops all the components in the entire TiDB cluster in order, which include PD, TiDB, TiKV, and the monitoring components. - -## Clean up cluster data - -``` -$ ansible-playbook unsafe_cleanup_data.yml -``` - -This operation stops the TiDB, Pump, TiKV and PD services, and cleans up the data directory of Pump, TiKV and PD. - -## Destroy a cluster - -``` -$ ansible-playbook unsafe_cleanup.yml -``` - -This operation stops the cluster and cleans up the data directory. - -> **Note:** -> -> If the deployment directory is a mount point, an error will be reported, but implementation results remain unaffected, so you can ignore it. diff --git a/maintain-tidb-using-tiup.md b/maintain-tidb-using-tiup.md index 8c7d8593bfcd3..1d93f8bba06de 100644 --- a/maintain-tidb-using-tiup.md +++ b/maintain-tidb-using-tiup.md @@ -1,7 +1,7 @@ --- title: TiUP Common Operations summary: Learn the common operations to operate and maintain a TiDB cluster using TiUP. -aliases: ['/docs/dev/maintain-tidb-using-tiup/','/docs/dev/how-to/maintain/tiup-operations/'] +aliases: ['/docs/dev/maintain-tidb-using-tiup/','/docs/dev/how-to/maintain/tiup-operations/','/docs/dev/maintain-tidb-using-ansible/','/docs/dev/how-to/deploy/orchestrated/ansible-operations/','/tidb/dev/maintain-tidb-using-ansible/'] --- # TiUP Common Operations @@ -15,6 +15,10 @@ This document describes the following common operations when you operate and mai - Stop the cluster - Destroy the cluster +> **Note:** +> +> Since TiDB v4.0, PingCAP no longer provides support for TiDB Ansible. Since TiDB v5.0, PingCAP no longer provides TiDB Ansible documents. If you want to read the document that introduces how to maintain a TiDB cluster using TiDB Ansible, see [TiDB Ansible Common Operations](https://docs.pingcap.com/tidb/v4.0/maintain-tidb-using-ansible). + ## View the cluster list You can manage multiple TiDB clusters using the TiUP cluster component. When a TiDB cluster is deployed, the cluster appears in the TiUP cluster list. diff --git a/offline-deployment-using-ansible.md b/offline-deployment-using-ansible.md deleted file mode 100644 index 341158cf0c9de..0000000000000 --- a/offline-deployment-using-ansible.md +++ /dev/null @@ -1,186 +0,0 @@ ---- -title: Deploy TiDB Offline Using TiDB Ansible -summary: Use TiDB Ansible to deploy a TiDB cluster offline. -aliases: ['/docs/dev/offline-deployment-using-ansible/','/docs/dev/how-to/deploy/orchestrated/offline-ansible/'] ---- - -# Deploy TiDB Offline Using TiDB Ansible - -> **Warning:** -> -> For production environments, it is recommended that you [deploy TiDB using TiUP offline](/production-offline-deployment-using-tiup.md). Since v4.0, PingCAP no longer provides support for deploying TiDB using TiDB Ansible (deprecated). If you really need to use it for deployment, be aware of any risk. You can [import the TiDB cluster deployed by TiDB Ansible to TiUP](/upgrade-tidb-using-tiup.md#import-the-tidb-ansible-cluster-to-tiup). -> -> If you only want to try out TiDB and explore new features, refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md). - -This guide describes how to deploy a TiDB cluster offline using TiDB Ansible. - -## Prepare - -Before you start, make sure that you have: - -1. A download machine - - - The machine must have access to the Internet in order to download TiDB Ansible, TiDB and related packages. - - For Linux operating system, it is recommended to install CentOS 7.3 or later. - -2. Several target machines and one control machine - - - For system requirements and configuration, see [Prepare the environment](/online-deployment-using-ansible.md#prepare). - - It is acceptable without access to the Internet. - -## Step 1: Install system dependencies on the control machine - -Take the following steps to install system dependencies on the control machine installed with the CentOS 7 system. - -1. Download the [`pip`](https://download.pingcap.org/ansible-system-rpms.el7.tar.gz) offline installation package on the download machine and then upload it to the control machine. - - > **Note:** - > - > This offline installation package includes `pip` and `sshpass`, and only supports the CentOS 7 system. - -2. Install system dependencies on the control machine. - - {{< copyable "shell-root" >}} - - ```bash - tar -xzvf ansible-system-rpms.el7.tar.gz && - cd ansible-system-rpms.el7 && - chmod u+x install_ansible_system_rpms.sh && - ./install_ansible_system_rpms.sh - ``` - -3. After the installation is finished, you can use `pip -V` to check whether it is successfully installed. - - {{< copyable "shell-root" >}} - - ```bash - pip -V - ``` - - ``` - pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7) - ``` - -> **Note:** -> -> If `pip` is already installed to your system, make sure that the version is 8.1.2 or later. Otherwise, compatibility error occurs when you install TiDB Ansible and its dependencies offline. - -## Step 2: Create the `tidb` user on the control machine and generate the SSH key - -See [Create the `tidb` user on the control machine and generate the SSH key](/online-deployment-using-ansible.md#step-2-create-the-tidb-user-on-the-control-machine-and-generate-the-ssh-key). - -## Step 3: Install TiDB Ansible and its dependencies offline on the control machine - -Currently, all the versions of TiDB Ansible from 2.4 to 2.7.11 are supported. The versions of TiDB Ansible and the related dependencies are listed in the `tidb-ansible/requirements.txt` file. The following installation steps take Ansible 2.5 as an example. - -1. Download [Ansible 2.5 offline installation package](https://download.pingcap.org/ansible-2.5.0-pip.tar.gz) on the download machine and then upload it to the control machine. - -2. Install TiDB Ansible and its dependencies offline. - - {{< copyable "shell-root" >}} - - ```bash - tar -xzvf ansible-2.5.0-pip.tar.gz && - cd ansible-2.5.0-pip/ && - chmod u+x install_ansible.sh && - ./install_ansible.sh - ``` - -3. View the version of TiDB Ansible. - - After TiDB Ansible is installed, you can view the version using `ansible --version`. - - {{< copyable "shell-root" >}} - - ```bash - ansible --version - ``` - - ``` - ansible 2.5.0 - ``` - -## Step 4: Download TiDB Ansible and TiDB packages on the download machine - -1. Install TiDB Ansible on the download machine. - - Use the following method to install Ansible online on the download machine installed with the CentOS 7 system. After TiDB Ansible is installed, you can view the version using `ansible --version`. - - {{< copyable "shell-root" >}} - - ```bash - yum install epel-release && - yum install ansible curl && - ansible --version - ``` - - ``` - ansible 2.5.0 - ``` - - > **Note:** - > - > Make sure that the version of Ansible is 2.5, otherwise a compatibility issue occurs. - -2. Download TiDB Ansible. - - Use the following command to download the corresponding version of TiDB Ansible from the [TiDB Ansible project](https://github.com/pingcap/tidb-ansible) GitHub repo. The default folder name is `tidb-ansible`. - - {{< copyable "shell-regular" >}} - - ```bash - git clone https://github.com/pingcap/tidb-ansible.git - ``` - - > **Note:** - > - > It is required to use the corresponding tidb-ansible version when you deploy and upgrade the TiDB cluster. If you deploy TiDB using a mismatched version of tidb-ansible (such as using tidb-ansible v2.1.4 to deploy TiDB v2.1.6), an error might occur. - -3. Run the `local_prepare.yml` playbook, and download TiDB binary online to the download machine. - - {{< copyable "shell-regular" >}} - - ```bash - cd tidb-ansible && - ansible-playbook local_prepare.yml - ``` - -4. After running the above command, copy the `tidb-ansible` folder to the `/home/tidb` directory of the control machine. The ownership authority of the file must be the `tidb` user. - -## Step 5: Configure the SSH mutual trust and sudo rules on the control machine - -See [Configure the SSH mutual trust and sudo rules on the control machine](/online-deployment-using-ansible.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine). - -## Step 6: Install the NTP service on the target machines - -See [Install the NTP service on the target machines](/online-deployment-using-ansible.md#step-6-install-the-ntp-service-on-the-target-machines). - -> **Note:** -> -> If the time and time zone of all your target machines are same, the NTP service is on and is normally synchronizing time, you can skip this step. See [How to check whether the NTP service is normal](/online-deployment-using-ansible.md#how-to-check-whether-the-ntp-service-is-normal). - -## Step 7: Configure the CPUfreq governor mode on the target machine - -See [Configure the CPUfreq governor mode on the target machine](/online-deployment-using-ansible.md#step-7-configure-the-cpufreq-governor-mode-on-the-target-machine). - -## Step 8: Mount the data disk ext4 filesystem with options on the target machines - -See [Mount the data disk ext4 filesystem with options on the target machines](/online-deployment-using-ansible.md#step-8-mount-the-data-disk-ext4-filesystem-with-options-on-the-target-machines). - -## Step 9: Edit the `inventory.ini` file to orchestrate the TiDB cluster - -See [Edit the `inventory.ini` file to orchestrate the TiDB cluster](/online-deployment-using-ansible.md#step-9-edit-the-inventoryini-file-to-orchestrate-the-tidb-cluster). - -## Step 10: Deploy the TiDB cluster - -1. You do not need to run the playbook in `ansible-playbook local_prepare.yml`. - -2. See [Deploy the TiDB cluster](/online-deployment-using-ansible.md#step-11-deploy-the-tidb-cluster). - -## Test the TiDB cluster - -See [Test the TiDB cluster](/online-deployment-using-ansible.md#test-the-tidb-cluster). - -> **Note:** -> -> By default, TiDB periodically shares usage details with PingCAP to help understand how to improve the product. For details about what is shared and how to disable the sharing, see [Telemetry](/telemetry.md). diff --git a/online-deployment-using-ansible.md b/online-deployment-using-ansible.md deleted file mode 100644 index 30d3121d54251..0000000000000 --- a/online-deployment-using-ansible.md +++ /dev/null @@ -1,992 +0,0 @@ ---- -title: Deploy TiDB Using TiDB Ansible -summary: Use TiDB Ansible to deploy a TiDB cluster. -aliases: ['/docs/dev/online-deployment-using-ansible/','/docs/dev/how-to/deploy/orchestrated/ansible/'] ---- - -# Deploy TiDB Using TiDB Ansible - -> **Warning:** -> -> For production environments, it is recommended that you [deploy TiDB using TiUP](/production-deployment-using-tiup.md). Since v4.0, PingCAP no longer provides support for deploying TiDB using TiDB Ansible (deprecated). If you really need to use it for deployment, be aware of any risk. You can [import the TiDB cluster deployed by TiDB Ansible to TiUP](/upgrade-tidb-using-tiup.md#import-the-tidb-ansible-cluster-to-tiup). -> -> If you only want to try out TiDB and explore new features, refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md). - -This guide describes how to deploy a TiDB cluster using TiDB Ansible. For the production environment, it is recommended to deploy TiDB using TiUP. - -## Overview - -Ansible is an IT automation tool that can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates. - -[TiDB Ansible](https://github.com/pingcap/tidb-ansible) is a TiDB cluster deployment tool developed by PingCAP, based on Ansible playbook. TiDB Ansible enables you to quickly deploy a new TiDB cluster which includes PD, TiDB, TiKV, and the cluster monitoring modules. - -You can use the TiDB Ansible configuration file to set up the cluster topology and complete all the following operation tasks: - -- Initialize operating system parameters -- Deploy the whole TiDB cluster -- [Start the TiDB cluster](/maintain-tidb-using-ansible.md#start-a-cluster) -- [Stop the TiDB cluster](/maintain-tidb-using-ansible.md#stop-a-cluster) -- [Modify component configuration](/upgrade-tidb-using-ansible.md#edit-the-configuration-file-of-tidb-cluster-components) -- [Scale the TiDB cluster](/scale-tidb-using-ansible.md) -- [Upgrade the component version](/upgrade-tidb-using-ansible.md#step-5-perform-a-rolling-update-to-tidb-cluster-components) -- [Enable the cluster binlog](/tidb-binlog/tidb-binlog-overview.md) -- [Clean up data of the TiDB cluster](/maintain-tidb-using-ansible.md#clean-up-cluster-data) -- [Destroy the TiDB cluster](/maintain-tidb-using-ansible.md#destroy-a-cluster) - -## Prepare - -Before you start, make sure you have: - -1. Several target machines that meet the following requirements: - - - 4 or more machines - - A standard TiDB cluster contains 6 machines. You can use 4 machines for testing. For more details, see [Software and Hardware Recommendations](/hardware-and-software-requirements.md). - - - x86_64 architecture (AMD64) with CentOS 7.3 (64 bit) or later; Or x86_64 architecture (ARM64) with CentOS 7.6 1810 or later. - - Network between machines - - > **Note:** - > - > When you deploy TiDB using TiDB Ansible, **use SSD disks for the data directory of TiKV and PD nodes**. Otherwise, it cannot pass the check. If you only want to try TiDB out and explore the features, it is recommended to [deploy TiDB using Docker Compose](/deploy-test-cluster-using-docker-compose.md) on a single machine. - -2. A control machine that meets the following requirements: - - > **Note:** - > - > The control machine can be one of the target machines. - - - CentOS 7.3 (64 bit) or later with Python 2.7 installed - - Access to the Internet - -## Step 1: Install system dependencies on the control machine - -Log in to the control machine using the `root` user account, and run the corresponding command according to your operating system. - -- If you use a control machine installed with CentOS 7, run the following command: - - {{< copyable "shell-root" >}} - - ```shell - yum -y install epel-release git curl sshpass && \ - yum -y install python2-pip - ``` - -- If you use a control machine installed with Ubuntu, run the following command: - - {{< copyable "shell-root" >}} - - ```shell - apt-get -y install git curl sshpass python-pip - ``` - -## Step 2: Create the `tidb` user on the control machine and generate the SSH key - -Make sure you have logged in to the control machine using the `root` user account, and then run the following command. - -1. Create the `tidb` user. - - {{< copyable "shell-root" >}} - - ```shell - useradd -m -d /home/tidb tidb - ``` - -2. Set a password for the `tidb` user account. - - {{< copyable "shell-root" >}} - - ```shell - passwd tidb - ``` - -3. Configure sudo without password for the `tidb` user account by adding `tidb ALL=(ALL) NOPASSWD: ALL` to the end of the sudo file: - - {{< copyable "shell-root" >}} - - ```shell - visudo - ``` - - ``` - tidb ALL=(ALL) NOPASSWD: ALL - ``` - -4. Generate the SSH key. - - Execute the `su` command to switch the user from `root` to `tidb`. - - {{< copyable "shell-root" >}} - - ```shell - su - tidb - ``` - - Create the SSH key for the `tidb` user account and hit the Enter key when `Enter passphrase` is prompted. After successful execution, the SSH private key file is `/home/tidb/.ssh/id_rsa`, and the SSH public key file is `/home/tidb/.ssh/id_rsa.pub`. - - {{< copyable "shell-regular" >}} - - ```shell - ssh-keygen -t rsa - ``` - - ``` - Generating public/private rsa key pair. - Enter file in which to save the key (/home/tidb/.ssh/id_rsa): - Created directory '/home/tidb/.ssh'. - Enter passphrase (empty for no passphrase): - Enter same passphrase again: - Your identification has been saved in /home/tidb/.ssh/id_rsa. - Your public key has been saved in /home/tidb/.ssh/id_rsa.pub. - The key fingerprint is: - SHA256:eIBykszR1KyECA/h0d7PRKz4fhAeli7IrVphhte7/So tidb@172.16.10.49 - The key's randomart image is: - +---[RSA 2048]----+ - |=+o+.o. | - |o=o+o.oo | - | .O.=.= | - | . B.B + | - |o B * B S | - | * + * + | - | o + . | - | o E+ . | - |o ..+o. | - +----[SHA256]-----+ - ``` - -## Step 3: Download TiDB Ansible to the control machine - -Log in to the control machine using the `tidb` user account and enter the `/home/tidb` directory. Run the following command to download TiDB Ansible from the master branch of the [TiDB Ansible project](https://github.com/pingcap/tidb-ansible). The default folder name is `tidb-ansible`. - -{{< copyable "shell-regular" >}} - -```shell -git clone https://github.com/pingcap/tidb-ansible.git -``` - -> **Note:** -> -> - To deploy and upgrade TiDB clusters, use the corresponding version of `tidb-ansible`. If you only modify the version in the `inventory.ini` file, errors might occur. -> - It is required to download `tidb-ansible` to the `/home/tidb` directory using the `tidb` user account. If you download it to the `/root` directory, a privilege issue occurs. - -If you have questions regarding which version to use, email to info@pingcap.com for more information or [file an issue](https://github.com/pingcap/tidb-ansible/issues/new). - -## Step 4: Install TiDB Ansible and its dependencies on the control machine - -Make sure you have logged in to the control machine using the `tidb` user account. - -It is required to use `pip` to install Ansible and its dependencies, otherwise a compatibility issue occurs. Currently, the release-2.0, release-2.1, release-3.1, and master branches of TiDB Ansible are compatible with Ansible 2.4 ~ 2.7.11 (2.4 ≤ Ansible ≤ 2.7.11). - -1. Install TiDB Ansible and the dependencies on the control machine: - - {{< copyable "shell-regular" >}} - - ```bash - cd /home/tidb/tidb-ansible && \ - sudo pip install -r ./requirements.txt - ``` - - The version information of Ansible and dependencies is in the `tidb-ansible/requirements.txt` file. - -2. View the version of TiDB Ansible: - - {{< copyable "shell-regular" >}} - - ```bash - ansible --version - ``` - - ``` - ansible 2.7.11 - ``` - -## Step 5: Configure the SSH mutual trust and sudo rules on the control machine - -Make sure you have logged in to the control machine using the `tidb` user account. - -1. Add the IPs of your target machines to the `[servers]` section of the `hosts.ini` file. - - {{< copyable "shell-regular" >}} - - ```bash - cd /home/tidb/tidb-ansible && \ - vi hosts.ini - ``` - - ``` - [servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.4 - 172.16.10.5 - 172.16.10.6 - - [all:vars] - username = tidb - ntp_server = pool.ntp.org - ``` - -2. Run the following command and input the `root` user account password of your target machines. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook -i hosts.ini create_users.yml -u root -k - ``` - - This step creates the `tidb` user account on the target machines, configures the sudo rules and the SSH mutual trust between the control machine and the target machines. - -To configure the SSH mutual trust and sudo without password manually, see [How to manually configure the SSH mutual trust and sudo without password](#how-to-manually-configure-the-ssh-mutual-trust-and-sudo-without-password). - -## Step 6: Install the NTP service on the target machines - -> **Note:** -> -> If the time and time zone of all your target machines are same, the NTP service is on and is normally synchronizing time, you can ignore this step. See [How to check whether the NTP service is normal](#how-to-check-whether-the-ntp-service-is-normal). - -Make sure you have logged in to the control machine using the `tidb` user account, run the following command: - -{{< copyable "shell-regular" >}} - -```bash -cd /home/tidb/tidb-ansible && \ -ansible-playbook -i hosts.ini deploy_ntp.yml -u tidb -b -``` - -The NTP service is installed and started using the software repository that comes with the system on the target machines. The default NTP server list in the installation package is used. The related `server` parameter is in the `/etc/ntp.conf` configuration file. - -To make the NTP service start synchronizing as soon as possible, the system executes the `ntpdate` command to set the local date and time by polling `ntp_server` in the `hosts.ini` file. The default server is `pool.ntp.org`, and you can also replace it with your NTP server. - -## Step 7: Configure the CPUfreq governor mode on the target machine - -For details about CPUfreq, see [the CPUfreq Governor documentation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/power_management_guide/cpufreq_governors). - -Set the CPUfreq governor mode to `performance` to make full use of CPU performance. - -### Check the governor modes supported by the system - -You can run the `cpupower frequency-info --governors` command to check the governor modes which the system supports: - -{{< copyable "shell-root" >}} - -```bash -cpupower frequency-info --governors -``` - -``` -analyzing CPU 0: - available cpufreq governors: performance powersave -``` - -Taking the above code for example, the system supports the `performance` and `powersave` modes. - -> **Note:** -> -> As the following shows, if it returns `Not Available`, it means that the current system does not support CPUfreq configuration and you can skip this step. - -{{< copyable "shell-root" >}} - -```bash -cpupower frequency-info --governors -``` - -``` -analyzing CPU 0: - available cpufreq governors: Not Available -``` - -### Check the current governor mode - -You can run the `cpupower frequency-info --policy` command to check the current CPUfreq governor mode: - -{{< copyable "shell-root" >}} - -```bash -cpupower frequency-info --policy -``` - -``` -analyzing CPU 0: - current policy: frequency should be within 1.20 GHz and 3.20 GHz. - The governor "powersave" may decide which speed to use - within this range. -``` - -As the above code shows, the current mode is `powersave` in this example. - -### Change the governor mode - -You can use either of the following two methods to change the governor mode. In the above example, the current governor mode is `powersave` and the following commands change it to `performance`. - -- Use the `cpupower frequency-set --governor` command to change the current mode: - - {{< copyable "shell-root" >}} - - ```shell - cpupower frequency-set --governor performance - ``` - -- Run the following command to set the mode on the target machine in batches: - - {{< copyable "shell-regular" >}} - - ```shell - ansible -i hosts.ini all -m shell -a "cpupower frequency-set --governor performance" -u tidb -b - ``` - -## Step 8: Mount the data disk ext4 filesystem with options on the target machines - -Log in to the target machines using the `root` user account. - -Format your data disks to the ext4 filesystem and add the `nodelalloc` and `noatime` mount options to the filesystem. It is required to add the `nodelalloc` option, or else the Ansible deployment cannot pass the test. The `noatime` option is optional. - -> **Note:** -> -> If your data disks have been formatted to ext4 and have added the mount options, you can uninstall it by running the `umount /dev/nvme0n1p1` command, follow the steps starting from editing the `/etc/fstab` file, and add the options again to the filesystem. - -Take the `/dev/nvme0n1` data disk as an example: - -1. View the data disk. - - {{< copyable "shell-root" >}} - - ```shell - fdisk -l - ``` - - ``` - Disk /dev/nvme0n1: 1000 GB - ``` - -2. Create the partition table. - - {{< copyable "shell-root" >}} - - ```shell - parted -s -a optimal /dev/nvme0n1 mklabel gpt -- mkpart primary ext4 1 -1 - ``` - - > **Note:** - > - > Use the `lsblk` command to view the device number of the partition: for a nvme disk, the generated device number is usually `nvme0n1p1`; for a regular disk (for example, `/dev/sdb`), the generated device number is usually `sdb1`. - -3. Format the data disk to the ext4 filesystem. - - {{< copyable "shell-root" >}} - - ```shell - mkfs.ext4 /dev/nvme0n1p1 - ``` - -4. View the partition UUID of the data disk. - - In this example, the UUID of `nvme0n1p1` is `c51eb23b-195c-4061-92a9-3fad812cc12f`. - - {{< copyable "shell-root" >}} - - ```shell - lsblk -f - ``` - - ``` - NAME FSTYPE LABEL UUID MOUNTPOINT - sda - ├─sda1 ext4 237b634b-a565-477b-8371-6dff0c41f5ab /boot - ├─sda2 swap f414c5c0-f823-4bb1-8fdf-e531173a72ed - └─sda3 ext4 547909c1-398d-4696-94c6-03e43e317b60 / - sr0 - nvme0n1 - └─nvme0n1p1 ext4 c51eb23b-195c-4061-92a9-3fad812cc12f - ``` - -5. Edit the `/etc/fstab` file and add the mount options. - - {{< copyable "shell-root" >}} - - ```shell - vi /etc/fstab - ``` - - ``` - UUID=c51eb23b-195c-4061-92a9-3fad812cc12f /data1 ext4 defaults,nodelalloc,noatime 0 2 - ``` - -6. Mount the data disk. - - {{< copyable "shell-root" >}} - - ```shell - mkdir /data1 && \ - mount -a - ``` - -7. Check using the following command. - - {{< copyable "shell-root" >}} - - ```shell - mount -t ext4 - ``` - - ``` - /dev/nvme0n1p1 on /data1 type ext4 (rw,noatime,nodelalloc,data=ordered) - ``` - - If the filesystem is ext4 and `nodelalloc` is included in the mount options, you have successfully mount the data disk ext4 filesystem with options on the target machines. - -## Step 9: Edit the `inventory.ini` file to orchestrate the TiDB cluster - -Log in to the control machine using the `tidb` user account, and edit the `tidb-ansible/inventory.ini` file to orchestrate the TiDB cluster. The standard TiDB cluster contains 6 machines: 2 TiDB instances, 3 PD instances, and 3 TiKV instances. - -- Deploy at least 3 instances for TiKV. -- Do not deploy TiKV together with TiDB or PD on the same machine. -- Use the first TiDB machine as the monitoring machine. - -> **Note:** -> -> It is required to use the internal IP address to deploy. If the SSH port of the target machines is not the default 22 port, you need to add the `ansible_port` variable. For example, `TiDB1 ansible_host=172.16.10.1 ansible_port=5555`. - -You can choose one of the following two types of cluster topology according to your scenario: - -- [The cluster topology of a single TiKV instance on each TiKV node](#option-1-use-the-cluster-topology-of-a-single-tikv-instance-on-each-tikv-node) - - In most cases, it is recommended to deploy one TiKV instance on each TiKV node for better performance. However, if the CPU and memory of your TiKV machines are much better than the required in [Hardware and Software Requirements](/hardware-and-software-requirements.md), and you have more than two disks in one node or the capacity of one SSD is larger than 2 TB, you can deploy no more than 2 TiKV instances on a single TiKV node. - -- [The cluster topology of multiple TiKV instances on each TiKV node](#option-2-use-the-cluster-topology-of-multiple-tikv-instances-on-each-tikv-node) - -### Option 1: Use the cluster topology of a single TiKV instance on each TiKV node - -| Name | Host IP | Services | -|:------|:------------|:-----------| -| node1 | 172.16.10.1 | PD1, TiDB1 | -| node2 | 172.16.10.2 | PD2, TiDB2 | -| node3 | 172.16.10.3 | PD3 | -| node4 | 172.16.10.4 | TiKV1 | -| node5 | 172.16.10.5 | TiKV2 | -| node6 | 172.16.10.6 | TiKV3 | - -```ini -[tidb_servers] -172.16.10.1 -172.16.10.2 - -[pd_servers] -172.16.10.1 -172.16.10.2 -172.16.10.3 - -[tikv_servers] -172.16.10.4 -172.16.10.5 -172.16.10.6 - -[monitoring_servers] -172.16.10.1 - -[grafana_servers] -172.16.10.1 - -[monitored_servers] -172.16.10.1 -172.16.10.2 -172.16.10.3 -172.16.10.4 -172.16.10.5 -172.16.10.6 -``` - -### Option 2: Use the cluster topology of multiple TiKV instances on each TiKV node - -Take two TiKV instances on each TiKV node as an example: - -| Name | Host IP | Services | -|:------|:------------|:-----------| -| node1 | 172.16.10.1 | PD1, TiDB1 | -| node2 | 172.16.10.2 | PD2, TiDB2 | -| node3 | 172.16.10.3 | PD3 | -| node4 | 172.16.10.4 | TiKV1-1, TiKV1-2 | -| node5 | 172.16.10.5 | TiKV2-1, TiKV2-2 | -| node6 | 172.16.10.6 | TiKV3-1, TiKV3-2 | - -```ini -[tidb_servers] -172.16.10.1 -172.16.10.2 - -[pd_servers] -172.16.10.1 -172.16.10.2 -172.16.10.3 - -# Note: To use labels in TiKV, you must also configure location_labels for PD at the same time. -[tikv_servers] -TiKV1-1 ansible_host=172.16.10.4 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv1" -TiKV1-2 ansible_host=172.16.10.4 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv1" -TiKV2-1 ansible_host=172.16.10.5 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv2" -TiKV2-2 ansible_host=172.16.10.5 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv2" -TiKV3-1 ansible_host=172.16.10.6 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv3" -TiKV3-2 ansible_host=172.16.10.6 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv3" - -[monitoring_servers] -172.16.10.1 - -[grafana_servers] -172.16.10.1 - -[monitored_servers] -172.16.10.1 -172.16.10.2 -172.16.10.3 -172.16.10.4 -172.16.10.5 -172.16.10.6 - -...... - -# Note: For labels in TiKV to work, you must also configure location_labels for PD when deploying the cluster. -[pd_servers:vars] -location_labels = ["host"] -``` - -**Edit the parameters in the service configuration file:** - -1. For the cluster topology of multiple TiKV instances on each TiKV node, you need to edit the `capacity` parameter under `block-cache-size` in `tidb-ansible/conf/tikv.yml`: - - ```yaml - storage: - block-cache: - capacity: "1GB" - ``` - - > **Note:** - > - > + The number of TiKV instances is the number of TiKV processes on each server. - > + Recommended configuration: `capacity` = MEM_TOTAL \* 0.5 / the number of TiKV instances - -2. For the cluster topology of multiple TiKV instances on each TiKV node, you need to edit the `high-concurrency`, `normal-concurrency` and `low-concurrency` parameters in the `tidb-ansible/conf/tikv.yml` file: - - ```yaml - readpool: - coprocessor: - # Notice: if CPU_NUM > 8, default thread pool size for coprocessors - # will be set to CPU_NUM * 0.8. - # high-concurrency: 8 - # normal-concurrency: 8 - # low-concurrency: 8 - ``` - - > **Note:** - > - > Recommended configuration: the number of TiKV instances \* the parameter value = the number of CPU cores \* 0.8. - -3. If multiple TiKV instances are deployed on a same physical disk, edit the `capacity` parameter in `conf/tikv.yml`: - - ```yaml - raftstore: - capacity: 0 - ``` - - > **Note:** - > - > Recommended configuration: `capacity` = total disk capacity / the number of TiKV instances. For example, `capacity: "100GB"`. - -## Step 10: Edit variables in the `inventory.ini` file - -This step describes how to edit the variable of deployment directory and other variables in the `inventory.ini` file. - -### Configure the deployment directory - -Edit the `deploy_dir` variable to configure the deployment directory. - -The global variable is set to `/home/tidb/deploy` by default, and it applies to all services. If the data disk is mounted on the `/data1` directory, you can set it to `/data1/deploy`. For example: - -```ini -## Global variables -[all:vars] -deploy_dir = /data1/deploy -``` - -> **Note:** -> -> To separately set the deployment directory for a service, you can configure the host variable while configuring the service host list in the `inventory.ini` file. It is required to add the first column alias, to avoid confusion in scenarios of mixed services deployment. - -```ini -TiKV1-1 ansible_host=172.16.10.4 deploy_dir=/data1/deploy -``` - -### Edit other variables (Optional) - -To enable the following control variables, use the capitalized `True`. To disable the following control variables, use the capitalized `False`. - -| Variable Name | Description | -| :---- | :------- | -| cluster_name | the name of a cluster, adjustable | -| cpu_architecture | CPU architecture. `amd64` by default, `arm64` optional | -| tidb_version | the version of TiDB, configured by default in TiDB Ansible branches | -| process_supervision | the supervision way of processes, `systemd` by default, `supervise` optional | -| timezone | the global default time zone configured when a new TiDB cluster bootstrap is initialized; you can edit it later using the global `time_zone` system variable and the session `time_zone` system variable as described in [Time Zone Support](/configure-time-zone.md); the default value is `Asia/Shanghai` and see [the list of time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for more optional values | -| enable_firewalld | to enable the firewall, closed by default; to enable it, add the ports in [network requirements](/hardware-and-software-requirements.md#network-requirements) to the allowlist | -| enable_ntpd | to monitor the NTP service of the managed node, True by default; do not close it | -| set_hostname | to edit the hostname of the managed node based on the IP, False by default | -| enable_binlog | whether to deploy Pump and enable the binlog, False by default, dependent on the Kafka cluster; see the `zookeeper_addrs` variable | -| zookeeper_addrs | the zookeeper address of the binlog Kafka cluster | -| deploy_without_tidb | the Key-Value mode, deploy only PD, TiKV and the monitoring service, not TiDB; set the IP of the tidb_servers host group to null in the `inventory.ini` file | -| alertmanager_target | optional: If you have deployed `alertmanager` separately, you can configure this variable using the `alertmanager_host:alertmanager_port` format | -| grafana_admin_user | the username of Grafana administrator; default `admin` | -| grafana_admin_password | the password of Grafana administrator account; default `admin`; used to import Dashboard and create the API key using TiDB Ansible; update this variable if you have modified it through Grafana web | -| collect_log_recent_hours | to collect the log of recent hours; default the recent 2 hours | -| enable_bandwidth_limit | to set a bandwidth limit when pulling the diagnostic data from the target machines to the control machine; used together with the `collect_bandwidth_limit` variable | -| collect_bandwidth_limit | the limited bandwidth when pulling the diagnostic data from the target machines to the control machine; unit: Kbit/s; default 10000, indicating 10Mb/s; for the cluster topology of multiple TiKV instances on each TiKV node, you need to divide the number of the TiKV instances on each TiKV node | -| prometheus_storage_retention | the retention time of the monitoring data of Prometheus (30 days by default); this is a new configuration in the `group_vars/monitoring_servers.yml` file in 2.1.7, 3.0 and the later tidb-ansible versions | - -## Step 11: Deploy the TiDB cluster - -When `ansible-playbook` runs Playbook, the default concurrent number is 5. If many target machines are deployed, you can add the `-f` parameter to specify the concurrency, such as `ansible-playbook deploy.yml -f 10`. - -The following example uses `tidb` as the user who runs the service. - -1. Edit the `tidb-ansible/inventory.ini` file to make sure `ansible_user = tidb`. - - ```ini - ## Connection - # ssh via normal user - ansible_user = tidb - ``` - - > **Note:** - > - > Do not configure `ansible_user` to `root`, because `tidb-ansible` limits the user that runs the service to the normal user. - - Run the following command and if all servers return `tidb`, then the SSH mutual trust is successfully configured: - - {{< copyable "shell-regular" >}} - - ```shell - ansible -i inventory.ini all -m shell -a 'whoami' - ``` - - Run the following command and if all servers return `root`, then sudo without password of the `tidb` user is successfully configured: - - {{< copyable "shell-regular" >}} - - ```shell - ansible -i inventory.ini all -m shell -a 'whoami' -b - ``` - -2. Run the `local_prepare.yml` playbook and download TiDB binary to the control machine. - - {{< copyable "shell-regular" >}} - - ```shell - ansible-playbook local_prepare.yml - ``` - -3. Initialize the system environment and modify the kernel parameters. - - {{< copyable "shell-regular" >}} - - ```shell - ansible-playbook bootstrap.yml - ``` - -4. Deploy the TiDB cluster software. - - {{< copyable "shell-regular" >}} - - ```shell - ansible-playbook deploy.yml - ``` - - > **Note:** - > - > You can use the `Report` button on the Grafana Dashboard to generate the PDF file. This function depends on the `fontconfig` package and English fonts. To use this function, log in to the `grafana_servers` machine and install it using the following command: - > - > {{< copyable "shell-regular" >}} - > - > ```bash - > sudo yum install fontconfig open-sans-fonts - > ``` - -5. Start the TiDB cluster. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook start.yml - ``` - -## Test the TiDB cluster - -Because TiDB is compatible with MySQL, you must use the MySQL client to connect to TiDB directly. It is recommended to configure load balancing to provide uniform SQL interface. - -1. Connect to the TiDB cluster using the MySQL client. - - {{< copyable "shell-regular" >}} - - ```shell - mysql -u root -h 172.16.10.1 -P 4000 - ``` - - > **Note:** - > - > The default port of TiDB service is `4000`. - -2. Access the monitoring platform using a web browser. - - - Address: - - Default account and password: `admin`; `admin` - -> **Note:** -> -> By default, TiDB periodically shares usage details with PingCAP to help understand how to improve the product. For details about what is shared and how to disable the sharing, see [Telemetry](/telemetry.md). - -## Deployment FAQs - -This section lists the common questions about deploying TiDB using TiDB Ansible. - -### How to customize the port? - -Edit the `inventory.ini` file and add the following host variable after the IP of the corresponding service: - -| Component | Variable Port | Default Port | Description | -|:--------------|:-------------------|:-------------|:-------------------------| -| TiDB | tidb_port | 4000 | the communication port for the application and DBA tools | -| TiDB | tidb_status_port | 10080 | the communication port to report TiDB status | -| TiKV | tikv_port | 20160 | the TiKV communication port | -| TiKV | tikv_status_port | 20180 | the communication port to report the TiKV status | -| PD | pd_client_port | 2379 | the communication port between TiDB and PD | -| PD | pd_peer_port | 2380 | the inter-node communication port within the PD cluster | -| Pump | pump_port | 8250 | the pump communication port | -| Prometheus | prometheus_port | 9090 | the communication port for the Prometheus service | -| Pushgateway | pushgateway_port | 9091 | the aggregation and report port for TiDB, TiKV, and PD monitor | -| Node_exporter | node_exporter_port | 9100 | the communication port to report the system information of every TiDB cluster node | -| Grafana | grafana_port | 3000 | the port for the external Web monitoring service and client (Browser) access | -| Kafka_exporter | kafka_exporter_port | 9308 | the communication port for Kafka_exporter, used to monitor the binlog Kafka cluster | - -### How to customize the deployment directory? - -Edit the `inventory.ini` file and add the following host variable after the IP of the corresponding service: - -| Component | Variable Directory | Default Directory | Description | -|:--------------|:----------------------|:------------------------------|:-----| -| Global | deploy_dir | /home/tidb/deploy | the deployment directory | -| TiDB | tidb_log_dir | {{ deploy_dir }}/log | the TiDB log directory | -| TiKV | tikv_log_dir | {{ deploy_dir }}/log | the TiKV log directory | -| TiKV | tikv_data_dir | {{ deploy_dir }}/data | the data directory | -| TiKV | wal_dir | "" | the rocksdb write-ahead log directory, consistent with the TiKV data directory when the value is null | -| TiKV | raftdb_path | "" | the raftdb directory, being tikv_data_dir/raft when the value is null | -| PD | pd_log_dir | {{ deploy_dir }}/log | the PD log directory | -| PD | pd_data_dir | {{ deploy_dir }}/data.pd | the PD data directory | -| Pump | pump_log_dir | {{ deploy_dir }}/log | the Pump log directory | -| Pump | pump_data_dir | {{ deploy_dir }}/data.pump | the Pump data directory | -| Prometheus | prometheus_log_dir | {{ deploy_dir }}/log | the Prometheus log directory | -| Prometheus | prometheus_data_dir | {{ deploy_dir }}/data.metrics | the Prometheus data directory | -| Pushgateway | pushgateway_log_dir | {{ deploy_dir }}/log | the pushgateway log directory | -| Node_exporter | node_exporter_log_dir | {{ deploy_dir }}/log | the node_exporter log directory | -| Grafana | grafana_log_dir | {{ deploy_dir }}/log | the Grafana log directory | -| Grafana | grafana_data_dir | {{ deploy_dir }}/data.grafana | the Grafana data directory | - -### How to check whether the NTP service is normal? - -1. Run the following command. If it returns `running`, then the NTP service is running: - - {{< copyable "shell-regular" >}} - - ```shell - sudo systemctl status ntpd.service - ``` - - ``` - ntpd.service - Network Time Service - Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled) - Active: active (running) since 一 2017-12-18 13:13:19 CST; 3s ago - ``` - -2. Run the ntpstat command. If it returns `synchronised to NTP server` (synchronizing with the NTP server), then the synchronization process is normal. - - {{< copyable "shell-regular" >}} - - ```shell - ntpstat - ``` - - ``` - synchronised to NTP server (85.199.214.101) at stratum 2 - time correct to within 91 ms - polling server every 1024 s - ``` - -> **Note:** -> -> For the Ubuntu system, you need to install the `ntpstat` package. - -- The following condition indicates the NTP service is not synchronizing normally: - - {{< copyable "shell-regular" >}} - - ```shell - ntpstat - ``` - - ``` - unsynchronised - ``` - -- The following condition indicates the NTP service is not running normally: - - {{< copyable "shell-regular" >}} - - ```shell - ntpstat - ``` - - ``` - Unable to talk to NTP daemon. Is it running? - ``` - -- To make the NTP service start synchronizing as soon as possible, run the following command. You can replace `pool.ntp.org` with other NTP servers. - - {{< copyable "shell-regular" >}} - - ```shell - sudo systemctl stop ntpd.service && \ - sudo ntpdate pool.ntp.org && \ - sudo systemctl start ntpd.service - ``` - -- To install the NTP service manually on the CentOS 7 system, run the following command: - - {{< copyable "shell-regular" >}} - - ```shell - sudo yum install ntp ntpdate && \ - sudo systemctl start ntpd.service && \ - sudo systemctl enable ntpd.service - ``` - -### How to modify the supervision method of a process from `supervise` to `systemd`? - -Run the following command: - -{{< copyable "shell-root" >}} - -```shell -process supervision, [systemd, supervise] -``` - -``` -process_supervision = systemd -``` - -For versions earlier than TiDB 1.0.4, the TiDB Ansible supervision method of a process is `supervise` by default. The previously installed cluster can remain the same. If you need to change the supervision method to `systemd`, stop the cluster and run the following command: - -{{< copyable "shell-regular" >}} - -```shell -ansible-playbook stop.yml && \ -ansible-playbook deploy.yml -D && \ -ansible-playbook start.yml -``` - -### How to manually configure the SSH mutual trust and sudo without password? - -1. Log in to the target machine respectively using the `root` user account, create the `tidb` user and set the login password. - - {{< copyable "shell-root" >}} - - ```shell - useradd tidb && \ - passwd tidb - ``` - -2. To configure sudo without password, run the following command, and add `tidb ALL=(ALL) NOPASSWD: ALL` to the end of the file: - - {{< copyable "shell-root" >}} - - ```shell - visudo - ``` - - ``` - tidb ALL=(ALL) NOPASSWD: ALL - ``` - -3. Use the `tidb` user to log in to the control machine, and run the following command. Replace `172.16.10.61` with the IP of your target machine, and enter the `tidb` user password of the target machine as prompted. Successful execution indicates that SSH mutual trust is already created. This applies to other machines as well. - - {{< copyable "shell-regular" >}} - - ```shell - ssh-copy-id -i ~/.ssh/id_rsa.pub 172.16.10.61 - ``` - -4. Log in to the control machine using the `tidb` user account, and log in to the IP of the target machine using SSH. If you do not need to enter the password and can successfully log in, then the SSH mutual trust is successfully configured. - - {{< copyable "shell-regular" >}} - - ```shell - ssh 172.16.10.61 - ``` - - ``` - [tidb@172.16.10.61 ~]$ - ``` - -5. After you login to the target machine using the `tidb` user, run the following command. If you do not need to enter the password and can switch to the `root` user, then sudo without password of the `tidb` user is successfully configured. - - {{< copyable "shell-regular" >}} - - ```shell - sudo -su root - ``` - - ``` - [root@172.16.10.61 tidb]# - ``` - -### Error: You need to install jmespath prior to running json_query filter - -1. See [Install TiDB Ansible and its dependencies on the control machine](#step-4-install-tidb-ansible-and-its-dependencies-on-the-control-machine) and use `pip` to install TiDB Ansible and the corresponding dependencies in the control machine. The `jmespath` dependent package is installed by default. - -2. Run the following command to check whether `jmespath` is successfully installed: - - {{< copyable "shell-regular" >}} - - ```bash - pip show jmespath - ``` - -3. Enter `import jmespath` in the Python interactive window of the control machine. - - - If no error displays, the dependency is successfully installed. - - If the `ImportError: No module named jmespath` error displays, the Python `jmespath` module is not successfully installed. - - {{< copyable "shell-regular" >}} - - ```shell - python - ``` - - ``` - Python 2.7.5 (default, Nov 6 2016, 00:28:07) - [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2 - Type "help", "copyright", "credits" or "license" for more information. - ``` - - {{< copyable "shell-regular" >}} - - ```shell - import jmespath - ``` - -### The `zk: node does not exist` error when starting Pump/Drainer - -Check whether the `zookeeper_addrs` configuration in `inventory.ini` is the same with the configuration in the Kafka cluster, and whether the namespace is filled in. The description about namespace configuration is as follows: - -```ini -# ZooKeeper connection string (see ZooKeeper docs for details). -# ZooKeeper address of the Kafka cluster. Example: -# zookeeper_addrs = "192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181" -# You can also append an optional chroot string to the URLs to specify the root directory for all Kafka znodes. Example: -# zookeeper_addrs = "192.168.0.11:2181,192.168.0.12:2181,192.168.0.13:2181/kafka/123" -``` diff --git a/production-deployment-using-tiup.md b/production-deployment-using-tiup.md index 0f4304375c87c..61f6f9e393ca5 100644 --- a/production-deployment-using-tiup.md +++ b/production-deployment-using-tiup.md @@ -1,7 +1,7 @@ --- title: Deploy a TiDB Cluster Using TiUP summary: Learn how to easily deploy a TiDB cluster using TiUP. -aliases: ['/docs/dev/production-deployment-using-tiup/','/docs/dev/how-to/deploy/orchestrated/tiup/','/docs/dev/tiflash/deploy-tiflash/','/docs/dev/reference/tiflash/deploy/','/tidb/dev/deploy-tidb-from-dbdeployer/','/docs/dev/deploy-tidb-from-dbdeployer/','/docs/dev/how-to/get-started/deploy-tidb-from-dbdeployer/','/tidb/dev/deploy-tidb-from-homebrew/','/docs/dev/deploy-tidb-from-homebrew/','/docs/dev/how-to/get-started/deploy-tidb-from-homebrew/'] +aliases: ['/docs/dev/production-deployment-using-tiup/','/docs/dev/how-to/deploy/orchestrated/tiup/','/docs/dev/tiflash/deploy-tiflash/','/docs/dev/reference/tiflash/deploy/','/tidb/dev/deploy-tidb-from-dbdeployer/','/docs/dev/deploy-tidb-from-dbdeployer/','/docs/dev/how-to/get-started/deploy-tidb-from-dbdeployer/','/tidb/dev/deploy-tidb-from-homebrew/','/docs/dev/deploy-tidb-from-homebrew/','/docs/dev/how-to/get-started/deploy-tidb-from-homebrew/','/docs/dev/online-deployment-using-ansible/','/docs/dev/how-to/deploy/orchestrated/ansible/','/tidb/dev/online-deployment-using-ansible/'] --- # Deploy a TiDB Cluster Using TiUP @@ -10,6 +10,10 @@ aliases: ['/docs/dev/production-deployment-using-tiup/','/docs/dev/how-to/deploy TiUP supports deploying TiDB, TiFlash, TiDB Binlog, TiCDC, and the monitoring system. This document introduces how to deploy TiDB clusters of different topologies. +> **Note:** +> +> Since TiDB v4.0, PingCAP no longer provides support for TiDB Ansible. Since TiDB v5.0, PingCAP no longer provides TiDB Ansible documents. If you want to read the document that introduces how to deploy a TiDB cluster using TiDB Ansible, see [Deploy TiDB Using TiDB Ansible](https://docs.pingcap.com/tidb/v4.0/online-deployment-using-ansible). + ## Step 1: Prerequisites and precheck Make sure that you have read the following documents: diff --git a/production-offline-deployment-using-tiup.md b/production-offline-deployment-using-tiup.md index 1cedf6f56e4a5..74ae6283587d8 100644 --- a/production-offline-deployment-using-tiup.md +++ b/production-offline-deployment-using-tiup.md @@ -1,13 +1,17 @@ --- title: Deploy a TiDB Cluster Offline Using TiUP summary: Introduce how to deploy a TiDB cluster offline using TiUP. -aliases: ['/docs/dev/production-offline-deployment-using-tiup/'] +aliases: ['/docs/dev/production-offline-deployment-using-tiup/','/docs/dev/offline-deployment-using-ansible/','/docs/dev/how-to/deploy/orchestrated/offline-ansible/','/tidb/dev/offline-deployment-using-ansible/'] --- # Deploy a TiDB Cluster Offline Using TiUP This document describes how to deploy a TiDB cluster offline using TiUP. +> **Note:** +> +> Since TiDB v4.0, PingCAP no longer provides support for TiDB Ansible. Since TiDB v5.0, PingCAP no longer provides TiDB Ansible documents. If you want to read the document that introduces how to deploy a TiDB cluster using TiDB Ansible offline, see [Deploy TiDB Offline Using TiDB Ansible](https://docs.pingcap.com/tidb/v4.0/offline-deployment-using-ansible). + ## Step 1: Prepare the TiUP offline component package ### Option 1: Download the official TiUP offline component package diff --git a/scale-tidb-using-ansible.md b/scale-tidb-using-ansible.md deleted file mode 100644 index 0c4bcc8231b9a..0000000000000 --- a/scale-tidb-using-ansible.md +++ /dev/null @@ -1,556 +0,0 @@ ---- -title: Scale the TiDB Cluster Using TiDB Ansible -summary: Use TiDB Ansible to increase/decrease the capacity of a TiDB/TiKV/PD node. -aliases: ['/docs/dev/scale-tidb-using-ansible/','/docs/dev/how-to/scale/with-ansible/'] ---- - -# Scale the TiDB Cluster Using TiDB Ansible - -The capacity of a TiDB cluster can be increased or decreased without affecting the online services. - -> **Warning:** -> -> In decreasing the capacity, if your cluster has a mixed deployment of other services, do not perform the following procedures. The following examples assume that the removed nodes have no mixed deployment of other services. - -Assume that the topology is as follows: - -| Name | Host IP | Services | -| ---- | ------- | -------- | -| node1 | 172.16.10.1 | PD1 | -| node2 | 172.16.10.2 | PD2 | -| node3 | 172.16.10.3 | PD3, Monitor | -| node4 | 172.16.10.4 | TiDB1 | -| node5 | 172.16.10.5 | TiDB2 | -| node6 | 172.16.10.6 | TiKV1 | -| node7 | 172.16.10.7 | TiKV2 | -| node8 | 172.16.10.8 | TiKV3 | -| node9 | 172.16.10.9 | TiKV4 | - -## Increase the capacity of a TiDB/TiKV node - -For example, if you want to add two TiDB nodes (node101, node102) with the IP addresses `172.16.10.101` and `172.16.10.102`, take the following steps: - -1. Edit the `inventory.ini` file and the `hosts.ini` file, and append the node information. - - - Edit the `inventory.ini` file: - - ```ini - [tidb_servers] - 172.16.10.4 - 172.16.10.5 - 172.16.10.101 - 172.16.10.102 - - [pd_servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - - [tikv_servers] - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitored_servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.4 - 172.16.10.5 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - 172.16.10.101 - 172.16.10.102 - - [monitoring_servers] - 172.16.10.3 - - [grafana_servers] - 172.16.10.3 - ``` - - Now the topology is as follows: - - | Name | Host IP | Services | - | ---- | ------- | -------- | - | node1 | 172.16.10.1 | PD1 | - | node2 | 172.16.10.2 | PD2 | - | node3 | 172.16.10.3 | PD3, Monitor | - | node4 | 172.16.10.4 | TiDB1 | - | node5 | 172.16.10.5 | TiDB2 | - | **node101** | **172.16.10.101**|**TiDB3** | - | **node102** | **172.16.10.102**|**TiDB4** | - | node6 | 172.16.10.6 | TiKV1 | - | node7 | 172.16.10.7 | TiKV2 | - | node8 | 172.16.10.8 | TiKV3 | - | node9 | 172.16.10.9 | TiKV4 | - - - Edit the `hosts.ini` file: - - ```ini - [servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.4 - 172.16.10.5 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - 172.16.10.101 - 172.16.10.102 - [all:vars] - username = tidb - ntp_server = pool.ntp.org - ``` - -2. Initialize the newly added node. - - 1. Configure the SSH mutual trust and sudo rules of the target machine on the control machine: - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook -i hosts.ini create_users.yml -l 172.16.10.101,172.16.10.102 -u root -k - ``` - - 2. Install the NTP service on the target machine: - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook -i hosts.ini deploy_ntp.yml -u tidb -b - ``` - - 3. Initialize the node on the target machine: - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook bootstrap.yml -l 172.16.10.101,172.16.10.102 - ``` - - > **Note:** - > - > If an alias is configured in the `inventory.ini` file, for example, `node101 ansible_host=172.16.10.101`, use `-l` to specify the alias when executing `ansible-playbook`. For example, `ansible-playbook bootstrap.yml -l node101,node102`. This also applies to the following steps. - -3. Deploy the newly added node: - - ``` - ansible-playbook deploy.yml -l 172.16.10.101,172.16.10.102 - ``` - -4. Start the newly added node: - - ``` - ansible-playbook start.yml -l 172.16.10.101,172.16.10.102 - ``` - -5. Update the Prometheus configuration and restart the cluster: - - ``` - ansible-playbook rolling_update_monitor.yml --tags=prometheus - ``` - -6. Monitor the status of the entire cluster and the newly added node by opening a browser to access the monitoring platform: `http://172.16.10.3:3000`. - -You can use the same procedure to add a TiKV node. But to add a PD node, some configuration files need to be manually updated. - -## Increase the capacity of a PD node - -For example, if you want to add a PD node (node103) with the IP address `172.16.10.103`, take the following steps: - -1. Edit the `inventory.ini` file and append the node information to the end of the `[pd_servers]` group: - - ```ini - [tidb_servers] - 172.16.10.4 - 172.16.10.5 - - [pd_servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.103 - - [tikv_servers] - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitored_servers] - 172.16.10.4 - 172.16.10.5 - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.103 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitoring_servers] - 172.16.10.3 - - [grafana_servers] - 172.16.10.3 - ``` - - Now the topology is as follows: - - | Name | Host IP | Services | - | ---- | ------- | -------- | - | node1 | 172.16.10.1 | PD1 | - | node2 | 172.16.10.2 | PD2 | - | node3 | 172.16.10.3 | PD3, Monitor | - | **node103** | **172.16.10.103** | **PD4** | - | node4 | 172.16.10.4 | TiDB1 | - | node5 | 172.16.10.5 | TiDB2 | - | node6 | 172.16.10.6 | TiKV1 | - | node7 | 172.16.10.7 | TiKV2 | - | node8 | 172.16.10.8 | TiKV3 | - | node9 | 172.16.10.9 | TiKV4 | - -2. Initialize the newly added node: - - ``` - ansible-playbook bootstrap.yml -l 172.16.10.103 - ``` - -3. Deploy the newly added node: - - ``` - ansible-playbook deploy.yml -l 172.16.10.103 - ``` - -4. Login the newly added PD node and edit the starting script: - - ``` - {deploy_dir}/scripts/run_pd.sh - ``` - - 1. Remove the `--initial-cluster="xxxx" \` configuration. - - > **Note:** - > - > You cannot add the `#` character at the beginning of the line. Otherwise, the following configuration cannot take effect. - - 2. Add `--join="http://172.16.10.1:2379" \`. The IP address (`172.16.10.1`) can be any of the existing PD IP address in the cluster. - - 3. Start the PD service in the newly added PD node: - - ``` - {deploy_dir}/scripts/start_pd.sh - ``` - - > **Note:** - > - > Before start, you need to ensure that the `health` status of the newly added PD node is "true", using [PD Control](/pd-control.md). Otherwise, the PD service might fail to start and an error message `["join meet error"] [error="etcdserver: unhealthy cluster"]` is returned in the log. - - 4. Use `pd-ctl` to check whether the new node is added successfully: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" - ``` - - > **Note:** - > - > `pd-ctl` is a command used to check the number of PD nodes. - -5. Start the monitoring service: - - ``` - ansible-playbook start.yml -l 172.16.10.103 - ``` - - > **Note:** - > - > If you use an alias (inventory_name), use the `-l` option to specify the alias. - -6. Update the cluster configuration: - - ``` - ansible-playbook deploy.yml - ``` - -7. Restart Prometheus, and enable the monitoring of PD nodes used for increasing the capacity: - - ``` - ansible-playbook stop.yml --tags=prometheus - ansible-playbook start.yml --tags=prometheus - ``` - -8. Monitor the status of the entire cluster and the newly added node by opening a browser to access the monitoring platform: `http://172.16.10.3:3000`. - -> **Note:** -> -> The PD Client in TiKV caches the list of PD nodes. Currently, the list is updated only if the PD leader is switched or the TiKV server is restarted to load the latest configuration. To avoid TiKV caching an outdated list, there should be at least two existing PD members in the PD cluster after increasing or decreasing the capacity of a PD node. If this condition is not met, transfer the PD leader manually to update the list of PD nodes. - -## Decrease the capacity of a TiDB node - -For example, if you want to remove a TiDB node (node5) with the IP address `172.16.10.5`, take the following steps: - -1. Stop all services on node5: - - ``` - ansible-playbook stop.yml -l 172.16.10.5 - ``` - -2. Edit the `inventory.ini` file and remove the node information: - - ```ini - [tidb_servers] - 172.16.10.4 - #172.16.10.5 # the removed node - - [pd_servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - - [tikv_servers] - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitored_servers] - 172.16.10.4 - #172.16.10.5 # the removed node - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitoring_servers] - 172.16.10.3 - - [grafana_servers] - 172.16.10.3 - ``` - - Now the topology is as follows: - - | Name | Host IP | Services | - | ---- | ------- | -------- | - | node1 | 172.16.10.1 | PD1 | - | node2 | 172.16.10.2 | PD2 | - | node3 | 172.16.10.3 | PD3, Monitor | - | node4 | 172.16.10.4 | TiDB1 | - | **node5** | **172.16.10.5** | **TiDB2 removed** | - | node6 | 172.16.10.6 | TiKV1 | - | node7 | 172.16.10.7 | TiKV2 | - | node8 | 172.16.10.8 | TiKV3 | - | node9 | 172.16.10.9 | TiKV4 | - -3. Update the Prometheus configuration and restart the cluster: - - ``` - ansible-playbook rolling_update_monitor.yml --tags=prometheus - ``` - -4. Monitor the status of the entire cluster by opening a browser to access the monitoring platform: `http://172.16.10.3:3000`. - -## Decrease the capacity of a TiKV node - -For example, if you want to remove a TiKV node (node9) with the IP address `172.16.10.9`, take the following steps: - -1. Remove the node from the cluster using `pd-ctl`: - - 1. View the store ID of node9: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d store - ``` - - 2. Remove node9 from the cluster, assuming that the store ID is 10: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d store delete 10 - ``` - -2. Use `pd-ctl` to check whether the node is successfully removed: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d store 10 - ``` - - > **Note:** - > - > It takes some time to remove the node. If the status of the node you remove becomes Tombstone, then this node is successfully removed. - -3. After the node is successfully removed, stop the services on node9: - - ``` - ansible-playbook stop.yml -l 172.16.10.9 - ``` - -4. Edit the `inventory.ini` file and remove the node information: - - ```ini - [tidb_servers] - 172.16.10.4 - 172.16.10.5 - - [pd_servers] - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - - [tikv_servers] - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - #172.16.10.9 # the removed node - - [monitored_servers] - 172.16.10.4 - 172.16.10.5 - 172.16.10.1 - 172.16.10.2 - 172.16.10.3 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - #172.16.10.9 # the removed node - - [monitoring_servers] - 172.16.10.3 - - [grafana_servers] - 172.16.10.3 - ``` - - Now the topology is as follows: - - | Name | Host IP | Services | - | ---- | ------- | -------- | - | node1 | 172.16.10.1 | PD1 | - | node2 | 172.16.10.2 | PD2 | - | node3 | 172.16.10.3 | PD3, Monitor | - | node4 | 172.16.10.4 | TiDB1 | - | node5 | 172.16.10.5 | TiDB2 | - | node6 | 172.16.10.6 | TiKV1 | - | node7 | 172.16.10.7 | TiKV2 | - | node8 | 172.16.10.8 | TiKV3 | - | **node9** | **172.16.10.9** | **TiKV4 removed** | - -5. Update the Prometheus configuration and restart the cluster: - - ``` - ansible-playbook rolling_update_monitor.yml --tags=prometheus - ``` - -6. Monitor the status of the entire cluster by opening a browser to access the monitoring platform: `http://172.16.10.3:3000`. - -## Decrease the capacity of a PD node - -For example, if you want to remove a PD node (node2) with the IP address `172.16.10.2`, take the following steps: - -1. Remove the node from the cluster using `pd-ctl`: - - 1. View the name of node2: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d member - ``` - - 2. Remove node2 from the cluster, assuming that the name is pd2: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d member delete name pd2 - ``` - -2. Use Grafana or `pd-ctl` to check whether the node is successfully removed: - - ``` - ./pd-ctl -u "http://172.16.10.1:2379" -d member - ``` - -3. After the node is successfully removed, stop the services on node2: - - ``` - ansible-playbook stop.yml -l 172.16.10.2 - ``` - - > **Note:** - > - > In this example, you can only stop the PD service on node2. If there are any other services deployed with the IP address `172.16.10.2`, use the `-t` option to specify the service (such as `-t tidb`). - -4. Edit the `inventory.ini` file and remove the node information: - - ```ini - [tidb_servers] - 172.16.10.4 - 172.16.10.5 - - [pd_servers] - 172.16.10.1 - #172.16.10.2 # the removed node - 172.16.10.3 - - [tikv_servers] - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitored_servers] - 172.16.10.4 - 172.16.10.5 - 172.16.10.1 - #172.16.10.2 # the removed node - 172.16.10.3 - 172.16.10.6 - 172.16.10.7 - 172.16.10.8 - 172.16.10.9 - - [monitoring_servers] - 172.16.10.3 - - [grafana_servers] - 172.16.10.3 - ``` - - Now the topology is as follows: - - | Name | Host IP | Services | - | ---- | ------- | -------- | - | node1 | 172.16.10.1 | PD1 | - | **node2** | **172.16.10.2** | **PD2 removed** | - | node3 | 172.16.10.3 | PD3, Monitor | - | node4 | 172.16.10.4 | TiDB1 | - | node5 | 172.16.10.5 | TiDB2 | - | node6 | 172.16.10.6 | TiKV1 | - | node7 | 172.16.10.7 | TiKV2 | - | node8 | 172.16.10.8 | TiKV3 | - | node9 | 172.16.10.9 | TiKV4 | - -5. Update the cluster configuration: - - ``` - ansible-playbook deploy.yml - ``` - -6. Restart Prometheus, and disable the monitoring of PD nodes used for increasing the capacity: - - ``` - ansible-playbook stop.yml --tags=prometheus - ansible-playbook start.yml --tags=prometheus - ``` - -7. To monitor the status of the entire cluster, open a browser to access the monitoring platform: `http://172.16.10.3:3000`. - -> **Note:** -> -> The PD Client in TiKV caches the list of PD nodes. Currently, the list is updated only if the PD leader is switched or the TiKV server is restarted to load the latest configuration. To avoid TiKV caching an outdated list, there should be at least two existing PD members in the PD cluster after increasing or decreasing the capacity of a PD node. If this condition is not met, transfer the PD leader manually to update the list of PD nodes. diff --git a/scale-tidb-using-tiup.md b/scale-tidb-using-tiup.md index ab498d7a71f41..3c622f4c4e164 100644 --- a/scale-tidb-using-tiup.md +++ b/scale-tidb-using-tiup.md @@ -1,7 +1,7 @@ --- title: Scale the TiDB Cluster Using TiUP summary: Learn how to scale the TiDB cluster using TiUP. -aliases: ['/docs/dev/scale-tidb-using-tiup/','/docs/dev/how-to/scale/with-tiup/'] +aliases: ['/docs/dev/scale-tidb-using-tiup/','/docs/dev/how-to/scale/with-tiup/','/docs/dev/scale-tidb-using-ansible/','/docs/dev/how-to/scale/with-ansible/','/tidb/dev/scale-tidb-using-ansible/'] --- # Scale the TiDB Cluster Using TiUP @@ -10,6 +10,10 @@ The capacity of a TiDB cluster can be increased or decreased without interruptin This document describes how to scale the TiDB, TiKV, PD, TiCDC, or TiFlash cluster using TiUP. If you have not installed TiUP, refer to the steps in [Install TiUP on the control machine](/upgrade-tidb-using-tiup.md#install-tiup-on-the-control-machine) and import the cluster into TiUP before you use TiUP to scale the TiDB cluster. +> **Note:** +> +> Since TiDB v4.0, PingCAP no longer provides support for TiDB Ansible. Since TiDB v5.0, PingCAP no longer provides TiDB Ansible documents. If you want to read the document that introduces how to scale a TiDB cluster using TiDB Ansible, see [Scale the TiDB Cluster Using TiDB Ansible](https://docs.pingcap.com/tidb/v4.0/scale-tidb-using-ansible). + To view the current cluster name list, run `tiup cluster list`. For example, if the original topology of the cluster is as follows: diff --git a/schedule-replicas-by-topology-labels.md b/schedule-replicas-by-topology-labels.md index 546bab38743aa..b279b7f5000fd 100644 --- a/schedule-replicas-by-topology-labels.md +++ b/schedule-replicas-by-topology-labels.md @@ -148,33 +148,6 @@ tikv_servers: For details, see [Geo-distributed Deployment topology](/geo-distributed-deployment-topology.md). -
- Configure a cluster using TiDB Ansible - -When using TiDB Ansible to deploy a cluster, you can directly configure the TiKV location in the `inventory.ini` file. `tidb-ansible` will generate the corresponding TiKV and PD configuration files during deployment. - -In the following example, a two-layer topology of `zone/host` is defined. The TiKV nodes of the cluster are distributed among three zones, each zone with two hosts. In z1, two TiKV instances are deployed per host. In z2 and z3, one TiKV instance is deployed per host. - -``` -[tikv_servers] -# z1 -tikv-1 labels="zone=z1,host=h1" -tikv-2 labels="zone=z1,host=h1" -tikv-3 labels="zone=z1,host=h2" -tikv-4 labels="zone=z1,host=h2" -# z2 -tikv-5 labels="zone=z2,host=h1" -tikv-6 labels="zone=z2,host=h2" -# z3 -tikv-7 labels="zone=z3,host=h1" -tikv-8 labels="zone=z3,host=h2" - -[pd_servers:vars] -location_labels = ["zone", "host"] -``` - -
- ## PD schedules based on topology label PD schedules replicas according to the label layer to make sure that different replicas of the same data are scattered as much as possible. @@ -185,7 +158,7 @@ Assume that the number of cluster replicas is 3 (`max-replicas=3`). Because ther Then, assume that the number of cluster replicas is 5 (`max-replicas=5`). Because there are only 3 zones in total, PD cannot guarantee the isolation of each replica at the zone level. In this situation, the PD scheduler will ensure replica isolation at the host level. In other words, multiple replicas of a Region might be distributed in the same zone but not on the same host. -In the case of the 5-replica configuration, if z3 fails or is isolated as a whole, and cannot be recovered after a period of time (controlled by `max-store-down-time`), PD will make up the 5 replicas through scheduling. At this time, only 3 hosts are available. This means that host-level isolation cannot be guaranteed and that multiple replicas might be scheduled to the same host. But if the `isolation-level` value is set to `zone` instead of being left empty, this specifies the minimum physical isolation requirements for Region replicas. That is to say, PD will ensure that replicas of the same Region are scattered among different zones. PD will not perform corresponding scheduling even if following this isolation restriction does not meet the requirement of `max-replicas` for multiple replicas. +In the case of the 5-replica configuration, if z3 fails or is isolated as a whole, and cannot be recovered after a period of time (controlled by `max-store-down-time`), PD will make up the 5 replicas through scheduling. At this time, only 3 hosts are available. This means that host-level isolation cannot be guaranteed and that multiple replicas might be scheduled to the same host. But if the `isolation-level` value is set to `zone` instead of being left empty, this specifies the minimum physical isolation requirements for Region replicas. That is to say, PD will ensure that replicas of the same Region are scattered among different zones. PD will not perform corresponding scheduling even if following this isolation restriction does not meet the requirement of `max-replicas` for multiple replicas. For example, a TiKV cluster is distributed across three data zones z1, z2, and z3. Each Region has three replicas as required, and PD distributes the three replicas of the same Region to these three data zones respectively. If a power outage occurs in z1 and cannot be recovered after a period of time, PD determines that the Region replicas on z1 are no longer available. However, because `isolation-level` is set to `zone`, PD needs to strictly guarantee that different replicas of the same Region will not be scheduled on the same data zone. Because both z2 and z3 already have replicas, PD will not perform any scheduling under the minimum isolation level restriction of `isolation-level`, even if there are only two replicas at this moment. diff --git a/telemetry.md b/telemetry.md index 89b5f47709c9d..63b37acbef8c4 100644 --- a/telemetry.md +++ b/telemetry.md @@ -120,25 +120,6 @@ server_configs: -
- Deployment using TiDB Ansible - -Locate the following contents in the configuration file `tidb-ansible/conf/tidb.yml`: - -```yaml -# enable-telemetry: true -``` - -And change this content as follow: - -```yaml -enable-telemetry: false -``` - -See [Deploy TiDB Using TiDB Ansible](/online-deployment-using-ansible.md) for details. - -
-
Deployment in Kubernetes via TiDB Operator @@ -229,29 +210,6 @@ server_configs:
-
- Deployment using TiDB Ansible - -Locate the following content in the `tidb-ansible/conf/pd.yml` configuration file: - -```yaml -dashboard: - ... - # enable-telemetry: true -``` - -And change the content as follows: - -```yaml -dashboard: - ... - enable-telemetry: false -``` - -See [Deploy TiDB Using TiDB Ansible](/online-deployment-using-ansible.md) for details. - -
-
Deployment in Kubernetes via TiDB Operator diff --git a/three-data-centers-in-two-cities-deployment.md b/three-data-centers-in-two-cities-deployment.md index 847fb9b356436..9080e3aacfaa3 100644 --- a/three-data-centers-in-two-cities-deployment.md +++ b/three-data-centers-in-two-cities-deployment.md @@ -50,7 +50,7 @@ The configuration of the three DCs in two cities (Seattle and San Francisco) dep - From the illustration above, you can see that Seattle has two DCs: IDC1 and IDC2. IDC1 has three sets of racks: RAC1, RAC2, and RAC3. IDC2 has two racks: RAC4 and RAC5. The IDC3 DC in San Francisco has the RAC6 rack. - From the RAC1 rack illustrated above, TiDB and PD services are deployed on the same server. Each of the two TiKV servers are deployed with two TiKV instances (tikv-server). This is similar to RAC2, RAC4, RAC5, and RAC6. -- The TiDB server, the control machine, and the monitoring server are on RAC3. The TiDB server is deployed for regular maintenance and backup. TiDB Ansible, Prometheus, Grafana, and the restore tools are deployed on the control machine and monitoring machine. +- The TiDB server, the control machine, and the monitoring server are on RAC3. The TiDB server is deployed for regular maintenance and backup. Prometheus, Grafana, and the restore tools are deployed on the control machine and monitoring machine. - Another backup server can be added to deploy Mydumper and Drainer. Drainer saves binlog data to a specified location by outputting files, to achieve incremental backup. ## Configuration diff --git a/tidb-binlog/deploy-tidb-binlog.md b/tidb-binlog/deploy-tidb-binlog.md index b5dc150007bfc..e2d5095c7d58c 100644 --- a/tidb-binlog/deploy-tidb-binlog.md +++ b/tidb-binlog/deploy-tidb-binlog.md @@ -6,12 +6,7 @@ aliases: ['/docs/dev/tidb-binlog/deploy-tidb-binlog/','/docs/dev/reference/tidb- # TiDB Binlog Cluster Deployment -This document describes two methods of deploying TiDB Binlog: - -- [Deploy TiDB Binlog using TiDB Ansible](#deploy-tidb-binlog-using-tidb-ansible) -- [Deploy TiDB Binlog using a Binary package](#deploy-tidb-binlog-using-a-binary-package) - -It is recommended to deploy TiDB Binlog using TiDB Ansible. If you just want to do a simple testing, you can deploy TiDB Binlog using a Binary package. +This document describes how to [deploy TiDB Binlog using a Binary package](#deploy-tidb-binlog-using-a-binary-package). ## Hardware requirements @@ -24,244 +19,6 @@ In environments of development, testing and production, the requirements on serv | Pump | 3 | 8 core+ | SSD, 200 GB+ | 16G | | Drainer | 1 | 8 core+ | SAS, 100 GB+ (If binlogs are output as local files, the disk size depends on how long these files are retained.) | 16G | -## Deploy TiDB Binlog using TiDB Ansible - -### Step 1: Download TiDB Ansible - -1. Use the TiDB user account to log in to the control machine and go to the `/home/tidb` directory. The information about the branch of TiDB Ansible and the corresponding TiDB version is as follows. If you have questions regarding which version to use, email to [info@pingcap.com](mailto:info@pingcap.com) for more information or [file an issue](https://github.com/pingcap/tidb-ansible/issues/new). - - | tidb-ansible branch | TiDB version | Note | - | :------------------- | :------------ | :---- | - | master | master version | This version includes the latest features with a daily update. | - -2. Use the following command to download the corresponding branch of TiDB Ansible from the [TiDB Ansible project](https://github.com/pingcap/tidb-ansible) on GitHub. The default folder name is `tidb-ansible`. - - - Download the master version: - - {{< copyable "shell-regular" >}} - - ```bash - git clone https://github.com/pingcap/tidb-ansible.git - ``` - -### Step 2: Deploy Pump - -1. Modify the `tidb-ansible/inventory.ini` file. - - 1. Set `enable_binlog = True` to start `binlog` of the TiDB cluster. - - ```ini - ## binlog trigger - enable_binlog = True - ``` - - 2. Add the target machine IPs for `pump_servers`. - - ```ini - ## Binlog Part - [pump_servers] - 172.16.10.72 - 172.16.10.73 - 172.16.10.74 - ``` - - Pump retains the data of the latest 7 days by default. You can modify the value of the `gc` variable in the `tidb-ansible/conf/pump.yml` file (or `tidb-ansible/conf/pump-cluster.yml` in TiDB 3.0.0~3.0.2) and remove the related comments: - - {{< copyable "" >}} - - ```yaml - global: - # an integer value to control the expiry date of the binlog data, which indicates for how long (in days) the binlog data would be stored - # must be bigger than 0 - # gc: 7 - ``` - - Make sure the space of the deployment directory is sufficient for storing Binlog. For more details, see [Configure the deployment directory](/online-deployment-using-ansible.md#configure-the-deployment-directory). You can also set a separate deployment directory for Pump. - - ```ini - ## Binlog Part - [pump_servers] - pump1 ansible_host=172.16.10.72 deploy_dir=/data1/pump - pump2 ansible_host=172.16.10.73 deploy_dir=/data2/pump - pump3 ansible_host=172.16.10.74 deploy_dir=/data3/pump - ``` - -2. Deploy and start the TiDB cluster containing Pump. - - After configuring the `inventory.ini` file, you can choose one method from below to deploy the TiDB cluster. - - **Method #1**: Add Pump on the existing TiDB cluster. - - 1. Deploy `pump_servers` and `node_exporters`. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook deploy.yml --tags=pump -l ${pump1_ip},${pump2_ip},[${alias1_name},${alias2_name}] - ``` - - > **Note:** - > - > Do not add a space after the commas in the above command. Otherwise, an error is reported. - - 2. Start `pump_servers`. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook start.yml --tags=pump - ``` - - 3. Update and restart `tidb_servers`. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook rolling_update.yml --tags=tidb - ``` - - 4. Update the monitoring data. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook rolling_update_monitor.yml --tags=prometheus - ``` - - **Method #2**: Deploy a TiDB cluster containing Pump from scratch. - - For how to use TiDB Ansible to deploy the TiDB cluster, see [Deploy TiDB Using TiDB Ansible](/online-deployment-using-ansible.md). - -3. Check the Pump status. - - Use `binlogctl` to check the Pump status. Change the `pd-urls` parameter to the PD address of the cluster. If `State` is `online`, Pump is started successfully. - - {{< copyable "shell-regular" >}} - - ```bash - cd /home/tidb/tidb-ansible && - resources/bin/binlogctl -pd-urls=http://172.16.10.72:2379 -cmd pumps - ``` - - ``` - INFO[0000] pump: {NodeID: ip-172-16-10-72:8250, Addr: 172.16.10.72:8250, State: online, MaxCommitTS: 403051525690884099, UpdateTime: 2018-12-25 14:23:37 +0800 CST} - INFO[0000] pump: {NodeID: ip-172-16-10-73:8250, Addr: 172.16.10.73:8250, State: online, MaxCommitTS: 403051525703991299, UpdateTime: 2018-12-25 14:23:36 +0800 CST} - INFO[0000] pump: {NodeID: ip-172-16-10-74:8250, Addr: 172.16.10.74:8250, State: online, MaxCommitTS: 403051525717360643, UpdateTime: 2018-12-25 14:23:35 +0800 CST} - ``` - -### Step 3: Deploy Drainer - -1. Obtain the value of `initial_commit_ts`. - - When Drainer starts for the first time, the timestamp information `initial_commit_ts` is required. - - - If the replication is started from the latest time point, you just need to set `initial_commit_ts` to `-1`. - - - If the downstream database is MySQL or TiDB, to ensure data integrity, you need to perform full data backup and recovery. In this case, the value of `initial_commit_ts` must be the timestamp information of the full backup. - - If you use mydumper to perform full data backup, you can get the timestamp by referring to the `Pos` field in the metadata file from the export directory. An example of the metadata file is as follows: - - ``` - Started dump at: 2019-12-30 13:25:41 - SHOW MASTER STATUS: - Log: tidb-binlog - Pos: 413580274257362947 - GTID: - Finished dump at: 2019-12-30 13:25:41 - ``` - -2. Modify the `tidb-ansible/inventory.ini` file. - - Add the target machine IPs for `drainer_servers`. Set `initial_commit_ts` to the value you have obtained, which is only used for the initial start of Drainer. - - - Assume that the downstream is MySQL with the alias `drainer_mysql`: - - ```ini - [drainer_servers] - drainer_mysql ansible_host=172.16.10.71 initial_commit_ts="402899541671542785" - ``` - - - Assume that the downstream is `file` with the alias `drainer_file`: - - ```ini - [drainer_servers] - drainer_file ansible_host=172.16.10.71 initial_commit_ts="402899541671542785" - ``` - -3. Modify the configuration file. - - - Assume that the downstream is MySQL: - - {{< copyable "shell-regular" >}} - - ```bash - cd /home/tidb/tidb-ansible/conf && - cp drainer-cluster.toml drainer_mysql_drainer.toml && - vi drainer_mysql_drainer.toml - ``` - - > **Note:** - > - > Name the configuration file as `alias_drainer.toml`. Otherwise, the customized configuration file cannot be found during the deployment process. - - Set `db-type` to `mysql` and configure the downstream MySQL information: - - {{< copyable "" >}} - - ```toml - # downstream storage, equal to --dest-db-type - # Valid values are "mysql", "file", "tidb", and "kafka". - db-type = "mysql" - - # the downstream MySQL protocol database - [syncer.to] - host = "172.16.10.72" - user = "root" - password = "123456" - port = 3306 - ``` - - - Assume that the downstream is incremental backup data: - - {{< copyable "shell-regular" >}} - - ```bash - cd /home/tidb/tidb-ansible/conf && - cp drainer-cluster.toml drainer_file_drainer.toml && - vi drainer_file_drainer.toml - ``` - - Set `db-type` to `file`. - - {{< copyable "" >}} - - ```toml - # downstream storage, equal to --dest-db-type - # Valid values are "mysql", "file", "tidb", and "kafka". - db-type = "file" - - # Uncomment this if you want to use "file" as "db-type". - [syncer.to] - # default data directory: "{{ deploy_dir }}/data.drainer" - dir = "data.drainer" - ``` - -4. Deploy Drainer. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook deploy_drainer.yml - ``` - -5. Start Drainer. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook start_drainer.yml - ``` - ## Deploy TiDB Binlog using a Binary package ### Download the official Binary package @@ -620,7 +377,7 @@ The following part shows how to use Pump and Drainer based on the nodes above. > **Note:** > - > If the downstream is MySQL/TiDB, to guarantee the data integrity, you need to obtain the `initial-commit-ts` value and make a full backup of the data and restore the data before the initial start of Drainer. For details, see [Deploy Drainer](#step-3-deploy-drainer). + > If the downstream is MySQL/TiDB, to guarantee the data integrity, you need to obtain the `initial-commit-ts` value and make a full backup of the data and restore the data before the initial start of Drainer. When Drainer is started for the first time, use the `initial-commit-ts` parameter. diff --git a/tidb-binlog/monitor-tidb-binlog-cluster.md b/tidb-binlog/monitor-tidb-binlog-cluster.md index 0efd10ab0fdd9..0c2525f3c1e01 100644 --- a/tidb-binlog/monitor-tidb-binlog-cluster.md +++ b/tidb-binlog/monitor-tidb-binlog-cluster.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/tidb-binlog/monitor-tidb-binlog-cluster/','/docs/dev/refere # TiDB Binlog Monitoring -After you have deployed TiDB Binlog using TiDB Ansible successfully, you can go to the Grafana Web (default address: , default account: admin, password: admin) to check the state of Pump and Drainer. +After you have deployed TiDB Binlog successfully, you can go to the Grafana Web (default address: , default account: admin, password: admin) to check the state of Pump and Drainer. ## Monitoring metrics diff --git a/tidb-binlog/upgrade-tidb-binlog.md b/tidb-binlog/upgrade-tidb-binlog.md index c1190e1f70650..56a8e9de16b43 100644 --- a/tidb-binlog/upgrade-tidb-binlog.md +++ b/tidb-binlog/upgrade-tidb-binlog.md @@ -6,26 +6,7 @@ aliases: ['/docs/dev/tidb-binlog/upgrade-tidb-binlog/','/docs/dev/reference/tidb # Upgrade TiDB Binlog -This document introduces how to upgrade TiDB Binlog that is deployed with TiDB Ansible and deployed manually to the latest [cluster](/tidb-binlog/tidb-binlog-overview.md) version. There is also a section on how to upgrade TiDB Binlog from an earlier incompatible version (Kafka/Local version) to the latest version. - -## Upgrade TiDB Binlog deployed with TiDB Ansible - -Follow the steps in this section if you deploy TiDB Binlog with [TiDB Ansible Playbook](https://github.com/pingcap/tidb-ansible). - -### Upgrade Pump - -First, upgrade the Pump component: - -1. Copy the new version of the binary `pump` file into the`({ resources_dir })/bin` directory. -2. Execute the `ansible-playbook rolling_update.yml --tags=pump` command to perform a rolling update for Pump. - -### Upgrade Drainer - -Second, upgrade the Drainer component: - -1. Copy the new version of the binary `drainer` file into the`({ resources_dir })/bin` directory. -2. Execute the `ansible-playbook stop_drainer.yml --tags=drainer` command. -3. Execute the `ansible-playbook start_drainer.yml --tags=drainer` command. +This document introduces how to upgrade TiDB Binlog that is deployed manually to the latest [cluster](/tidb-binlog/tidb-binlog-overview.md) version. There is also a section on how to upgrade TiDB Binlog from an earlier incompatible version (Kafka/Local version) to the latest version. ## Upgrade TiDB Binlog deployed manually diff --git a/tidb-lightning/deploy-tidb-lightning.md b/tidb-lightning/deploy-tidb-lightning.md index 9e9765382f001..f93f16ef6c6f7 100644 --- a/tidb-lightning/deploy-tidb-lightning.md +++ b/tidb-lightning/deploy-tidb-lightning.md @@ -6,7 +6,7 @@ aliases: ['/docs/dev/tidb-lightning/deploy-tidb-lightning/','/docs/dev/reference # TiDB Lightning Deployment -This document describes the hardware requirements of TiDB Lightning using the Local-backend, and how to deploy it using TiDB Ansible or manually. +This document describes the hardware requirements of TiDB Lightning using the Local-backend, and how to deploy it manually. If you do not want the TiDB services to be impacted, read [TiDB Lightning TiDB-backend](/tidb-lightning/tidb-lightning-backends.md#tidb-lightning-tidb-backend) for the changes to the deployment steps. @@ -73,70 +73,7 @@ If the data source consists of CSV files, see [CSV support](/tidb-lightning/migr ## Deploy TiDB Lightning -This section describes two deployment methods of TiDB Lightning: - -- [Deploy TiDB Lightning using TiDB Ansible](#deploy-tidb-lightning-using-tidb-ansible) -- [Deploy TiDB Lightning manually](#deploy-tidb-lightning-manually) - -### Deploy TiDB Lightning using TiDB Ansible - -You can deploy TiDB Lightning using TiDB Ansible together with the [deployment of the TiDB cluster itself using TiDB Ansible](/online-deployment-using-ansible.md). - -1. Edit `inventory.ini` to configure an IP address for `tidb-lightning`. - - ```ini - ... - - [lightning_server] - 192.168.20.10 - - ... - ``` - -2. Configure `tidb-lightning` by editing the settings under `group_vars/*.yml`. - - * `group_vars/lightning_server.yml` - - ```yaml - --- - dummy: - - # The listening port for metrics gathering. Should be open to the monitoring servers. - tidb_lightning_pprof_port: 8289 - - # The file path that tidb-lightning reads the data source (Dumpling SQL dump or CSV) from. - data_source_dir: "{{ deploy_dir }}/mydumper" - ``` - -3. Deploy the cluster. - - ```sh - ansible-playbook bootstrap.yml && - ansible-playbook deploy.yml - ``` - -4. Mount the data source to the path specified in the `data_source_dir` setting. - -5. Log in to the `tidb-lightning` server and edit the `conf/tidb-lighting.toml` file as follows: - - ``` - [tikv-importer] - # Uses the Local-backend. - backend = "local" - # Sets the directory for temporarily storing the sorted key-value pairs. - # The target directory must be empty. - "sorted-kv-dir" = "/mnt/ssd/sorted-kv-dir" - - [tidb] - # An address of pd-server. - pd-addr = "172.16.31.4:2379 - ``` - -6. Log in to the `tidb-lightning` server, and manually run the following command to start Lightning and import the data into the TiDB cluster. - - ```sh - scripts/start_lightning.sh - ``` +This section describes how to [deploy TiDB Lightning manually](#deploy-tidb-lightning-manually). ### Deploy TiDB Lightning manually diff --git a/tidb-lightning/monitor-tidb-lightning.md b/tidb-lightning/monitor-tidb-lightning.md index b989eb390fbc7..bc7ea4832f3ab 100644 --- a/tidb-lightning/monitor-tidb-lightning.md +++ b/tidb-lightning/monitor-tidb-lightning.md @@ -10,8 +10,7 @@ aliases: ['/docs/dev/tidb-lightning/monitor-tidb-lightning/','/docs/dev/referenc ## Monitor configuration -- If TiDB Lightning is installed using TiDB Ansible, simply add the servers to the `[monitored_servers]` section in the `inventory.ini` file. Then the Prometheus server can collect their metrics. -- If TiDB Lightning is manually installed, follow the instructions below. +If TiDB Lightning is manually installed, follow the instructions below. The metrics of `tidb-lightning` can be gathered directly by Prometheus as long as it is discovered. You can set the metrics port in `tidb-lightning.toml`: @@ -47,8 +46,7 @@ scrape_configs: [Grafana](https://grafana.com/) is a web interface to visualize Prometheus metrics as dashboards. -If TiDB Lightning is installed using TiDB Ansible, its dashboard is already installed. -Otherwise, the dashboard JSON can be imported from . +When you [deploy a TiDB cluster using TiUP](/production-deployment-using-tiup.md) and have added Grafana and Prometheus in the topology configuration, a set of [Grafana + Prometheus monitoring platform](/tidb-monitoring-framework.md) is deployed simultaneously. In this situation, you must first import [the JSON file of the dashboard](https://raw.githubusercontent.com/pingcap/tidb-ansible/master/scripts/lightning.json). ### Row 1: Speed diff --git a/tidb-lightning/tidb-lightning-backends.md b/tidb-lightning/tidb-lightning-backends.md index 05e8a611ba65b..7793db8ef6331 100644 --- a/tidb-lightning/tidb-lightning-backends.md +++ b/tidb-lightning/tidb-lightning-backends.md @@ -61,37 +61,6 @@ The speed of TiDB Lightning using TiDB-backend is limited by the SQL processing * An SSD large enough to store the entire data source, preferring higher read speed * 1 Gigabit network card -#### Deploy TiDB Lightning using TiDB Ansible - -1. The `[importer_server]` section in `inventory.ini` can be left blank. - - ```ini - ... - - [importer_server] - # keep empty - - [lightning_server] - 192.168.20.10 - - ... - ``` - -2. The `tikv_importer_port` setting in `group_vars/all.yml` is ignored, and the file `group_vars/importer_server.yml` does not need to be changed. But you need to edit `conf/tidb-lightning.yml` and change the `backend` setting to `tidb`. - - ```yaml - ... - tikv_importer: - backend: "tidb" # <-- change this - ... - ``` - -3. Bootstrap and deploy the cluster as usual. - -4. Mount the data source for TiDB Lightning as usual. - -5. Start `tidb-lightning` as usual. - #### Manual deployment You do not need to download and configure `tikv-importer`. You can download TiDB Lightning from [here](/download-ecosystem-tools.md#tidb-lightning). @@ -273,10 +242,7 @@ password = "" ### Deployment for Importer-backend mode -This section describes two deployment methods of TiDB Lightning in the Importer-backend mode: - -- [Deploy TiDB Lightning using TiDB Ansible](#deploy-tidb-lightning-using-tidb-ansible) -- [Deploy TiDB Lightning manually](#deploy-tidb-lightning-manually) +This section describes how to [deploy TiDB Lightning manually](#deploy-tidb-lightning-manually) in the Importer-backend mode: #### Hardware requirements @@ -302,81 +268,6 @@ To achieve the best performance, it is recommended to use the following hardware If you have sufficient machines, you can deploy multiple `tidb lightning` + `tikv importer` servers, with each working on a distinct set of tables, to import the data in parallel. -#### Deploy TiDB Lightning using TiDB Ansible - -You can deploy TiDB Lightning using TiDB Ansible together with the [deployment of the TiDB cluster itself using TiDB Ansible](/online-deployment-using-ansible.md). - -1. Edit `inventory.ini` to add the addresses of the `tidb-lightning` and `tikv-importer` servers. - - ```ini - ... - - [importer_server] - 192.168.20.9 - - [lightning_server] - 192.168.20.10 - - ... - ``` - -2. Configure these tools by editing the settings under `group_vars/*.yml`. - - * `group_vars/all.yml` - - ```yaml - ... - # The listening port of tikv-importer. Should be open to the tidb-lightning server. - tikv_importer_port: 8287 - ... - ``` - - * `group_vars/lightning_server.yml` - - ```yaml - --- - dummy: - - # The listening port for metrics gathering. Should be open to the monitoring servers. - tidb_lightning_pprof_port: 8289 - - # The file path that tidb-lightning reads the data source (Mydumper SQL dump or CSV) from. - data_source_dir: "{{ deploy_dir }}/mydumper" - ``` - - * `group_vars/importer_server.yml` - - ```yaml - --- - dummy: - - # The file path to store engine files. Should reside on a partition with a large capacity. - import_dir: "{{ deploy_dir }}/data.import" - ``` - -3. Deploy the cluster. - - ```sh - ansible-playbook bootstrap.yml && - ansible-playbook deploy.yml - ``` - -4. Mount the data source to the path specified in the `data_source_dir` setting. - -5. Log in to the `tikv-importer` server, and manually run the following command to start Importer. - - ```sh - scripts/start_importer.sh - ``` - -6. Log in to the `tidb-lightning` server, and manually run the following command to start Lightning and import the data into the TiDB cluster. - - ```sh - scripts/start_lightning.sh - ``` - -7. After completion, run `scripts/stop_importer.sh` on the `tikv-importer` server to stop Importer. - #### Deploy TiDB Lightning manually ##### Step 1: Deploy a TiDB cluster diff --git a/tidb-lightning/tidb-lightning-faq.md b/tidb-lightning/tidb-lightning-faq.md index d7951e6a85e1b..ab3a32bc4ded3 100644 --- a/tidb-lightning/tidb-lightning-faq.md +++ b/tidb-lightning/tidb-lightning-faq.md @@ -110,16 +110,12 @@ Yes, as long as every `tidb-lightning` instance operates on different tables. To stop the `tikv-importer` process, you can choose the corresponding operation according to your deployment method. -- For deployment using TiDB Ansible: run `scripts/stop_importer.sh` on the Importer server. - - For manual deployment: if `tikv-importer` is running in foreground, press Ctrl+C to exit. Otherwise, obtain the process ID using the `ps aux | grep tikv-importer` command and then terminate the process using the `kill «pid»` command. ## How to stop the `tidb-lightning` process? To stop the `tidb-lightning` process, you can choose the corresponding operation according to your deployment method. -- For deployment using TiDB Ansible: run `scripts/stop_lightning.sh` on the Lightning server. - - For manual deployment: if `tidb-lightning` is running in foreground, press Ctrl+C to exit. Otherwise, obtain the process ID using the `ps aux | grep tidb-lighting` command and then terminate the process using the `kill -2 «pid»` command. ## Why the `tidb-lightning` process suddenly quits while running in background? diff --git a/tidb-monitoring-api.md b/tidb-monitoring-api.md index 5054d582d4bcd..f5e6313b60b29 100644 --- a/tidb-monitoring-api.md +++ b/tidb-monitoring-api.md @@ -75,7 +75,6 @@ curl http://127.0.0.1:2379/pd/api/v1/stores The metrics interface monitors the state and performance of the entire TiDB cluster. -- If you use TiDB Ansible to deploy the TiDB cluster, the monitoring system (Prometheus and Grafana) is deployed at the same time. - If you use other deployment ways, [deploy Prometheus and Grafana](/deploy-monitoring-services.md) before using this interface. After Prometheus and Grafana are successfully deployed, [configure Grafana](/deploy-monitoring-services.md#configure-grafana). diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index f5c900614fc67..90d28c38238ed 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/tiflash/monitor-tiflash/','/docs/dev/reference/tiflash/moni This document describes the monitoring items of TiFlash. -If you use TiUP or TiDB Ansible to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). +If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & Grafana) is deployed at the same time. For more information, see [Overview of the Monitoring Framework](/tidb-monitoring-framework.md). The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, and so on. A lot of metrics are there to help you diagnose. diff --git a/tikv-control.md b/tikv-control.md index ef5560e5bca8c..62bf254362b57 100644 --- a/tikv-control.md +++ b/tikv-control.md @@ -8,7 +8,6 @@ aliases: ['/docs/dev/tikv-control/','/docs/dev/reference/tools/tikv-control/'] TiKV Control (`tikv-ctl`) is a command line tool of TiKV, used to manage the cluster. Its installation directory is as follows: -* If the cluster is deployed using TiDB Ansible, `tikv-ctl` directory is in the `resources/bin` subdirectory under the `ansible` directory. * If the cluster is deployed using TiUP, `tikv-ctl` directory is in the in `~/.tiup/components/ctl/{VERSION}/` directory. ## Use TiKV Control in TiUP diff --git a/tispark-overview.md b/tispark-overview.md index 7668bd8eece69..303c1263ed1df 100644 --- a/tispark-overview.md +++ b/tispark-overview.md @@ -90,8 +90,6 @@ spark-shell --jars $TISPARK_FOLDER/tispark-${name_with_version}.jar If you do not have a Spark cluster, we recommend using the standalone mode. To use the Spark Standalone model, you can simply place a compiled version of Spark on each node of the cluster. If you encounter problems, see its [official website](https://spark.apache.org/docs/latest/spark-standalone.html). And you are welcome to [file an issue](https://github.com/pingcap/tispark/issues/new) on our GitHub. -If you are using TiDB Ansible to deploy a TiDB cluster, you can also use TiDB Ansible to deploy a Spark standalone cluster, and TiSpark is also deployed at the same time. - #### Download and install You can download [Apache Spark](https://spark.apache.org/downloads.html) diff --git a/tiup/tiup-cluster.md b/tiup/tiup-cluster.md index 2965ffc1a03b0..669007e59fdef 100644 --- a/tiup/tiup-cluster.md +++ b/tiup/tiup-cluster.md @@ -41,7 +41,7 @@ Available Commands: display Display information of a TiDB cluster list List all clusters audit Show audit log of cluster operation - import Import an exist TiDB cluster from TiDB-Ansible + import Import an exist TiDB cluster from TiDB Ansible edit-config Edit TiDB cluster config reload Reload a TiDB cluster's config and restart if needed patch Replace the remote package with a specified package and restart the service diff --git a/troubleshoot-high-disk-io.md b/troubleshoot-high-disk-io.md index 6274abd261693..c462f8a05b51f 100644 --- a/troubleshoot-high-disk-io.md +++ b/troubleshoot-high-disk-io.md @@ -13,7 +13,7 @@ If TiDB's response slows down after you have troubleshot the CPU bottleneck and ### Locate I/O issues from monitor -The quickest way to locate I/O issues is to view the overall I/O status from the monitor, such as the Grafana dashboard which is deployed by default by TiDB Ansible and TiUP. The dashboard panels related to I/O include **Overview**, **Node_exporter**, and **Disk-Performance**. +The quickest way to locate I/O issues is to view the overall I/O status from the monitor, such as the Grafana dashboard which is deployed by default by TiUP. The dashboard panels related to I/O include **Overview**, **Node_exporter**, and **Disk-Performance**. #### The first type of monitoring panels @@ -78,7 +78,7 @@ In addition, some other panel metrics might help you determine whether the bottl ### I/O issues found in alerts -The cluster deployment tools (TiDB Ansible and TiUP) deploy the cluster with alert components by default that have built-in alert items and thresholds. The following alert items are related to I/O: +The cluster deployment tool (TiUP) deploys the cluster with alert components by default that have built-in alert items and thresholds. The following alert items are related to I/O: - TiKV_write_stall - TiKV_raft_log_lag diff --git a/troubleshoot-tidb-lightning.md b/troubleshoot-tidb-lightning.md index 5a8857fbe2617..23925b4d90ee3 100644 --- a/troubleshoot-tidb-lightning.md +++ b/troubleshoot-tidb-lightning.md @@ -120,13 +120,7 @@ See the [Checkpoints control](/tidb-lightning/tidb-lightning-checkpoints.md#chec **Solutions**: -1. Ensure Lightning and the source database are using the same time zone. When deploying via TiDB Ansible, the timezone is defined in `inventory.ini`. - - ```ini - # inventory.ini - [all:vars] - timezone = Asia/Shanghai - ``` +1. Ensure Lightning and the source database are using the same time zone. When executing Lightning directly, the time zone can be forced using the `$TZ` environment variable. diff --git a/upgrade-tidb-using-ansible.md b/upgrade-tidb-using-ansible.md deleted file mode 100644 index 1d5b78ce60d07..0000000000000 --- a/upgrade-tidb-using-ansible.md +++ /dev/null @@ -1,230 +0,0 @@ ---- -title: Upgrade TiDB Using TiDB Ansible -summary: Learn how to upgrade TiDB using TiDB Ansible. -aliases: ['/docs/dev/upgrade-tidb-using-ansible/','/docs/dev/how-to/upgrade/from-previous-version/','/docs/dev/how-to/upgrade/rolling-updates-with-ansible/'] ---- - -# Upgrade TiDB Using TiDB Ansible - -This document is targeted for users who want to upgrade from TiDB 2.0, 2.1, 3.0, or 3.1 versions to the latest development version (`latest`), or from an earlier development version to the latest development version. The latest development version is compatible with [TiDB Binlog of the cluster version](/tidb-binlog/tidb-binlog-overview.md). - -> **Warning:** -> -> The latest development version of TiDB is not a stable version. Do not use it in the production environment. - -## Upgrade caveat - -- Rolling back to 2.1.x or earlier versions after upgrading is not supported. -- Before upgrading to `latest` from 2.0.6 or earlier versions, check if there are any running DDL operations, especially time-consuming ones like `Add Index`. If there are any, wait for the DDL operations to finish before you upgrade. -- Parallel DDL is supported in TiDB 2.1 and later versions. Therefore, for clusters with a TiDB version earlier than 2.0.1, rolling update to `latest` is not supported. To upgrade, you can choose either of the following two options: - - - Stop the cluster and upgrade to `latest` directly. - - Roll update to 2.0.1 or later 2.0.x versions, and then roll update to the `latest` version. - -> **Note:** -> -> Do not execute any DDL statements during the upgrading process, otherwise the undefined behavior error might occur. - -## Step 1: Install Ansible and dependencies on the control machine - -> **Note:** -> -> If you have installed Ansible and its dependencies, you can skip this step. - -The latest development version of TiDB Ansible depends on Ansible 2.4.2 ~ 2.7.11 (`2.4.2 ≦ ansible ≦ 2.7.11`, Ansible 2.7.11 recommended) and the Python modules of `jinja2 ≧ 2.9.6` and `jmespath ≧ 0.9.0`. - -To make it easy to manage dependencies, use `pip` to install Ansible and its dependencies. For details, see [Install Ansible and its dependencies on the control machine](/online-deployment-using-ansible.md#step-4-install-tidb-ansible-and-its-dependencies-on-the-control-machine). For offline environment, see [Install Ansible and its dependencies offline on the control machine](/offline-deployment-using-ansible.md#step-3-install-tidb-ansible-and-its-dependencies-offline-on-the-control-machine). - -After the installation is finished, you can view the version information using the following command: - -{{< copyable "shell-regular" >}} - -```bash -ansible --version -``` - -``` -ansible 2.7.11 -``` - -{{< copyable "shell-regular" >}} - -```bash -pip show jinja2 -``` - -``` -Name: Jinja2 -Version: 2.10 -``` - -{{< copyable "shell-regular" >}} - -```bash -pip show jmespath -``` - -``` -Name: jmespath -Version: 0.9.0 -``` - -> **Note:** -> -> - You must install Ansible and its dependencies following the above procedures. -> - Make sure that the Jinja2 version is correct, otherwise an error occurs when you start Grafana. -> - Make sure that the jmespath version is correct, otherwise an error occurs when you perform a rolling update to TiKV. - -## Step 2: Download TiDB Ansible to the control machine - -1. Log in to the control machine using the `tidb` user account and enter the `/home/tidb` directory. - -2. Back up the `tidb-ansible` folders of TiDB 2.0, 2.1, 3.0, or an earlier `latest` version using the following command: - - {{< copyable "shell-regular" >}} - - ```bash - mv tidb-ansible tidb-ansible-bak - ``` - -3. Download the tidb-ansible with the tag corresponding to the `latest` version of TiDB. For more details, See [Download TiDB-Ansible to the control machine](/online-deployment-using-ansible.md#step-3-download-tidb-ansible-to-the-control-machine). The default folder name is `tidb-ansible`. - - {{< copyable "shell-regular" >}} - - ```bash - git clone https://github.com/pingcap/tidb-ansible.git - ``` - -## Step 3: Edit the `inventory.ini` file and the configuration file - -Log in to the control machine using the `tidb` user account and enter the `/home/tidb/tidb-ansible` directory. - -### Edit the `inventory.ini` file - -Edit the `inventory.ini` file. For IP information, see the `/home/tidb/tidb-ansible-bak/inventory.ini` backup file. - -> **Note:** -> -> Pay special attention to the following variables configuration. For variable meaning, see [Description of other variables](/online-deployment-using-ansible.md#edit-other-variables-optional). - -1. Make sure that `ansible_user` is the normal user. For unified privilege management, remote installation using the root user is no longer supported. The default configuration uses the `tidb` user as the SSH remote user and the program running user. - - ``` - ## Connection - # ssh via normal user - ansible_user = tidb - ``` - - You can refer to [How to configure SSH mutual trust and sudo rules on the control machine](/online-deployment-using-ansible.md#step-5-configure-the-ssh-mutual-trust-and-sudo-rules-on-the-control-machine) to automatically configure the mutual trust among hosts. - -2. Keep the `process_supervision` variable consistent with that in the previous version. It is recommended to use `systemd` by default. - - ``` - # process supervision, [systemd, supervise] - process_supervision = systemd - ``` - - If you need to modify this variable, see [How to modify the supervision method of a process from `supervise` to `systemd`](/online-deployment-using-ansible.md#how-to-modify-the-supervision-method-of-a-process-from-supervise-to-systemd). Before you upgrade, first use the `/home/tidb/tidb-ansible-bak/` backup branch to modify the supervision method of a process. - -3. Add the `cpu_architecture` parameter according to the CPU architecture. The default value is `amd64`. - -### Edit the configuration file of TiDB cluster components - -If you have previously customized the configuration file of TiDB cluster components, refer to the backup file to modify the corresponding configuration file in the `/home/tidb/tidb-ansible/conf` directory. - -**Note the following parameter changes:** - -- In the TiKV configuration, `end-point-concurrency` is changed to three parameters: `high-concurrency`, `normal-concurrency` and `low-concurrency`. - - ```yaml - readpool: - coprocessor: - # Notice: if CPU_NUM > 8, default thread pool size for coprocessors - # will be set to CPU_NUM * 0.8. - # high-concurrency: 8 - # normal-concurrency: 8 - # low-concurrency: 8 - ``` - - > **Note:** - > - > For the cluster topology of multiple TiKV instances (processes) on a single machine, you need to modify the three parameters above. - - Recommended configuration: the number of TiKV instances \* the parameter value = the number of CPU cores \* 0.8. - -- In the TiKV configuration, the `block-cache-size` parameter of different CFs is changed to `block-cache`. - - ``` - storage: - block-cache: - capacity: "1GB" - ``` - - > **Note:** - > - > For the cluster topology of multiple TiKV instances (processes) on a single machine, you need to modify the `capacity` parameter. - - Recommended configuration: `capacity` = MEM_TOTAL \* 0.5 / the number of TiKV instances. If the `capacity` parameter is configured reasonably in the current version, it needs no modification. - -- In the TiKV configuration, you need to configure the `tikv_status_port` port for the multiple instances on a single machine scenario. If the current version has the above configuration, it needs no modification. -- Before you configure `tikv_status_port`, check whether a port conflict exists. - - ``` - [tikv_servers] - TiKV1-1 ansible_host=172.16.10.4 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv1" - TiKV1-2 ansible_host=172.16.10.4 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv1" - TiKV2-1 ansible_host=172.16.10.5 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv2" - TiKV2-2 ansible_host=172.16.10.5 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv2" - TiKV3-1 ansible_host=172.16.10.6 deploy_dir=/data1/deploy tikv_port=20171 tikv_status_port=20181 labels="host=tikv3" - TiKV3-2 ansible_host=172.16.10.6 deploy_dir=/data2/deploy tikv_port=20172 tikv_status_port=20182 labels="host=tikv3" - ``` - -## Step 4: Download TiDB latest binary to the control machine - -Make sure that `tidb_version = latest` in the `tidb-ansible/inventory.ini` file, and then run the following command to download TiDB latest binary to the control machine: - -{{< copyable "shell-regular" >}} - -```bash -ansible-playbook local_prepare.yml -``` - -## Step 5: Perform a rolling update to TiDB cluster components - -- If the `process_supervision` variable uses the default `systemd` parameter, perform a rolling update to the TiDB cluster using the following command corresponding to your current TiDB cluster version. - - - When the TiDB cluster version < 3.0.0, use `excessive_rolling_update.yml`. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook excessive_rolling_update.yml - ``` - - - When the TiDB cluster version ≧ 3.0.0, use `rolling_update.yml` for both rolling updates and daily rolling restarts. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook rolling_update.yml - ``` - -- If the `process_supervision` variable uses the `supervise` parameter, perform a rolling update to the TiDB cluster using `rolling_update.yml`, no matter what version the current TiDB cluster is. - - {{< copyable "shell-regular" >}} - - ```bash - ansible-playbook rolling_update.yml - ``` - -## Step 6: Perform a rolling update to TiDB monitoring components - -{{< copyable "shell-regular" >}} - -```bash -ansible-playbook rolling_update_monitor.yml -``` - -> **Note:** -> -> By default, TiDB (starting from v4.0.2) periodically shares usage details with PingCAP to help understand how to improve the product. For details about what is shared and how to disable the sharing, see [Telemetry](/telemetry.md). diff --git a/upgrade-tidb-using-tiup.md b/upgrade-tidb-using-tiup.md index 3dc45cd204ad8..04ca021c490f3 100644 --- a/upgrade-tidb-using-tiup.md +++ b/upgrade-tidb-using-tiup.md @@ -1,7 +1,7 @@ --- title: Upgrade TiDB Using TiUP summary: Learn how to upgrade TiDB using TiUP. -aliases: ['/docs/dev/upgrade-tidb-using-tiup/','/docs/dev/how-to/upgrade/using-tiup/'] +aliases: ['/docs/dev/upgrade-tidb-using-tiup/','/docs/dev/how-to/upgrade/using-tiup/','/docs/dev/upgrade-tidb-using-ansible/','/docs/dev/how-to/upgrade/from-previous-version/','/docs/dev/how-to/upgrade/rolling-updates-with-ansible/','/tidb/dev/upgrade-tidb-using-ansible/'] --- # Upgrade TiDB Using TiUP @@ -10,6 +10,10 @@ This document is targeted for users who want to upgrade from TiDB 3.0 or 3.1 ver If you have deployed the TiDB cluster using TiDB Ansible, you can use TiUP to import the TiDB Ansible configuration and perform the upgrade. +> **Note:** +> +> Since TiDB v4.0, PingCAP no longer provides support for TiDB Ansible. Since TiDB v5.0, PingCAP no longer provides TiDB Ansible documents. If you want to read the document that introduces how to upgrade a TiDB cluster using TiDB Ansible, see [Upgrade TiDB Using TiDB Ansible](https://docs.pingcap.com/tidb/v4.0/upgrade-tidb-using-ansible). + ## Upgrade caveat - After the upgrade, rolling back to 3.0 or earlier versions is not supported. @@ -83,7 +87,7 @@ tiup update cluster > **Note:** > -> + Currently, TiUP only supports the `systemd` supervision method of a process. If you have previously selected the `supervise` method when deploying TiDB with TiDB Ansible, you need to modify the supervision method from `supervise` to `systemd` according to [Deploy TiDB Using TiDB Ansible](/online-deployment-using-ansible.md#how-to-modify-the-supervision-method-of-a-process-from-supervise-to-systemd). +> + Currently, TiUP only supports the `systemd` supervision method of a process. If you have previously selected the `supervise` method when deploying TiDB with TiDB Ansible, you need to modify the supervision method from `supervise` to `systemd` according to [Deploy TiDB Using TiDB Ansible](https://docs.pingcap.com/tidb/v4.0/upgrade-tidb-using-ansible#how-to-modify-the-supervision-method-of-a-process-from-supervise-to-systemd). > + If the original cluster is deployed using TiUP, you can skip this step. > + Currently, the `inventory.ini` configuration file is identified by default. If your configuration file uses another name, specify this name. > + Ensure that the state of the current cluster is consistent with the topology in `inventory.ini`; that components of the cluster are operating normally. Otherwise, the cluster metadata becomes abnormal after the import.