From a98876bab3b49338033fcd82309ae7d0ed5303b2 Mon Sep 17 00:00:00 2001 From: GMHDBJD <35025882+GMHDBJD@users.noreply.github.com> Date: Mon, 25 Mar 2024 19:38:16 +0800 Subject: [PATCH] rename `tidb_ddl_version` to `tidb_enable_fast_create_table` (#16789) --- TOC-tidb-cloud.md | 2 +- TOC.md | 2 +- accelerated-table-creation.md | 57 +++++++++++++++++++++++++++++++++++ basic-features.md | 2 +- ddl-v2.md | 56 ---------------------------------- releases/release-7.6.0.md | 6 ++-- system-variables.md | 11 ++++--- 7 files changed, 69 insertions(+), 67 deletions(-) create mode 100644 accelerated-table-creation.md delete mode 100644 ddl-v2.md diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index 5c46152b16969..b50aae1a4cdce 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -603,7 +603,7 @@ - [`SESSION_CONNECT_ATTRS`](/performance-schema/performance-schema-session-connect-attrs.md) - [Metadata Lock](/metadata-lock.md) - [Use UUIDs](/best-practices/uuid.md) - - [TiDB DDL V2](/ddl-v2.md) + - [TiDB Accelerated Table Creation](/accelerated-table-creation.md) - [System Variables](/system-variables.md) - [Server Status Variables](/status-variables.md) - Storage Engines diff --git a/TOC.md b/TOC.md index 4fc8ade65b8bd..ed8ec3c253e6d 100644 --- a/TOC.md +++ b/TOC.md @@ -983,7 +983,7 @@ - [`SESSION_CONNECT_ATTRS`](/performance-schema/performance-schema-session-connect-attrs.md) - [`SYS`](/sys-schema.md) - [Metadata Lock](/metadata-lock.md) - - [TiDB DDL V2](/ddl-v2.md) + - [TiDB Accelerated Table Creation](/accelerated-table-creation.md) - UI - TiDB Dashboard - [Overview](/dashboard/dashboard-intro.md) diff --git a/accelerated-table-creation.md b/accelerated-table-creation.md new file mode 100644 index 0000000000000..f97eec9d5cfd0 --- /dev/null +++ b/accelerated-table-creation.md @@ -0,0 +1,57 @@ +--- +title: TiDB Accelerated Table Creation +summary: Learn the concept, principles, and implementation details of performance optimization for creating tables in TiDB. +aliases: ['/tidb/dev/ddl-v2/'] +--- + +# TiDB Accelerated Table Creation + +TiDB v7.6.0 introduces the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_enable_fast_create_table-new-in-v800) to support accelerating table creation, which improves the efficiency of bulk table creation. Starting from v8.0.0, this system variable is renamed to [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). + +TiDB uses the online asynchronous schema change algorithm to change the metadata. All DDL jobs are submitted to the `mysql.tidb_ddl_job` table, and the owner node pulls the DDL job to execute. After executing each phase of the online DDL algorithm, the DDL job is marked as completed and moved to the `mysql.tidb_ddl_history` table. Therefore, DDL statements can only be executed on the owner node and cannot be linearly extended. + +However, for some DDL statements, it is not necessary to strictly follow the online DDL algorithm. For example, the `CREATE TABLE` statement only has two states for the job: `none` and `public`. Therefore, TiDB can simplify the execution process of DDL, and executes the `CREATE TABLE` statement on a non-owner node to accelerate table creation. + +> **Warning:** +> +> This feature is currently an experimental feature and it is not recommended to use in a production environment. This feature might change or be removed without prior notice. If you find a bug, please give feedback by raising an [issue](https://github.com/pingcap/tidb/issues) on GitHub. + +## Compatibility with TiDB tools + +- [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) does not support replicating the tables that are created by `tidb_enable_fast_create_table`. + +## Limitation + +You can now use performance optimization for table creation only in the [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) statement, and this statement must not include any foreign key constraints. + +## Use `tidb_enable_fast_create_table` to accelerate table creation + +You can enable or disable performance optimization for creating tables by specifying the value of the system variable [`tidb_enable_fast_create_table`](/system-variables.md#tidb_enable_fast_create_table-new-in-v800). + +To enable performance optimization for creating tables, set the value of this variable to `ON`: + +```sql +SET GLOBAL tidb_enable_fast_create_table = ON; +``` + +To disable performance optimization for creating tables, set the value of this variable to `OFF`: + +```sql +SET GLOBAL tidb_enable_fast_create_table = OFF; +``` + +## Implementation principle + +The detailed implementation principle of performance optimization for table creation is as follows: + +1. Create a `CREATE TABLE` Job. + + The corresponding DDL Job is generated by parsing the `CREATE TABLE` statement. + +2. Execute the `CREATE TABLE` job. + + The TiDB node that receives the `CREATE TABLE` statement executes it directly, and then persists the table structure to TiKV. At the same time, the `CREATE TABLE` job is marked as completed and inserted into the `mysql.tidb_ddl_history` table. + +3. Synchronize the table information. + + TiDB notifies other nodes to synchronize the newly created table structure. diff --git a/basic-features.md b/basic-features.md index ed54778107a00..0821d194c9a63 100644 --- a/basic-features.md +++ b/basic-features.md @@ -132,7 +132,7 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u | [Metadata lock](/metadata-lock.md) | Y | Y | Y | Y | N | N | N | N | N | N | N | | [`FLASHBACK CLUSTER`](/sql-statements/sql-statement-flashback-cluster.md) | Y | Y | Y | Y | N | N | N | N | N | N | N | | [Pause](/sql-statements/sql-statement-admin-pause-ddl.md)/[Resume](/sql-statements/sql-statement-admin-resume-ddl.md) DDL | Y | Y | N | N | N | N | N | N | N | N | N | -| [TiDB DDL V2](/ddl-v2.md) | E | N | N | N | N | N | N | N | N | N | N | +| [TiDB Accelerated Table Creation](/accelerated-table-creation.md) | N | N | N | N | N | N | N | N | N | N | N | ## Transactions diff --git a/ddl-v2.md b/ddl-v2.md deleted file mode 100644 index 59db024d00c94..0000000000000 --- a/ddl-v2.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -title: Use TiDB DDL V2 to Accelerate Table Creation -summary: Learn the concept, principles, and implementation details of TiDB DDL V2 for acceleration table creation. ---- - -# Use TiDB DDL V2 to Accelerate Table Creation - -Starting from v7.6.0, the new version V2 of TiDB DDL supports creating tables quickly, which improves the efficiency of bulk table creation. - -TiDB uses the online asynchronous schema change algorithm to change the metadata. All DDL jobs are submitted to the `mysql.tidb_ddl_job` table, and the owner node pulls the DDL job to execute. After executing each phase of the online DDL algorithm, the DDL job is marked as completed and moved to the `mysql.tidb_ddl_history` table. Therefore, DDL statements can only be executed on the owner node and cannot be linearly extended. - -However, for some DDL statements, it is not necessary to strictly follow the online DDL algorithm. For example, the `CREATE TABLE` statement only has two states for the job: `none` and `public`. Therefore, TiDB can simplify the execution process of DDL, and executes the `CREATE TABLE` statement on a non-owner node to accelerate table creation. - -> **Warning:** -> -> This feature is currently an experimental feature and it is not recommended to use in a production environment. This feature might change or be removed without prior notice. If you find a bug, please give feedback by raising an [issue](https://github.com/pingcap/tidb/issues) on GitHub. - -## Compatibility with TiDB tools - -- [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) does not support replicating the tables that are created by TiDB DDL V2. - -## Limitation - -You can now use TiDB DDL V2 only in the [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) statement, and this statement must not include any foreign key constraints. - -## Use TiDB DDL V2 - -You can enable or disable TiDB DDL V2 by specifying the value of the system variable [`tidb_ddl_version`](/system-variables.md#tidb_ddl_version-new-in-v760) . - -To enable TiDB DDL V2, set the value of this variable to `2`: - -```sql -SET GLOBAL tidb_ddl_version = 2; -``` - -To disable TiDB DDL V2, set the value of this variable to `1`: - -```sql -SET GLOBAL tidb_ddl_version = 1; -``` - -## Implementation principle - -The detailed implementation principle of TiDB DDL V2 for accelerating table creation is as follows: - -1. Create a `CREATE TABLE` Job. - - This step is the same as that of the V1 implementation. The corresponding DDL Job is generated by parsing the `CREATE TABLE` statement. - -2. Execute the `CREATE TABLE` job. - - Different from V1, in TiDB DDL V2, the TiDB node that receives the `CREATE TABLE` statement executes it directly, and then persists the table structure to TiKV. At the same time, the `CREATE TABLE` job is marked as completed and inserted into the `mysql.tidb_ddl_history` table. - -3. Synchronize the table information. - - TiDB notifies other nodes to synchronize the newly created table structure. diff --git a/releases/release-7.6.0.md b/releases/release-7.6.0.md index 0cac1432d7e3f..b5bee4048eb99 100644 --- a/releases/release-7.6.0.md +++ b/releases/release-7.6.0.md @@ -111,9 +111,9 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.6/quick-start-with- * Improve the performance of creating tables by 10 times (experimental) [#49752](https://github.com/pingcap/tidb/issues/49752) @[gmhdbjd](https://github.com/gmhdbjd) - In previous versions, when migrating tens of thousands of tables from the upstream database to TiDB, it is time-consuming and inefficient for TiDB to create these tables. Starting from v7.6.0, TiDB introduces a new TiDB DDL V2 architecture. You can enable it by configuring the system variable [`tidb_ddl_version`](/system-variables.md#tidb_ddl_version-new-in-v760). Compared with previous versions, the new version of the DDL improves the performance of creating batch tables by 10 times, and significantly reduces time for creating tables. + In previous versions, when migrating tens of thousands of tables from the upstream database to TiDB, it is time-consuming and inefficient for TiDB to create these tables. Starting from v7.6.0, TiDB introduces a new TiDB DDL V2 architecture. You can enable it by configuring the system variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760). Compared with previous versions, the new version of the DDL improves the performance of creating batch tables by 10 times, and significantly reduces time for creating tables. - For more information, see [documentation](/ddl-v2.md). + For more information, see [documentation](https://docs.pingcap.com/tidb/v7.6/ddl-v2). * Support periodic full compaction (experimental) [#12729](https://github.com/tikv/tikv/issues/12729) [afeinberg](https://github.com/afeinberg) @@ -269,7 +269,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.6/quick-start-with- | [`tidb_auto_analyze_partition_batch_size`](/system-variables.md#tidb_auto_analyze_partition_batch_size-new-in-v640) | Modified | Changes the default value from `1` to `128` after further tests. | | [`tidb_sysproc_scan_concurrency`](/system-variables.md#tidb_sysproc_scan_concurrency-new-in-v650) | Modified | In a large-scale cluster, the concurrency of `scan` operations can be adjusted higher to meet the needs of `ANALYZE`. Therefore, change the maximum value from `256` to `4294967295`. | | [`tidb_analyze_distsql_scan_concurrency`](/system-variables.md#tidb_analyze_distsql_scan_concurrency-new-in-v760) | Newly added | Sets the concurrency of the `scan` operation when executing the `ANALYZE` operation. The default value is `4`. | -| [`tidb_ddl_version`](/system-variables.md#tidb_ddl_version-new-in-v760) | Newly added | Controls whether to enable [TiDB DDL V2](/ddl-v2.md). Set the value to `2` to enable it and `1` to disable it. The default value is `1`. When TiDB DDL V2 is enabled, DDL statements will be executed using TiDB DDL V2. The execution speed of DDL statements for creating tables is increased by 10 times compared with TiDB DDL V1. | +| [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760) | Newly added | Controls whether to enable [TiDB DDL V2](https://docs.pingcap.com/tidb/v7.6/ddl-v2). Set the value to `2` to enable it and `1` to disable it. The default value is `1`. When TiDB DDL V2 is enabled, DDL statements will be executed using TiDB DDL V2. The execution speed of DDL statements for creating tables is increased by 10 times compared with TiDB DDL V1. | | [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-new-in-v760) | Newly added | Controls whether to support creating `Global indexes` for partitioned tables. The default value is `OFF`. `Global index` is currently in the development stage. **It is not recommended to modify the value of this system variable**. | | [`tidb_idle_transaction_timeout`](/system-variables.md#tidb_idle_transaction_timeout-new-in-v760) | Newly added | Controls the idle timeout for transactions in a user session. When a user session is in a transactional state and remains idle for a duration exceeding the value of this variable, TiDB will terminate the session. The default value `0` means unlimited. | | [`tidb_opt_enable_fuzzy_binding`](/system-variables.md#tidb_opt_enable_fuzzy_binding-new-in-v760) | Newly added | Controls whether to enable the cross-database binding feature. The default value `OFF` means cross-database binding is disabled. | diff --git a/system-variables.md b/system-variables.md index 615d8523d7c4c..c144c397e3567 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1712,7 +1712,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Unit: Threads - This variable is used to set the concurrency of the DDL operation in the `re-organize` phase. -### `tidb_ddl_version` New in v7.6.0 +### `tidb_enable_fast_create_table` New in v8.0.0 > **Warning:** > @@ -1721,10 +1721,11 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Scope: GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Default value: `1` -- Value range: `1` or `2` -- This variable is used to control whether to enable [TiDB DDL V2](/ddl-v2.md). Setting it to `2` enables this feature, and setting it to `1` disables it. The default value is `1`. After you enable it, TiDB uses TiDB DDL V2 to execute DDL statements. -- Starting from v7.6.0, TiDB only supports accelerating table creation by the [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) statement. +- Type: Boolean +- Default value: `OFF` +- This variable is used to control whether to enable [TiDB Accelerated Table Creation](/accelerated-table-creation.md). +- Starting from v8.0.0, TiDB supports accelerating table creation by the [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) statement using `tidb_enable_fast_create_table`. +- This variable is renamed from the variable [`tidb_ddl_version`](https://docs.pingcap.com/tidb/v7.6/system-variables#tidb_ddl_version-new-in-v760) that is introduced in v7.6.0. Starting from v8.0.0, `tidb_ddl_version` no longer takes effect. ### tidb_default_string_match_selectivity New in v6.2.0