update docs related to partition table dynamic mode (pingcap#5810)

ti-chi-bot · Jun 21, 2021 · 3098559 · 3098559
1 parent 5f4c3e9
commit 3098559
Show file tree

Hide file tree

Showing 3 changed files with 203 additions and 0 deletions.
diff --git a/experimental-features.md b/experimental-features.md
@@ -24,6 +24,7 @@ This document introduces the experimental features of TiDB in different versions
 
 + List Partition (Introduced in v5.0)
 + List COLUMNS Partition (Introduced in v5.0)
++ [Dynamic Mode for Partitioned Tables](/partitioned-table.md#dynamic-mode). (Introduced in v5.1)
 + The expression index feature. The expression index is also called the function-based index. When you create an index, the index fields do not have to be a specific column but can be an expression calculated from one or more columns. This feature is useful for quickly accessing the calculation-based tables. See [Expression index](/sql-statements/sql-statement-create-index.md) for details. (Introduced in v4.0)
 + [Generated Columns](/generated-columns.md).
 + [User-Defined Variables](/user-defined-variables.md).

diff --git a/partitioned-table.md b/partitioned-table.md
@@ -1240,3 +1240,195 @@ select * from t;
 The `tidb_enable_list_partition` environment variable controls whether to enable the partitioned table feature. If this variable is set to `OFF`, the partition information will be ignored when a table is created, and this table will be created as a normal table.
 
 This variable is only used in table creation. After the table is created, modify this variable value takes no effect. For details, see [system variables](/system-variables.md#tidb_enable_list_partition-new-in-v50).
+
+### Dynamic mode
+
+> **Warning:**
+>
+> This is still an experimental feature. It is **NOT** recommended that you use it in the production environment.
+
+TiDB accesses partitioned tables in one of the two modes: `dynamic` mode and `static` mode. Currently, `static` mode is used by default. If you want to enable `dynamic` mode, you need to manually set the `tidb_partition_prune_mode` variable to `dynamic`.
+
+{{< copyable "sql" >}}
+
+```sql
+set @@session.tidb_partition_prune_mode = 'dynamic'
+```
+
+In `static` mode, TiDB accesses each partition separately using multiple operators, and then merges the results using `Union`. The following example is a simple read operation where TiDB merges the results of two corresponding partitions using `Union`:
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> create table t1(id int, age int, key(id)) partition by range(id) (
+    ->     partition p0 values less than (100),
+    ->     partition p1 values less than (200),
+    ->     partition p2 values less than (300),
+    ->     partition p3 values less than (400));
+Query OK, 0 rows affected (0.01 sec)
+
+mysql> explain select * from t1 where id < 150;
++------------------------------+----------+-----------+------------------------+--------------------------------+
+| id                           | estRows  | task      | access object          | operator info                  |
++------------------------------+----------+-----------+------------------------+--------------------------------+
+| PartitionUnion_9             | 6646.67  | root      |                        |                                |
+| ├─TableReader_12             | 3323.33  | root      |                        | data:Selection_11              |
+| │ └─Selection_11             | 3323.33  | cop[tikv] |                        | lt(test.t1.id, 150)            |
+| │   └─TableFullScan_10       | 10000.00 | cop[tikv] | table:t1, partition:p0 | keep order:false, stats:pseudo |
+| └─TableReader_18             | 3323.33  | root      |                        | data:Selection_17              |
+|   └─Selection_17             | 3323.33  | cop[tikv] |                        | lt(test.t1.id, 150)            |
+|     └─TableFullScan_16       | 10000.00 | cop[tikv] | table:t1, partition:p1 | keep order:false, stats:pseudo |
++------------------------------+----------+-----------+------------------------+--------------------------------+
+7 rows in set (0.00 sec)
+```
+
+In `dynamic` mode, each operator supports direct access to multiple partitions, so TiDB no longer uses `Union`.
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> set @@session.tidb_partition_prune_mode = 'dynamic';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> explain select * from t1 where id < 150;
++-------------------------+----------+-----------+-----------------+--------------------------------+
+| id                      | estRows  | task      | access object   | operator info                  |
++-------------------------+----------+-----------+-----------------+--------------------------------+
+| TableReader_7           | 3323.33  | root      | partition:p0,p1 | data:Selection_6               |
+| └─Selection_6           | 3323.33  | cop[tikv] |                 | lt(test.t1.id, 150)            |
+|   └─TableFullScan_5     | 10000.00 | cop[tikv] | table:t1        | keep order:false, stats:pseudo |
++-------------------------+----------+-----------+-----------------+--------------------------------+
+3 rows in set (0.00 sec)
+```
+
+From the above query results, you can see that the `Union` operator in the execution plan disappears while the partition pruning still takes effect and the execution plan only accesses `p0` and `p1`.
+
+`dynamic` mode makes execution plans simpler and clearer. Omitting the Union operation can improve the execution efficiency and avoid the problem of Union concurrent execution. In addition, `dynamic` mode also solves two problems that cannot be solved in `static` mode:
+
++ Plan Cache cannot be used. (See example 1 and 2)
++ Execution plans with IndexJoin cannot be used. (See example 3 and 4)
+
+**Example 1**：In the following example, the Plan Cache feature is enabled in the configuration file and the same query is executed twice in `static` mode:
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> set @a=150;
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> set @@tidb_partition_prune_mode = 'static';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> prepare stmt from 'select * from t1 where id < ?';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> execute stmt using @a;
+Empty set (0.00 sec)
+
+mysql> execute stmt using @a;
+Empty set (0.00 sec)
+
+-- In static mode, when the same query is executed twice, the cache cannot be hit at the second time.
+mysql> select @@last_plan_from_cache;
++------------------------+
+| @@last_plan_from_cache |
++------------------------+
+|                      0 |
++------------------------+
+1 row in set (0.00 sec)
+```
+
+The `last_plan_from_cache` variable can show whether the last query hits the Plan Cache or not. From example 1, you can see that in `static` mode, even if the same query is executed multiple times on the partitioned table, the Plan Cache is not hit.
+
+**Example 2**: In the following example, the same operations are performed in `dynamic` mode as done in example 1:
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> set @@tidb_partition_prune_mode = 'dynamic';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> prepare stmt from 'select * from t1 where id < ?';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> execute stmt using @a;
+Empty set (0.00 sec)
+
+mysql> execute stmt using @a;
+Empty set (0.00 sec)
+
+-- In dynamic mode, the cache can be hit at the second time.
+mysql> select @@last_plan_from_cache;
++------------------------+
+| @@last_plan_from_cache |
++------------------------+
+|                      1 |
++------------------------+
+1 row in set (0.00 sec)
+```
+
+From example 2, you can see that in `dynamic` mode, querying the partitioned table hits the Plan Cache.
+
+**Example 3**: In the following example, a query is performed in `static` mode using the execution plan with IndexJoin:
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> create table t2(id int, code int);
+Query OK, 0 rows affected (0.01 sec)
+
+mysql> set @@tidb_partition_prune_mode = 'static';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> explain select /*+ TIDB_INLJ(t1, t2) */ t1.* from t1, t2 where t2.code = 0 and t2.id = t1.id;
++--------------------------------+----------+-----------+------------------------+------------------------------------------------+
+| id                             | estRows  | task      | access object          | operator info                                  |
++--------------------------------+----------+-----------+------------------------+------------------------------------------------+
+| HashJoin_13                    | 12.49    | root      |                        | inner join, equal:[eq(test.t1.id, test.t2.id)] |
+| ├─TableReader_42(Build)        | 9.99     | root      |                        | data:Selection_41                              |
+| │ └─Selection_41               | 9.99     | cop[tikv] |                        | eq(test.t2.code, 0), not(isnull(test.t2.id))   |
+| │   └─TableFullScan_40         | 10000.00 | cop[tikv] | table:t2               | keep order:false, stats:pseudo                 |
+| └─PartitionUnion_15(Probe)     | 39960.00 | root      |                        |                                                |
+|   ├─TableReader_18             | 9990.00  | root      |                        | data:Selection_17                              |
+|   │ └─Selection_17             | 9990.00  | cop[tikv] |                        | not(isnull(test.t1.id))                        |
+|   │   └─TableFullScan_16       | 10000.00 | cop[tikv] | table:t1, partition:p0 | keep order:false, stats:pseudo                 |
+|   ├─TableReader_24             | 9990.00  | root      |                        | data:Selection_23                              |
+|   │ └─Selection_23             | 9990.00  | cop[tikv] |                        | not(isnull(test.t1.id))                        |
+|   │   └─TableFullScan_22       | 10000.00 | cop[tikv] | table:t1, partition:p1 | keep order:false, stats:pseudo                 |
+|   ├─TableReader_30             | 9990.00  | root      |                        | data:Selection_29                              |
+|   │ └─Selection_29             | 9990.00  | cop[tikv] |                        | not(isnull(test.t1.id))                        |
+|   │   └─TableFullScan_28       | 10000.00 | cop[tikv] | table:t1, partition:p2 | keep order:false, stats:pseudo                 |
+|   └─TableReader_36             | 9990.00  | root      |                        | data:Selection_35                              |
+|     └─Selection_35             | 9990.00  | cop[tikv] |                        | not(isnull(test.t1.id))                        |
+|       └─TableFullScan_34       | 10000.00 | cop[tikv] | table:t1, partition:p3 | keep order:false, stats:pseudo                 |
++--------------------------------+----------+-----------+------------------------+------------------------------------------------+
+17 rows in set, 1 warning (0.00 sec)
+```
+
+From example 3, you can see that even if the `TIDB_INLJ` hint is used, the query on the partitioned table cannot select the execution plan with IndexJoin.
+
+**Example 4**: In the following example, the query is performed in `dynamic` mode using the execution plan with IndexJoin:
+
+{{< copyable "sql" >}}
+
+```sql
+mysql> set @@tidb_partition_prune_mode = 'dynamic';
+Query OK, 0 rows affected (0.00 sec)
+
+mysql> explain select /*+ TIDB_INLJ(t1, t2) */ t1.* from t1, t2 where t2.code = 0 and t2.id = t1.id;
++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+
+| id                              | estRows  | task      | access object          | operator info                                                                                                       |
++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+
+| IndexJoin_11                    | 12.49    | root      |                        | inner join, inner:IndexLookUp_10, outer key:test.t2.id, inner key:test.t1.id, equal cond:eq(test.t2.id, test.t1.id) |
+| ├─TableReader_16(Build)         | 9.99     | root      |                        | data:Selection_15                                                                                                   |
+| │ └─Selection_15                | 9.99     | cop[tikv] |                        | eq(test.t2.code, 0), not(isnull(test.t2.id))                                                                        |
+| │   └─TableFullScan_14          | 10000.00 | cop[tikv] | table:t2               | keep order:false, stats:pseudo                                                                                      |
+| └─IndexLookUp_10(Probe)         | 1.25     | root      | partition:all          |                                                                                                                     |
+|   ├─Selection_9(Build)          | 1.25     | cop[tikv] |                        | not(isnull(test.t1.id))                                                                                             |
+|   │ └─IndexRangeScan_7          | 1.25     | cop[tikv] | table:t1, index:id(id) | range: decided by [eq(test.t1.id, test.t2.id)], keep order:false, stats:pseudo                                      |
+|   └─TableRowIDScan_8(Probe)     | 1.25     | cop[tikv] | table:t1               | keep order:false, stats:pseudo                                                                                      |
++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+
+8 rows in set (0.00 sec)
+```
+
+From example 4, you can see that in `dynamic` mode, the execution plan with IndexJoin is selected when you execute the query.
diff --git a/system-variables.md b/system-variables.md
@@ -501,6 +501,16 @@ Constraint checking is always performed in place for pessimistic transactions (d
 - Default value: `OFF`
 - This variable is used to set whether to enable the `LIST (COLUMNS) TABLE PARTITION` feature.
 
+### `tidb_partition_prune_mode` <span class="version-mark">New in v5.1</span>
+
+> **Warning:**
+>
+> Currently, the dynamic mode for partitioned tables is an experimental feature. It is not recommended that you use it in the production environment.
+
+- Scope: SESSION | GLOBAL
+- Default value: `static`
+- Specifies whether to enable `dynamic` mode for partitioned tables. For details about the dynamic mode, see [Dynamic Mode for Partitioned Tables](/partitioned-table.md#dynamic-mode).
+
 ### tidb_enable_noop_functions <span class="version-mark">New in v4.0</span>
 
 - Scope: SESSION | GLOBAL