Deprecate config item max-memory and add items `server-memory-quota…

…` and `memory-usage-alarm-ratio` (pingcap#4333)

* add tidb memory configuration item docs

* Apply suggestions from code review

Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com>

* Update tidb-configuration-file.md

Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com>

* Update tidb-configuration-file.md

Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com>

configure-memory-usage.md

@@ -59,3 +59,62 @@ set @@tidb_mem_quota_query = 8 << 20;
 -- Set the threshold value of memory quota for a single SQL query to 8KB:
 set @@tidb_mem_quota_query = 8 << 10;
 ```
+
+## Configure the memory usage threshold of a tidb-server instance
+
+In the TiDB configuration file, you can set the memory usage threshold of a tidb-server instance by configuring [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota).
+
+The following example sets the total memory usage of a tidb-server instance to 32 GB:
+
+{{< copyable "" >}}
+
+```toml
+[performance]
+server-memory-quota = 34359738368
+```
+
+In this configuration, when the memory usage of a tidb-server instance reaches 32 GB, the instance starts to kill running SQL statements randomly until the memory usage drops below 32 GB. SQL operations that are forced to terminate return an `Out Of Global Memory Limit!` error message to the client.
+
+> **Warning:**
+>
+> + `server-memory-quota` is still an experimental feature. It is **NOT** recommended that you use it in a production environment.
+> + The default value of `server-memory-quota` is `0`, which means no memory limit.
+
+## Trigger the alarm of excessive memory usage
+
+In the default configuration, a tidb-server instance prints an alarm log and records associated status files when the machine memory usage reaches 80% of its total memory. You can set the memory usage ratio threshold by configuring `memory-usage-alarm-ratio`. For detailed alarm rules, refer to the description of [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio).
+
+Note that after the alarm is triggered once, it will be triggered again only if the memory usage rate has been below the threshold for more than ten seconds and reaches the threshold again. In addition, to avoid storing excessive status files generated by alarms, currently, TiDB only retains the status files generated during the recent five alarms.
+
+The following example constructs a memory-intensive SQL statement that triggers the alarm:
+
+1. Set `memory-usage-alarm-ratio` to `0.8`:
+
+    {{< copyable "" >}}
+
+    ```toml
+    mem-quota-query = 34359738368  // Increases the memory limit of each query to construct SQL statements that take up larger memory.
+    [performance]
+    memory-usage-alarm-ratio = 0.8
+    ```
+
+2. Execute `CREATE TABLE t(a int);` and insert 1000 rows of data.
+
+3. Execute `select * from t t1 join t t1 join t t3 order by t1.a`. This SQL statement outputs one billion records, which consumes a large amount of memory and therefore triggers the alarm.
+
+4. Check the `tidb.log` file which records the total system memory, current system memory usage, memory usage of the tidb-server instance, and the directory of status files.
+
+    ```
+    [2020/11/30 15:25:17.252 +08:00] [WARN] [memory_usage_alarm.go:141] ["tidb-server has the risk of OOM. Running SQLs and heap profile will be recorded in record path"] ["is server-memory-quota set"=false] ["system memory total"=33682427904] ["system memory usage"=27142864896] ["tidb-server memory usage"=22417922896] [memory-usage-alarm-ratio=0.8] ["record path"="/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record"]
+    ```
+
+    The fields of the example log file above are described as follows:
+
+    * `is server-memory-quota set` indicates whether [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) is set.
+    * `system memory total` indicates the total memory of the current system.
+    * `system memory usage` indicates the current system memory usage.
+    * `tidb-server memory usage` indicates the memory usage of the tidb-server instance.
+    * `memory-usage-alarm-ratio` indicates the value of [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio).
+    * `record path` indicates the directory of status files.
+
+5. You can see a set of files in the directory of status files (In the above example, the directory is `/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage/record`), including `goroutinue`, `heap`, and `running_sql`. These three files are suffixed with the time when status files are logged. They respectively record goroutine stack information, the usage status of heap memory, and the running SQL information when the alarm is triggered. For the format of log content in `running_sql`, refer to [`expensive-queries`](/identify-expensive-queries.md).

system-variables.md

@@ -879,3 +879,10 @@ explain select * from t where age=5;
 +-------------------------------+------------+-----------+-----------------------------+-------------------------------+
 3 rows in set (0.00 sec)
 ```
+
+### `tidb_memory_usage_alarm_ratio`
+
+- Scope: SESSION
+- Default value: 0.8
+- TiDB triggers an alarm when the percentage of the memory it takes exceeds a certain threshold. For the detailed usage description of this feature, see [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio).
+- You can set the initial value of this variable by configuring [`memory-usage-alarm-ratio`](/tidb-configuration-file.md#memory-usage-alarm-ratio).

tidb-configuration-file.md

@@ -286,11 +286,22 @@ Configuration items related to performance.
 - Default value: `0`
 - The default `0` indicates using all the CPUs on the machine. You can also set it to n, and then TiDB uses n CPUs.
 
-### `max-memory`
+### `server-memory-quota`
 
-- The maximum memory limit for the Prepared Least Recently Used (LRU) caching. If this value exceeds `performance.max-memory * (1 - prepared-plan-cache.memory-guard-ratio)`, the elements in the LRU are removed.
-- Default value: `0`
-- This configuration takes effect when `prepared-plan-cache.enabled` is `true` (default). When the size of the LRU is greater than `prepared-plan-cache.capacity`, the elements in the LRU are also removed.
+> **Warning:**
+>
+> `server-memory-quota` is still an experimental feature. It is **NOT** recommended that you use it in a production environment.
+
++ The memory usage limit of tidb-server instances. <!-- New in TiDB v5.0 --> This configuration item completely supersedes the previous [`max-memory`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file#max-memory).
++ Default value: `0` (in bytes), which means no memory limit.
+
+### `memory-usage-alarm-ratio`
+
++ TiDB triggers an alarm when the memory usage of tidb-server instance exceeds a certain threshold. The valid value for this configuration item ranges from `0` to `1`. If it is configured as `0` or `1`, this alarm feature is disabled.
++ Default value: `0.8`
++ When the memory usage alarm is enabled, if [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) is not set, then the threshold of memory usage is ```the `memory-usage-alarm-ratio` value * the system memory size```; if [`server-memory-quota`](/tidb-configuration-file.md#server-memory-quota) is set to a value greater than 0, then the threshold of memory usage is ```the `memory-usage-alarm-ratio` value * the `server-memory-quota` value```.
++ When TiDB detects that the memory usage of the tidb-server instance exceeds the threshold, it considers that there might be a risk of OOM. Therefore, it records ten SQL statements with the highest memory usage, ten SQL statements with the longest running time, and the heap profile among all SQL statements currently being executed to the directory [`tmp-storage-path/record`](/tidb-configuration-file.md#tmp-storage-path) and outputs a log containing the keyword `tidb-server has the risk of OOM`.
++ The value of this configuration item is the initial value of the system variable [`tidb_memory_usage_alarm_ratio`](/system-variables.md#tidb_memory_usage_alarm_ratio).
 
 ### `stmt-count-limit`