diff --git a/docs/cn/guides/51-access-data-lake/02-iceberg.md b/docs/cn/guides/51-access-data-lake/02-iceberg.md index cbee5324b0..b3ddc4a3b6 100644 --- a/docs/cn/guides/51-access-data-lake/02-iceberg.md +++ b/docs/cn/guides/51-access-data-lake/02-iceberg.md @@ -1,19 +1,19 @@ --- -title: Apache Iceberg +title: Apache Iceberg™ --- import FunctionDescription from '@site/src/components/FunctionDescription'; -Databend 支持集成 [Apache Iceberg](https://iceberg.apache.org/) catalog,从而增强了其数据管理和分析的兼容性和多功能性。通过将 Apache Iceberg 强大的元数据和存储管理功能无缝集成到平台中,扩展了 Databend 的功能。 +Databend 支持集成 [Apache Iceberg™](https://iceberg.apache.org/) catalog,从而增强了其数据管理和分析的兼容性和多功能性。通过将 Apache Iceberg™ 强大的元数据和存储管理功能无缝集成到平台中,扩展了 Databend 的功能。 -## Apache Iceberg 快速入门 +## Iceberg 快速入门 -如果您想快速试用 Apache Iceberg 并在本地进行表操作实验,可以使用 [基于 Docker 的入门项目](https://github.com/databendlabs/iceberg-quick-start)。此设置允许您: +如果您想快速试用 Iceberg 并在本地进行表操作实验,可以使用基于 [Docker 的入门项目](https://github.com/databendlabs/iceberg-quick-start)。此设置允许您: - 运行支持 Iceberg 的 Spark - 使用 REST catalog (Iceberg REST Fixture) -- 使用 MinIO 模拟 S3 兼容的对象存储 +- 使用 MinIO 模拟与 S3 兼容的对象存储 - 将示例 TPC-H 数据加载到 Iceberg 表中以进行查询测试 ### 前提条件 @@ -32,7 +32,7 @@ docker compose up -d - `spark-iceberg`: 带有 Iceberg 的 Spark 3.4 - `rest`: Iceberg REST Catalog -- `minio`: S3 兼容的对象存储 +- `minio`: 与 S3 兼容的对象存储 - `mc`: MinIO 客户端(用于设置存储桶) ```bash @@ -205,10 +205,10 @@ ORDER BY ## 数据类型映射 -下表映射了 Apache Iceberg 和 Databend 之间的数据类型。请注意,Databend 目前不支持表中未列出的 Iceberg 数据类型。 +下表映射了 Apache Iceberg™ 和 Databend 之间的数据类型。请注意,Databend 目前不支持表中未列出的 Iceberg 数据类型。 -| Apache Iceberg | Databend | +| Apache Iceberg™ | Databend | | ------------------------------- | ----------------------- | | BOOLEAN | [BOOLEAN](/sql/sql-reference/data-types/boolean) | | INT | [INT32](/sql/sql-reference/data-types/numeric#integer-data-types) | @@ -255,32 +255,32 @@ CONNECTION=( | 参数 | 是否必须? | 描述 | |------------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `` | 是 | 您要创建的 catalog 的名称。 | -| `TYPE` | 是 | 指定 catalog 类型。对于 Iceberg,设置为 `ICEBERG`。 | +| `TYPE` | 是 | 指定 catalog 类型。对于 Apache Iceberg™,设置为 `ICEBERG`。 | | `CONNECTION` | 是 | Iceberg catalog 的连接参数。 | -| `TYPE` (在 `CONNECTION` 中) | 是 | 连接类型。对于 Iceberg,通常设置为 `rest` 以进行基于 REST 的连接。 | +| `TYPE` (在 `CONNECTION` 中) | 是 | 连接类型。对于 Iceberg,通常设置为 `rest` 以进行基于 REST 的连接。 | | `ADDRESS` | 是 | Iceberg 服务的地址或 URL(例如,`http://127.0.0.1:8181`)。 | -| `WAREHOUSE` | 是 | Iceberg 计算集群的位置,通常是 S3 bucket 或兼容的对象存储系统。 | +| `WAREHOUSE` | 是 | Iceberg warehouse 的位置,通常是 S3 bucket 或兼容的对象存储系统。 | | `` | 是 | 用于建立与外部存储连接的连接参数。所需的参数因特定的存储服务和身份验证方法而异。 有关可用参数的完整列表,请参见下表。 | -| 连接参数 | 描述 | -|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------| -| `s3.endpoint` | S3 endpoint. | -| `s3.access-key-id` | S3 access key ID. | -| `s3.secret-access-key` | S3 secret access key. | -| `s3.session-token` | S3 session token,使用临时凭证时需要。 | -| `s3.region` | S3 区域。 | -| `client.region` | 用于 S3 客户端的区域,优先于 `s3.region`。 | -| `s3.path-style-access` | S3 Path Style Access. | -| `s3.sse.type` | S3 服务器端加密 (SSE) 类型。 | -| `s3.sse.key` | S3 SSE 密钥。如果加密类型为 `kms`,则为 KMS 密钥 ID。如果加密类型为 `custom`,则为 base-64 AES256 对称密钥。 | -| `s3.sse.md5` | S3 SSE MD5 校验和。 | -| `client.assume-role.arn` | 要承担的 IAM 角色的 ARN,而不是使用默认凭证链。 | -| `client.assume-role.external-id` | 用于承担 IAM 角色的可选外部 ID。 | -| `client.assume-role.session-name` | 用于承担 IAM 角色的可选会话名称。 | -| `s3.allow-anonymous` | 允许匿名访问的选项(例如,对于公共存储桶/文件夹)。 | -| `s3.disable-ec2-metadata` | 用于禁用从 EC2 元数据加载凭证的选项(通常与 `s3.allow-anonymous` 一起使用)。 | -| `s3.disable-config-load` | 用于禁用从配置文件和环境变量加载配置的选项。 | +| 连接参数 | 描述 | +|------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| +| `s3.endpoint` | S3 端点。 | +| `s3.access-key-id` | S3 访问密钥 ID。 | +| `s3.secret-access-key` | S3 密钥。 | +| `s3.session-token` | S3 会话令牌,使用临时凭证时需要。 | +| `s3.region` | S3 区域。 | +| `client.region` | 用于 S3 客户端的区域,优先于 `s3.region`。 | +| `s3.path-style-access` | S3 Path Style Access。 | +| `s3.sse.type` | S3 服务器端加密 (SSE) 类型。 | +| `s3.sse.key` | S3 SSE 密钥。如果加密类型为 `kms`,则为 KMS 密钥 ID。如果加密类型为 `custom`,则为 base-64 AES256 对称密钥。 | +| `s3.sse.md5` | S3 SSE MD5 校验和。 | +| `client.assume-role.arn` | 要承担的 IAM 角色的 ARN,而不是使用默认凭证链。 | +| `client.assume-role.external-id` | 用于承担 IAM 角色的可选外部 ID。 | +| `client.assume-role.session-name` | 用于承担 IAM 角色的可选会话名称。 | +| `s3.allow-anonymous` | 允许匿名访问的选项(例如,对于公共存储桶/文件夹)。 | +| `s3.disable-ec2-metadata` | 用于禁用从 EC2 元数据加载凭证的选项(通常与 `s3.allow-anonymous` 一起使用)。 | +| `s3.disable-config-load` | 用于禁用从配置文件和环境变量加载配置的选项。 | ### SHOW CREATE CATALOG @@ -312,11 +312,11 @@ SHOW CATALOGS [LIKE ''] USE CATALOG ``` -## 缓存 Iceberg Catalog +## Caching Iceberg Catalog Databend 提供了一个专门为 Iceberg catalogs 设计的 Catalog Metadata Cache。当第一次在 Iceberg 表上执行查询时,元数据会被缓存在内存中。默认情况下,此缓存保持有效 10 分钟,之后会异步刷新。这确保了对 Iceberg 表的查询更快,避免了重复的元数据检索。 -如果需要最新的元数据,可以使用以下命令手动刷新缓存: +如果您需要最新的元数据,可以使用以下命令手动刷新缓存: ```sql USE CATALOG iceberg; @@ -324,7 +324,7 @@ ALTER DATABASE tpch REFRESH CACHE; -- 刷新 tpch 数据库的元数据缓存 ALTER TABLE tpch.lineitem REFRESH CACHE; -- 刷新 lineitem 表的元数据缓存 ``` -如果不想使用元数据缓存,可以通过在 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 配置文件中将 `iceberg_table_meta_count` 设置为 `0` 来完全禁用它: +如果您不想使用元数据缓存,可以通过在 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 配置文件中将 `iceberg_table_meta_count` 设置为 `0` 来完全禁用它: ```toml ... @@ -335,9 +335,9 @@ iceberg_table_meta_count = 0 ... ``` -除了元数据缓存外,Databend 还支持 Iceberg catalog 表的表数据缓存,类似于 Fuse 表。有关数据缓存的更多信息,请参阅 [Query Configurations](../10-deploy/04-references/02-node-config/02-query-config.md) 参考中的 `[cache] Section`。 +除了元数据缓存之外,Databend 还支持 Iceberg catalog 表的表数据缓存,类似于 Fuse 表。有关数据缓存的更多信息,请参阅 [Query Configurations](../10-deploy/04-references/02-node-config/02-query-config.md) 参考中的 `[cache] Section`。 -## Iceberg Table Functions +## Apache Iceberg™ Table Functions Databend 提供了以下表函数来查询 Iceberg 元数据,允许用户有效地检查快照和清单: @@ -346,7 +346,7 @@ Databend 提供了以下表函数来查询 Iceberg 元数据,允许用户有 ## 使用示例 -此示例展示了如何使用基于 REST 的连接创建 Iceberg catalog,指定服务地址、计算集群位置 (S3) 和可选参数,如 AWS 区域和自定义 endpoint: +此示例展示了如何使用基于 REST 的连接创建 Iceberg catalog,指定服务地址、计算集群位置 (S3) 以及可选参数(如 AWS 区域和自定义端点): ```sql CREATE CATALOG ctl diff --git a/docs/cn/guides/54-query/03-udf.md b/docs/cn/guides/54-query/03-udf.md index 6cca55df0e..11b62a3433 100644 --- a/docs/cn/guides/54-query/03-udf.md +++ b/docs/cn/guides/54-query/03-udf.md @@ -8,51 +8,66 @@ import EEFeature from '@site/src/components/EEFeature'; -用户自定义函数 (UDFs) 通过支持匿名 lambda 表达式和预定义的处理程序(Python、JavaScript 和 WebAssembly)来定义 UDF,从而提供增强的灵活性。这些功能允许用户创建根据其特定数据处理需求量身定制的自定义操作。Databend UDF 分为以下类型: +用户自定义函数(UDF)通过支持匿名 lambda 表达式和预定义的处理程序(Python、JavaScript 和 WebAssembly)来定义 UDF,从而提供增强的灵活性。这些功能允许用户创建根据其特定数据处理需求量身定制的自定义操作。Databend UDF 分为以下类型: - [Lambda UDFs](#lambda-udfs) -- [嵌入式 UDFs](#embedded-udfs) +- [Embedded UDFs](#embedded-udfs) ## Lambda UDFs -lambda UDF 允许用户直接在其查询中使用匿名函数(lambda 表达式)来定义自定义操作。这些 lambda 表达式通常简洁明了,可用于执行仅使用内置函数可能无法实现的特定数据转换或计算。 +lambda UDF 允许用户直接在其查询中使用匿名函数(lambda 表达式)定义自定义操作。这些 lambda 表达式通常简洁明了,可用于执行特定的数据转换或计算,而这些转换或计算可能无法仅使用内置函数来实现。 ### 使用示例 此示例创建 UDF,以使用 SQL 查询从表中的 JSON 数据中提取特定值。 ```sql --- 定义 UDF -CREATE FUNCTION get_v1 AS (input_json) -> input_json['v1']; -CREATE FUNCTION get_v2 AS (input_json) -> input_json['v2']; +CREATE OR REPLACE TABLE sale_items ( + item_id INT, + details VARIANT +); + +INSERT INTO sale_items VALUES + (1, PARSE_JSON('{"name": "T-Shirt", "price": 20.00, "discount_pct": 10}')), -- 10% discount + (2, PARSE_JSON('{"name": "Jeans", "price": 50.00, "discount_pct": 25}')), -- 25% discount + (3, PARSE_JSON('{"name": "Jacket", "price": 100.00, "discount_pct": 0}')), -- No discount + (4, PARSE_JSON('{"name": "Socks", "price": 5.00, "discount_pct": 50}')); -- 50% discount + +-- Define a Lambda UDF to calculate the final price after discount +-- WITH EXPLICIT CASTING +CREATE OR REPLACE FUNCTION calculate_final_price AS (item_info) -> + (item_info['price']::FLOAT) * (1 - (item_info['discount_pct']::FLOAT) / 100.0); SHOW USER FUNCTIONS; +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ +--| name | is_aggregate | description | arguments | language | created_on | +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ +--| calculate_final_price | 0 | | {"parameters":["item_info"]} | SQL | YYYY-MM-DD HH:MM:SS.ffffff | +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ -┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ name │ is_aggregate │ description │ arguments │ language │ created_on │ -├────────┼───────────────────┼─────────────┼───────────────────────────────┼──────────┼────────────────────────────┤ -│ get_v1 │ NULL │ │ {"parameters":["input_json"]} │ SQL │ 2024-11-18 23:20:28.432842 │ -│ get_v2 │ NULL │ │ {"parameters":["input_json"]} │ SQL │ 2024-11-18 23:21:46.838744 │ -└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ - --- 创建表 -CREATE TABLE json_table(time TIMESTAMP, data JSON); - --- 插入时间事件 -INSERT INTO json_table VALUES('2022-06-01 00:00:00.00000', PARSE_JSON('{"v1":1.5, "v2":20.5}')); - --- 从事件中获取 v1 和 v2 值 -SELECT get_v1(data), get_v2(data) FROM json_table; -+------------+------------+ -| data['v1'] | data['v2'] | -+------------+------------+ -| 1.5 | 20.5 | -+------------+------------+ +-- Use the Lambda UDF to get item names and their final prices +SELECT + item_id, + details['name']::STRING AS item_name, + details['price']::FLOAT AS original_price, + calculate_final_price(details) AS final_price +FROM sale_items +ORDER BY item_id; + +-- Expected output for the SELECT query (final_price should now have values): +--+---------+-----------+----------------+-------------+ +--| item_id | item_name | original_price | final_price | +--+---------+-----------+----------------+-------------+ +--| 1 | T-Shirt | 20.00 | 18.00 | +--| 2 | Jeans | 50.00 | 37.50 | +--| 3 | Jacket | 100.00 | 100.00 | +--| 4 | Socks | 5.00 | 2.50 | +--+---------+-----------+----------------+-------------+ ``` -## 嵌入式 UDFs +## Embedded UDFs -嵌入式 UDF 允许您在 SQL 中嵌入使用以下编程语言编写的代码: +通过嵌入式 UDF,您可以将使用以下编程语言编写的代码嵌入到 SQL 中: - [Python](#python) - [JavaScript](#javascript) @@ -62,12 +77,12 @@ SELECT get_v1(data), get_v2(data) FROM json_table; :::note - 尚不支持使用 WebAssembly 创建聚合 UDF。 -- 如果您的程序内容很大,您可以压缩它,然后将其传递到 Stage。有关 WebAssembly,请参见[使用示例](#usage-examples-2)。 +- 如果您的程序内容很大,您可以对其进行压缩,然后将其传递到 Stage。有关 WebAssembly,请参见[使用示例](#usage-examples-2)。 ::: ### Python (需要 Databend Enterprise) -Python UDF 允许您通过 Databend 的内置处理程序从 SQL 查询中调用 Python 代码,从而可以在 SQL 查询中无缝集成 Python 逻辑。 +通过 Python UDF,您可以经由 Databend 的内置处理程序从 SQL 查询调用 Python 代码,从而可以在 SQL 查询中无缝集成 Python 逻辑。 :::note Python UDF 必须仅使用 Python 的标准库;不允许第三方导入。 @@ -75,126 +90,117 @@ Python UDF 必须仅使用 Python 的标准库;不允许第三方导入。 #### 数据类型映射 -请参见开发者指南中的 [数据类型映射](/developer/drivers/python#data-type-mappings)。 +| Databend | Python | +| --------- | ----------------- | +| BOOLEAN | bool | +| TINYINT | int | +| SMALLINT | int | +| INT | int | +| BIGINT | int | +| FLOAT | float | +| DOUBLE | float | +| DECIMAL | decimal.Decimal | +| DATE | datetime.date | +| TIMESTAMP | datetime.datetime | +| VARCHAR | str | +| BINARY | bytes | +| ARRAY | list | +| TUPLE | tuple | +| MAP | dict | +| VARIANT | str | +| BITMAP | str | +| GEOMETRY | str | #### 使用示例 -此示例定义了一个用于情感分析的 Python UDF,创建了一个表,插入了示例数据,并对文本数据执行情感分析。 - -1. 定义一个名为 `sentiment_analysis` 的 Python UDF。 - -```sql --- 创建情感分析函数 -CREATE OR REPLACE FUNCTION sentiment_analysis(STRING) RETURNS STRING -LANGUAGE python HANDLER = 'sentiment_analysis_handler' -AS $$ -def remove_stop_words(text, stop_words): - """ - 从文本中删除常见的停用词。 - - Args: - text (str): 输入文本。 - stop_words (set): 要删除的停用词集。 - - Returns: - str: 删除停用词后的文本。 - """ - return ' '.join([word for word in text.split() if word.lower() not in stop_words]) - -def calculate_sentiment(text, positive_words, negative_words): - """ - 计算文本的情感得分。 - - Args: - text (str): 输入文本。 - positive_words (set): 一组积极词。 - negative_words (set): 一组消极词。 - - Returns: - int: 情感得分。 - """ - words = text.split() - score = sum(1 for word in words if word in positive_words) - sum(1 for word in words if word in negative_words) - return score - -def get_sentiment_label(score): - """ - 根据情感得分确定情感标签。 - - Args: - score (int): 情感得分。 - - Returns: - str: 情感标签(“积极”、“消极”、“中性”)。 - """ - if score > 0: - return 'Positive' - elif score < 0: - return 'Negative' - else: - return 'Neutral' - -def sentiment_analysis_handler(text): - """ - 分析输入文本的情感。 - - Args: - text (str): 输入文本。 - - Returns: - str: 情感分析结果,包括得分和标签。 - """ - stop_words = set(["a", "an", "the", "and", "or", "but", "if", "then", "so"]) - positive_words = set(["good", "happy", "joy", "excellent", "positive", "love"]) - negative_words = set(["bad", "sad", "pain", "terrible", "negative", "hate"]) - - clean_text = remove_stop_words(text, stop_words) - sentiment_score = calculate_sentiment(clean_text, positive_words, negative_words) - sentiment_label = get_sentiment_label(sentiment_score) - - return f'Sentiment Score: {sentiment_score}; Sentiment Label: {sentiment_label}' -$$; -``` - -2. 使用 `sentiment_analysis` 函数对文本数据执行情感分析。 - ```sql -CREATE OR REPLACE TABLE texts ( - original_text STRING +-- Create a table with user interaction logs +CREATE TABLE user_interaction_logs ( + log_id INT, + log_data VARIANT -- JSON interaction log ); --- 插入示例数据 -INSERT INTO texts (original_text) -VALUES -('The quick brown fox feels happy and joyful'), -('A hard journey, but it was painful and sad'), -('Uncertain outcomes leave everyone unsure and hesitant'), -('The movie was excellent and everyone loved it'), -('A terrible experience that made me feel bad'); - +-- Insert sample interaction log data +INSERT INTO user_interaction_logs VALUES + (1, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:00:00Z", "action": "view_product", "details": {"product_id": "p789", "category": "electronics", "price": 99.99}}')), + (2, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:05:10Z", "action": "add_to_cart", "details": {"product_id": "p789", "quantity": 1, "category": "electronics"}}')), + (3, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:02:30Z", "action": "search", "details": {"query": "wireless headphones", "results_count": 15}}')), + (4, PARSE_JSON('{"user_id": "u789", "timestamp": "2023-01-15T10:08:00Z", "action": "purchase", "details": {"order_id": "o555", "total_amount": 125.50, "item_count": 2}}')), + (5, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:10:00Z", "action": "view_page", "details": {"page_name": "homepage"}}')), + (6, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:12:00Z", "action": "purchase", "details": {"order_id": "o556", "total_amount": 25.00, "item_count": 1}}')); + +-- Create a Python UDF to extract features from interaction logs +CREATE OR REPLACE FUNCTION extract_interaction_features_py(VARIANT) +RETURNS VARCHAR +LANGUAGE python HANDLER = 'extract_features' +AS $$ +import json + +def extract_features(log): + log_dict = log if isinstance(log, dict) else {} + action = log_dict.get('action', '').lower() + details = log_dict.get('details', {}) + if not isinstance(details, dict): + details = {} + + is_search_action = False + has_product_interaction = False + product_category_if_any = None + search_query_length = 0 + purchase_value_bucket = None + + if action == 'search': + is_search_action = True + search_query_length = len(details.get('query', '')) + + if action in ['view_product', 'add_to_cart', 'remove_from_cart']: + has_product_interaction = True + product_category_if_any = details.get('category') + + if action == 'purchase': + has_product_interaction = True + + if action == 'purchase': + total_amount = details.get('total_amount', 0.0) + if not isinstance(total_amount, (int, float)): + total_amount = 0.0 + + if total_amount < 50: + purchase_value_bucket = 'Low' + elif total_amount < 200: + purchase_value_bucket = 'Medium' + else: + purchase_value_bucket = 'High' + + result_dict = { + "is_search_action": is_search_action, + "has_product_interaction": has_product_interaction, + "product_category_if_any": product_category_if_any, + "search_query_length": search_query_length, + "purchase_value_bucket": purchase_value_bucket + } + return json.dumps(result_dict) +$$; +-- Use the Python UDF to extract features SELECT - original_text, - sentiment_analysis(original_text) AS processed_text + log_id, + log_data['user_id']::STRING AS user_id, + log_data['action']::STRING AS action, + extract_interaction_features_py(log_data) AS extracted_features FROM - texts; - -| original_text | processed_text | -|----------------------------------------------------------|---------------------------------------------------| -| The quick brown fox feels happy and joyful | Sentiment Score: 1; Sentiment Label: Positive | -| A hard journey, but it was painful and sad | Sentiment Score: -1; Sentiment Label: Negative | -| Uncertain outcomes leave everyone unsure and hesitant | Sentiment Score: 0; Sentiment Label: Neutral | -| The movie was excellent and everyone loved it | Sentiment Score: 1; Sentiment Label: Positive | -| A terrible experience that made me feel bad | Sentiment Score: -2; Sentiment Label: Negative | + user_interaction_logs +ORDER BY + log_id; ``` + ### JavaScript -JavaScript UDF 允许您通过 Databend 的内置处理程序从 SQL 查询中调用 JavaScript 代码,从而可以在 SQL 查询中无缝集成 JavaScript 逻辑。 +通过 JavaScript UDF,您可以经由 Databend 的内置处理程序从 SQL 查询调用 JavaScript 代码,从而可以在 SQL 查询中无缝集成 JavaScript 逻辑。 #### 数据类型映射 -下表显示了 Databend 和 JavaScript 之间的类型映射: | Databend Type | JS Type | | ----------------- | ---------- | @@ -210,78 +216,94 @@ JavaScript UDF 允许您通过 Databend 的内置处理程序从 SQL 查询中 | BIGINT UNSIGNED | Number | | FLOAT | Number | | DOUBLE | Number | -| STRING | String | +| VARCHAR | String | | DATE / TIMESTAMP | Date | | DECIMAL | BigDecimal | | BINARY | Uint8Array | #### 使用示例 -此示例定义了一个名为 "gcd_js" 的 JavaScript UDF,用于计算两个整数的最大公约数 (GCD),并在 SQL 查询中应用它: - ```sql -CREATE FUNCTION gcd_js (INT, INT) RETURNS BIGINT LANGUAGE javascript HANDLER = 'gcd_js' AS $$ -export function gcd_js(a, b) { - while (b != 0) { - let t = b; - b = a % b; - a = t; - } - return a; -} -$$ +-- Create a table with user interaction logs +CREATE TABLE user_interaction_logs ( + log_id INT, + log_data VARIANT -- JSON interaction log +); -SELECT - number, - gcd_js((number * 3), (number * 6)) -FROM - numbers(5) -WHERE - (number > 0) -ORDER BY 1; -``` +-- Insert sample interaction log data +INSERT INTO user_interaction_logs VALUES + (1, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:00:00Z", "action": "view_product", "details": {"product_id": "p789", "category": "electronics", "price": 99.99}}')), + (2, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:05:10Z", "action": "add_to_cart", "details": {"product_id": "p789", "quantity": 1, "category": "electronics"}}')), + (3, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:02:30Z", "action": "search", "details": {"query": "wireless headphones", "results_count": 15}}')), + (4, PARSE_JSON('{"user_id": "u789", "timestamp": "2023-01-15T10:08:00Z", "action": "purchase", "details": {"order_id": "o555", "total_amount": 125.50, "item_count": 2}}')), + (5, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:10:00Z", "action": "view_page", "details": {"page_name": "homepage"}}')), + (6, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:12:00Z", "action": "purchase", "details": {"order_id": "o556", "total_amount": 25.00, "item_count": 1}}')); + + +```md +-- Create a JavaScript UDF to extract features from interaction logs +CREATE FUNCTION extract_interaction_features_js(VARIANT) +RETURNS VARIANT +LANGUAGE javascript HANDLER = 'extractFeatures' +AS $$ +export function extractFeatures(log) { + const action = (log.action || '').toLowerCase(); + const details = log.details || {}; + + let isSearchAction = false; + let hasProductInteraction = false; + let productCategoryIfAny = null; + let searchQueryLength = 0; + let purchaseValueBucket = null; + + if (action === 'search') { + isSearchAction = true; + searchQueryLength = (details.query || '').length; + } -此示例定义了一个聚合 UDF,该 UDF 通过根据一组值的相应权重对其进行聚合来计算该组值的加权平均值: + if (['view_product', 'add_to_cart', 'remove_from_cart'].includes(action)) { + hasProductInteraction = true; + productCategoryIfAny = details.category || null; + } + + if (action === 'purchase' && !productCategoryIfAny) { + hasProductInteraction = true; + } -```sql -CREATE FUNCTION weighted_avg (INT, INT) STATE {sum INT, weight INT} RETURNS FLOAT -LANGUAGE javascript AS $$ -export function create_state() { - return {sum: 0, weight: 0}; -} -export function accumulate(state, value, weight) { - state.sum += value * weight; - state.weight += weight; - return state; -} -export function retract(state, value, weight) { - state.sum -= value * weight; - state.weight -= weight; - return state; -} -export function merge(state1, state2) { - state1.sum += state2.sum; - state1.weight += state2.weight; - return state1; -} -export function finish(state) { - return state.sum / state.weight; + if (action === 'purchase') { + const totalAmount = details.total_amount || 0.0; + if (totalAmount < 50) { + purchaseValueBucket = 'Low'; + } else if (totalAmount < 200) { + purchaseValueBucket = 'Medium'; + } else { + purchaseValueBucket = 'High'; + } + } + + return { + is_search_action: isSearchAction, + has_product_interaction: has_product_interaction, + product_category_if_any: productCategoryIfAny, + search_query_length: searchQueryLength, + purchase_value_bucket: purchaseValueBucket + }; } $$; ``` ### WebAssembly -WebAssembly UDF 允许用户使用编译为 WebAssembly 的语言定义自定义逻辑或操作。然后可以直接在 SQL 查询中调用这些 UDF,以执行特定的计算或数据转换。 +WebAssembly UDF 允许用户使用编译为 WebAssembly 的语言定义自定义逻辑或操作。然后,可以在 SQL 查询中直接调用这些 UDF,以执行特定的计算或数据转换。 -#### 使用示例 +#### Usage Examples -在此示例中,创建了 "wasm_gcd" 函数来计算两个整数的最大公约数 (GCD)。该函数使用 WebAssembly 定义,其实现在 'test10_udf_wasm_gcd.wasm.zst' 二进制文件中。 +在此示例中,创建 "wasm_gcd" 函数来计算两个整数的最大公约数 (GCD)。该函数使用 WebAssembly 定义,其实现在 'test10_udf_wasm_gcd.wasm.zst' 二进制文件中。 在执行之前,函数实现会经过一系列步骤。首先,它被编译成一个二进制文件,然后被压缩成 'test10_udf_wasm_gcd.wasm.zst'。最后,压缩后的文件会提前上传到 Stage。 :::note -该函数可以使用 Rust 实现,如 https://github.com/risingwavelabs/arrow-udf/blob/main/arrow-udf-wasm/examples/wasm.rs 提供的示例所示 +该函数可以使用 Rust 实现,如 https://github.com/arrow-udf/arrow-udf/blob/main/arrow-udf-example/src/lib.rs 上的示例所示。 ::: ```sql @@ -297,6 +319,6 @@ WHERE ORDER BY 1; ``` -## 管理 UDF +## Managing UDFs -Databend 提供了各种命令来管理 UDF。有关详细信息,请参见 [用户自定义函数](/sql/sql-commands/ddl/udf/)。 \ No newline at end of file +Databend 提供了各种命令来管理 UDF。有关详细信息,请参见 [User-Defined Function](/sql/sql-commands/ddl/udf/)。{/*examples*/} diff --git a/docs/cn/release-notes/databend.md b/docs/cn/release-notes/databend.md index 6d02b973a1..f1ddaf8f76 100644 --- a/docs/cn/release-notes/databend.md +++ b/docs/cn/release-notes/databend.md @@ -8,7 +8,7 @@ slug: / import StepsWrap from '@site/src/components/StepsWrap'; import StepContent from '@site/src/components/Steps/step-content'; -本页提供有关 Databend 的最新功能、增强功能和错误修复的信息。 +This page provides information about recent features, enhancements, and bug fixes for Databend. @@ -37,7 +37,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2025年4月21日 (v1.2.725) +## Apr 21, 2025 (v1.2.725) @@ -45,7 +45,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2025年2月14日 (v1.2.697) +## Feb 14, 2025 (v1.2.697) @@ -53,7 +53,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2025年1月2日 (v1.2.680) +## Jan 2, 2025 (v1.2.680) @@ -61,7 +61,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年8月19日 (v1.2.615) +## Aug 19, 2024 (v1.2.615) @@ -69,7 +69,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年8月13日 (v1.2.609) +## Aug 13, 2024 (v1.2.609) @@ -77,7 +77,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年6月16日 (v1.2.530) +## Jun 16, 2024 (v1.2.530) @@ -85,7 +85,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年5月6日 (v1.2.452) +## May 6, 2024 (v1.2.452) @@ -93,7 +93,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年4月8日 (v1.2.410) +## Apr 8, 2024 (v1.2.410) @@ -101,7 +101,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年3月11日 (v1.2.371) +## Mar 11, 2024 (v1.2.371) @@ -109,7 +109,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年2月22日 (v1.2.344) +## Feb 22, 2024 (v1.2.344) @@ -117,7 +117,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年1月25日 (v1.2.307) +## Jan 25, 2024 (v1.2.307) @@ -125,7 +125,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年1月11日 (v1.2.292) +## Jan 11, 2024 (v1.2.292) @@ -133,7 +133,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年1月10日 (v1.2.290) +## Jan 10, 2024 (v1.2.290) @@ -141,7 +141,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2024年1月2日 (v1.2.279) +## Jan 2, 2024 (v1.2.279) @@ -149,7 +149,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2023年12月30日 (v1.2.275) +## Dec 30, 2023 (v1.2.275) @@ -157,7 +157,7 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2023年12月20日 (v1.2.262) +## Dec 20, 2023 (v1.2.262) @@ -165,10 +165,10 @@ import MD17 from '@site/docs/release-stable/2023-12-13_v1.2.233.md'; -## 2023年12月13日 (v1.2.233) +## Dec 13, 2023 (v1.2.233) - \ No newline at end of file + diff --git a/docs/cn/sql-reference/00-sql-reference/30-table-engines/00-fuse.md b/docs/cn/sql-reference/00-sql-reference/30-table-engines/00-fuse.md index 16cc81c3e3..d60dfae142 100644 --- a/docs/cn/sql-reference/00-sql-reference/30-table-engines/00-fuse.md +++ b/docs/cn/sql-reference/00-sql-reference/30-table-engines/00-fuse.md @@ -1,12 +1,29 @@ +```md --- -title: Fuse Engine +title: Fuse Engine 表 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + -Databend 采用 Fuse Engine 作为其默认引擎,提供了一个用户界面友好的数据管理系统,让人联想到 Git。用户可以轻松地查询任何给定时刻的数据,并毫不费力地将数据恢复到任何所需的点。 +## 概述 + +Databend 使用 Fuse Engine 作为其默认存储引擎,提供一个类似 Git 的数据管理系统,具有以下特点: + +- **基于快照的架构**: 查询和恢复任何时间点的数据,具有数据变更历史以进行恢复 +- **高性能**: 针对分析工作负载进行了优化,具有自动索引和 Bloom Filter +- **高效存储**: 使用 Parquet 格式,具有高压缩率,可实现最佳存储效率 +- **灵活配置**: 可自定义压缩、索引和存储选项 +- **数据维护**: 自动数据保留、快照管理和变更跟踪功能 + +## 何时使用 Fuse Engine + +适用于: +- **分析**: 具有列式存储的 OLAP 查询 +- **数仓**: 大量历史数据 +- **时间回溯**: 访问历史数据版本 +- **云存储**: 针对对象存储进行了优化 ## 语法 @@ -20,26 +37,169 @@ CREATE TABLE ( 有关 `CREATE TABLE` 语法的更多详细信息,请参见 [CREATE TABLE](../../10-sql-commands/00-ddl/01-table/10-ddl-create-table.md)。 -| 参数 | 描述 | -| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `ENGINE` | 如果未明确指定引擎,Databend 将自动默认使用 Fuse Engine 创建表,这等效于 `ENGINE = FUSE`。 | -| `CLUSTER BY` | 指定由多个表达式组成的数据的排序方法。有关更多信息,请参见 [Cluster Key](/guides/performance/cluster-key)。 | -| `` | Fuse Engine 提供了各种选项(不区分大小写),允许您自定义表的属性。有关详细信息,请参见 [Fuse Engine Options](#fuse-engine-options)。
- 使用空格分隔多个选项。
- 使用 [ALTER TABLE OPTION](../../10-sql-commands/00-ddl/01-table/90-alter-table-option.md) 修改表的选项。
- 使用 [SHOW CREATE TABLE](../../10-sql-commands/00-ddl/01-table/show-create-table.md) 显示表的选项。 | +## 参数 + +以下是创建 Fuse Engine 表的主要参数: + +#### `ENGINE` +- **描述:** + 如果未显式指定引擎,Databend 将自动默认使用 Fuse Engine 创建表,这等效于 `ENGINE = FUSE`。 + +--- + +#### `CLUSTER BY` +- **描述:** + 指定由多个表达式组成的数据的排序方法。 有关更多信息,请参见 [Cluster Key](/guides/performance/cluster-key)。 + +--- + +#### `` +- **描述:** + Fuse Engine 提供了各种选项(不区分大小写),允许您自定义表的属性。 + - 有关详细信息,请参见 [Fuse Engine Options](#fuse-engine-options)。 + - 使用空格分隔多个选项。 + - 使用 [ALTER TABLE OPTION](../../10-sql-commands/00-ddl/01-table/90-alter-table-option.md) 修改表的选项。 + - 使用 [SHOW CREATE TABLE](../../10-sql-commands/00-ddl/01-table/show-create-table.md) 显示表的选项。 + +--- ## Fuse Engine Options -以下是可用的 Fuse Engine 选项: - - -| 选项 | 语法 | 描述 | -| ------------------------------ | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| bloom_index_columns | `bloom_index_columns = ' [, ...]'` | 指定用于 Bloom 索引的列。这些列的数据类型可以是 Map、Number、String、Date 或 Timestamp。如果未指定特定列,则默认在所有支持的列上创建 Bloom 索引。`bloom_index_columns=''` 禁用 Bloom 索引。 | -| compression | `compression = ''` | 指定引擎的压缩方法。压缩选项包括 lz4、zstd、snappy 或 none。默认情况下,对象存储中的压缩方法为 zstd,文件系统 (fs) 存储中的压缩方法为 lz4。 | -| storage_format | `storage_format = ''` | 指定数据的存储方式。默认情况下,storage_format 设置为 **Parquet**,它提供高压缩率,非常适合云原生对象存储。此外,还支持实验性的 **Native** 格式,优化了文件系统等存储设备的内存复制开销。 | -| snapshot_loc | `snapshot_loc = ''` | 以字符串格式指定位置参数,允许轻松共享表,而无需复制数据。 | -| block_size_threshold | `block_size_threshold = ` | 指定最大块大小(以字节为单位)。默认为 104,857,600 字节。 | -| block_per_segment | `block_per_segment = ` | 指定一个段中的最大块数。默认为 1,000。 | -| row_per_block | `row_per_block = ` | 指定文件中的最大行数。默认为 1,000,000。 | -| change_tracking | `change_tracking = True / False` | 在 Fuse 引擎中将此选项设置为 `True` 允许跟踪表的更改。
为表创建流将自动将 `change_tracking` 设置为 `True`,并向表中引入额外的隐藏列作为更改跟踪元数据。有关更多信息,请参见 [流的工作原理](/guides/load-data/continuous-data-pipelines/stream#how-stream-works)。 | -| data_retention_period_in_hours | `data_retention_period_in_hours = ` | 指定保留表数据的小时数。最小值为 1 小时。最大值由 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 配置文件中的 `data_retention_time_in_days_max` 设置定义,如果未指定,则默认为 2,160 小时(90 天 x 24 小时)。 | -| data_retention_num_snapshots_to_keep | `data_retention_num_snapshots_to_keep = ` | 指定要为表保留的快照数。此选项与 `enable_auto_vacuum` 设置结合使用,以提供对每个表的快照保留策略的精细控制。设置后,在 vacuum 操作后,只会保留指定数量的最新快照。 | \ No newline at end of file +以下是可用的 Fuse Engine 选项,按其用途分组: + +--- + +### `compression` +- **语法:** + `compression = ''` +- **描述:** + 指定引擎的压缩方法。 压缩选项包括 lz4、zstd、snappy 或 none。 压缩方法在对象存储中默认为 zstd,在文件系统 (fs) 存储中默认为 lz4。 + +--- + +### `snapshot_loc` +- **语法:** + `snapshot_loc = ''` +- **描述:** + 以字符串格式指定位置参数,允许轻松共享表而无需复制数据。 + +--- + + +### `block_size_threshold` +- **语法:** + `block_size_threshold = ` +- **描述:** + 指定最大块大小(以字节为单位)。 默认为 104,857,600 字节。 + +--- + +### `block_per_segment` +- **语法:** + `block_per_segment = ` +- **描述:** + 指定段中的最大块数。 默认为 1,000。 + +--- + +### `row_per_block` +- **语法:** + `row_per_block = ` +- **描述:** + 指定文件中的最大行数。 默认为 1,000,000。 + +--- + +### `bloom_index_columns` +- **语法:** + `bloom_index_columns = ' [, ...]'` +- **描述:** + 指定要用于 Bloom Index 的列。 这些列的数据类型可以是 Map、Number、String、Date 或 Timestamp。 如果未指定任何特定列,则默认情况下会在所有受支持的列上创建 Bloom Index。 `bloom_index_columns=''` 禁用 Bloom Index。 + +--- + +### `change_tracking` +- **语法:** + `change_tracking = True / False` +- **描述:** + 在 Fuse Engine 中将此选项设置为 `True` 允许跟踪表的更改。 为表创建 Stream 将自动将 `change_tracking` 设置为 `True`,并向表中引入额外的隐藏列作为更改跟踪元数据。 有关更多信息,请参见 [How Stream Works](/guides/load-data/continuous-data-pipelines/stream#how-stream-works)。 + +--- + +### `data_retention_period_in_hours` +- **语法:** + `data_retention_period_in_hours = ` +- **描述:** + 指定保留表数据的小时数。 最小值为 1 小时。 最大值由 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 配置文件中的 `data_retention_time_in_days_max` 设置定义,如果未指定,则默认为 2,160 小时(90 天 x 24 小时)。 + +--- + +### `enable_auto_vacuum` +- **语法:** + `enable_auto_vacuum = 0 / 1` +- **描述:** + 控制表是否在 mutations 期间自动触发 vacuum 操作。 这可以全局设置为所有表的设置,也可以在表级别进行配置。 表级别选项的优先级高于同名的会话/全局设置。 启用后(设置为 1),在 INSERT 或 ALTER TABLE 等 mutations 之后将自动触发 vacuum 操作,根据配置的保留策略清理表数据。 + + **示例:** + ```sql + -- 全局设置 enable_auto_vacuum,适用于所有会话中的所有表 + SET GLOBAL enable_auto_vacuum = 1; + + -- 创建一个禁用自动 vacuum 的表(覆盖全局设置) + CREATE OR REPLACE TABLE t1 (id INT) ENABLE_AUTO_VACUUM = 0; + INSERT INTO t1 VALUES(1); -- 即使有全局设置也不会触发 vacuum + + -- 创建另一个继承全局设置的表 + CREATE OR REPLACE TABLE t2 (id INT); + INSERT INTO t2 VALUES(1); -- 由于全局设置将触发 vacuum + + -- 为现有表启用自动 vacuum + ALTER TABLE t1 SET OPTIONS(ENABLE_AUTO_VACUUM = 1); + INSERT INTO t1 VALUES(2); -- 现在将触发 vacuum + + -- 表选项优先于全局设置 + SET GLOBAL enable_auto_vacuum = 0; -- 全局关闭 + -- t1 仍然会 vacuum,因为表设置覆盖了全局设置 + INSERT INTO t1 VALUES(3); -- 仍然会触发 vacuum + INSERT INTO t2 VALUES(2); -- 不再触发 vacuum + ``` + +--- + +### `data_retention_num_snapshots_to_keep` +- **语法:** + `data_retention_num_snapshots_to_keep = ` +- **描述:** + 指定在 vacuum 操作期间要保留的快照数。 这可以全局设置为所有表的设置,也可以在表级别进行配置。 表级别选项的优先级高于同名的会话/全局设置。 设置后,在 vacuum 操作后只会保留指定数量的最新快照。 覆盖 `data_retention_time_in_days` 设置。 如果设置为 0,则将忽略此设置。 此选项与 `enable_auto_vacuum` 设置结合使用,以提供对快照保留策略的精细控制。 + + **示例:** + ```sql + -- 将全局保留设置为所有会话中所有表的 10 个快照 + SET GLOBAL data_retention_num_snapshots_to_keep = 10; + + -- 创建一个具有自定义快照保留的表(覆盖全局设置) + CREATE OR REPLACE TABLE t1 (id INT) + enable_auto_vacuum = 1 + data_retention_num_snapshots_to_keep = 5; + + -- 创建另一个继承全局设置的表 + CREATE OR REPLACE TABLE t2 (id INT) enable_auto_vacuum = 1; + + -- 当触发 vacuum 时: + -- t1 将保留 5 个快照(表设置) + -- t2 将保留 10 个快照(全局设置) + + -- 更改全局设置 + SET GLOBAL data_retention_num_snapshots_to_keep = 20; + + -- 表选项仍然优先: + -- t1 仍然只会保留 5 个快照 + -- t2 现在将保留 20 个快照 + + -- 修改现有表的快照保留 + ALTER TABLE t1 SET OPTIONS(data_retention_num_snapshots_to_keep = 3); + -- 现在,当触发 vacuum 时,t1 将保留 3 个快照 + ``` + +--- +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md b/docs/cn/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md new file mode 100644 index 0000000000..495cf40a7b --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md @@ -0,0 +1,9 @@ +--- +id: iceberg +title: Apache Iceberg™ Tables +sidebar_label: Apache Iceberg™ Tables +--- + +import Content from '../../../guides/51-access-data-lake/02-iceberg.md'; + + \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/30-table-engines/03-hive.md b/docs/cn/sql-reference/00-sql-reference/30-table-engines/03-hive.md new file mode 100644 index 0000000000..997dcdbe08 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/30-table-engines/03-hive.md @@ -0,0 +1,9 @@ +--- +id: hive +title: Apache Hive Tables +sidebar_label: Apache Hive Tables +--- + +import Content from '../../../guides/51-access-data-lake/01-hive.md'; + + diff --git a/docs/cn/sql-reference/00-sql-reference/30-table-engines/_04-delta.md b/docs/cn/sql-reference/00-sql-reference/30-table-engines/_04-delta.md new file mode 100644 index 0000000000..d8ab2d76ea --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/30-table-engines/_04-delta.md @@ -0,0 +1,9 @@ +--- +id: delta +title: Delta Lake 引擎 +sidebar_label: Delta Lake 引擎 +--- + +import Content from '../../../guides/51-access-data-lake/03-delta.md'; + + \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/information-schema-tables.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/information-schema-tables.md new file mode 100644 index 0000000000..4217acd7a5 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/information-schema-tables.md @@ -0,0 +1,44 @@ +--- +title: information_schema.tables +--- + +`information_schema.tables` 系统表是一个视图,提供了关于所有数据库中所有表的元数据,包括它们的 schema、类型、引擎和创建细节。它还包括存储指标,如数据长度、索引长度和行数,从而可以深入了解表的结构和使用情况。 + +另请参阅:[system.tables](system-tables.md) + +```sql title='Examples:' +SELECT * FROM information_schema.tables LIMIT 3; + +┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ table_catalog │ table_schema │ table_name │ table_type │ engine │ create_time │ drop_time │ data_length │ index_length │ table_rows │ auto_increment │ table_collation │ data_free │ table_comment │ +├────────────────────┼────────────────────┼────────────┼────────────┼────────┼────────────────────────────┼─────────────────────┼──────────────────┼──────────────────┼──────────────────┼────────────────┼─────────────────┼───────────┼───────────────┤ +│ default │ default │ json_table │ BASE TABLE │ FUSE │ 2024-11-18 23:22:21.576421 │ NULL │ 68 │ 425 │ 1 │ NULL │ NULL │ NULL │ │ +│ information_schema │ information_schema │ views │ VIEW │ VIEW │ 2024-11-20 21:04:12.952792 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ +│ information_schema │ information_schema │ statistics │ VIEW │ VIEW │ 2024-11-20 21:04:12.952795 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ +└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` + +要显示 `information_schema.tables` 的 schema,请使用 `DESCRIBE information_schema.tables`: + +```sql +DESCRIBE information_schema.tables; + +┌────────────────────────────────────────────────────────────────────────────────────┐ +│ Field │ Type │ Null │ Default │ Extra │ +├─────────────────┼─────────────────┼────────┼──────────────────────────────┼────────┤ +│ table_catalog │ VARCHAR │ NO │ '' │ │ +│ table_schema │ VARCHAR │ NO │ '' │ │ +│ table_name │ VARCHAR │ NO │ '' │ │ +│ table_type │ VARCHAR │ NO │ '' │ │ +│ engine │ VARCHAR │ NO │ '' │ │ +│ create_time │ TIMESTAMP │ NO │ '1970-01-01 00:00:00.000000' │ │ +│ drop_time │ TIMESTAMP │ YES │ NULL │ │ +│ data_length │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ index_length │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ table_rows │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ auto_increment │ NULL │ NO │ NULL │ │ +│ table_collation │ NULL │ NO │ NULL │ │ +│ data_free │ NULL │ NO │ NULL │ │ +│ table_comment │ VARCHAR │ NO │ '' │ │ +└────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-build-options.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-build-options.md new file mode 100644 index 0000000000..8c4b5f8b43 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-build-options.md @@ -0,0 +1,21 @@ +--- +title: system.build_options +--- + +该系统表描述了当前 Databend 版本的构建选项。 + +- `cargo_features`: 已启用的包功能,这些功能在 `Cargo.toml` 的 `[features]` 部分中列出。 +- `target_features`: 已为当前编译目标启用的平台功能。参考:[Conditional Compilation - `target_feature`](https://doc.rust-lang.org/reference/conditional-compilation.html#target_feature)。 + +```sql +SELECT * FROM system.build_options; ++----------------+---------------------+ +| cargo_features | target_features | ++----------------+---------------------+ +| default | fxsr | +| | llvm14-builtins-abi | +| | sse | +| | sse2 | ++----------------+---------------------+ +4 rows in set (0.031 sec) +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-caches.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-caches.md new file mode 100644 index 0000000000..709e7363ae --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-caches.md @@ -0,0 +1,21 @@ +--- +title: system.caches +--- + +Databend 中管理的各种缓存的概述。 + +下表显示了缓存名称、缓存中的项目数以及缓存大小: +```sql +SELECT * FROM system.caches; ++--------------------------------+-----------+------+ +| name | num_items | size | ++--------------------------------+-----------+------+ +| table_snapshot_cache | 2 | 2 | +| table_snapshot_statistic_cache | 0 | 0 | +| segment_info_cache | 64 | 64 | +| bloom_index_filter_cache | 0 | 0 | +| bloom_index_meta_cache | 0 | 0 | +| prune_partitions_cache | 2 | 2 | +| file_meta_data_cache | 0 | 0 | ++--------------------------------+-----------+------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-clusters.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-clusters.md new file mode 100644 index 0000000000..ed08cb4f39 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-clusters.md @@ -0,0 +1,22 @@ +--- +title: system.clusters +--- + +包含有关集群节点的信息。 + +:::note +您可以使用配置选项 `disable_system_table_load` 禁用对 `clusters` 表的访问。 + +例如,Databend Cloud 的用户将无法看到此表。 +::: + +```sql +SELECT * FROM system.clusters; ++------------------------+---------+------+ +| name | host | port | ++------------------------+---------+------+ +| 2KTgGnTDuKHw3wu9CCVIf6 | 0.0.0.0 | 9093 | +| bZTEWpQGLwRgcRyHre1xL3 | 0.0.0.0 | 9092 | +| plhQlHvVfT0p1T5QdnvhC4 | 0.0.0.0 | 9091 | ++------------------------+---------+------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-columns.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-columns.md new file mode 100644 index 0000000000..7c7192d211 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-columns.md @@ -0,0 +1,42 @@ +--- +title: system.columns +--- + +包含表列的信息。 + +```sql +DESC system.columns; ++--------------------+---------+------+---------+-------+ +| Field | Type | Null | Default | Extra | ++--------------------+---------+------+---------+-------+ +| name | VARCHAR | NO | | | +| database | VARCHAR | NO | | | +| table | VARCHAR | NO | | | +| type | VARCHAR | NO | | | +| default_kind | VARCHAR | NO | | | +| default_expression | VARCHAR | NO | | | +| is_nullable | BOOLEAN | NO | false | | +| comment | VARCHAR | NO | | | + +``` + +```sql ++--------------------------+----------+---------------------+------------------+-------------------+--------------+--------------------+-------------+---------+ +| name | database | table | column_type | data_type | default_kind | default_expression | is_nullable | comment | ++--------------------------+----------+---------------------+------------------+-------------------+--------------+--------------------+-------------+---------+ +| id | system | processes | String | VARCHAR | | | NO | | +| type | system | processes | String | VARCHAR | | | NO | | +| host | system | processes | Nullable(String) | VARCHAR | | | YES | | +| user | system | processes | String | VARCHAR | | | NO | | +| command | system | processes | String | VARCHAR | | | NO | | +| database | system | processes | String | VARCHAR | | | NO | | +| extra_info | system | processes | String | VARCHAR | | | NO | | +| memory_usage | system | processes | Int64 | BIGINT | | | NO | | +| data_read_bytes | system | processes | UInt64 | BIGINT UNSIGNED | | | NO | | +| data_write_bytes | system | processes | UInt64 | BIGINT UNSIGNED | | | NO | | +| scan_progress_read_rows | system | processes | UInt64 | BIGINT UNSIGNED | | | NO | | +| scan_progress_read_bytes | system | processes | UInt64 | BIGINT UNSIGNED | | | NO | | +| mysql_connection_id | system | processes | Nullable(UInt32) | INT UNSIGNED | | | YES | | +.... + +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-configs.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-configs.md new file mode 100644 index 0000000000..1d88ab2c47 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-configs.md @@ -0,0 +1,132 @@ +包含 Databend 服务器配置的信息。 + +:::note +您可以使用配置选项 `disable_system_table_load` 禁用对 `configs` 表的访问。 + +例如,Databend Cloud 的用户将无法看到此表。 +::: + +```sql +mysql> SELECT * FROM system.configs; ++---------+----------------------------------------+--------------------------------+-------------+ +| group | name | value | description | ++---------+----------------------------------------+--------------------------------+-------------+ +| query | tenant_id | admin | | +| query | cluster_id | | | +| query | num_cpus | 0 | | +| query | mysql_handler_host | 127.0.0.1 | | +| query | mysql_handler_port | 3307 | | +| query | max_active_sessions | 256 | | +| query | max_server_memory_usage | 0 | | +| query | max_memory_limit_enabled | false | | +| query | clickhouse_handler_host | 127.0.0.1 | | +| query | clickhouse_handler_port | 9000 | | +| query | clickhouse_http_handler_host | 127.0.0.1 | | +| query | clickhouse_http_handler_port | 8124 | | +| query | http_handler_host | 127.0.0.1 | | +| query | http_handler_port | 8000 | | +| query | http_handler_result_timeout_secs | 60 | | +| query | flight_api_address | 127.0.0.1:9090 | | +| query | admin_api_address | 127.0.0.1:8080 | | +| query | metric_api_address | 127.0.0.1:7070 | | +| query | http_handler_tls_server_cert | | | +| query | http_handler_tls_server_key | | | +| query | http_handler_tls_server_root_ca_cert | | | +| query | api_tls_server_cert | | | +| query | api_tls_server_key | | | +| query | api_tls_server_root_ca_cert | | | +| query | rpc_tls_server_cert | | | +| query | rpc_tls_server_key | | | +| query | rpc_tls_query_server_root_ca_cert | | | +| query | rpc_tls_query_service_domain_name | localhost | | +| query | table_engine_memory_enabled | true | | +| query | wait_timeout_mills | 5000 | | +| query | max_query_log_size | 10000 | | +| query | management_mode | false | | +| query | jwt_key_file | | | +| query | jwt_key_files | | | +| query | users | | | +| query | share_endpoint_address | | | +| query | share_endpoint_auth_token_file | | | +| query | quota | null | | +| query | internal_enable_sandbox_tenant | false | | +| log | level | INFO | | +| log | dir | ./.databend/logs | | +| log | query_enabled | false | | +| log | file.on | true | | +| log | file.level | INFO | | +| log | file.dir | ./.databend/logs | | +| log | file.format | json | | +| log | stderr.on | false | | +| log | stderr.level | INFO | | +| log | stderr.format | text | | +| meta | embedded_dir | .databend/meta | | +| meta | endpoints | | | +| meta | username | root | | +| meta | password | | | +| meta | client_timeout_in_second | 10 | | +| meta | auto_sync_interval | 0 | | +| meta | rpc_tls_meta_server_root_ca_cert | | | +| meta | rpc_tls_meta_service_domain_name | localhost | | +| meta | unhealth_endpoint_evict_time | 120 | | +| cache | enable_table_meta_cache | true | | +| cache | table_meta_snapshot_count | 256 | | +| cache | table_meta_segment_count | 10240 | | +| cache | table_meta_statistic_count | 256 | | +| cache | enable_table_index_bloom | true | | +| cache | table_bloom_index_meta_count | 3000 | | +| cache | table_bloom_index_filter_count | 1048576 | | +| cache | data_cache_storage | none | | +| cache | table_data_cache_population_queue_size | 65536 | | +| cache | disk.max_bytes | 21474836480 | | +| cache | disk.path | ./.databend/_cache | | +| cache | table_data_deserialized_data_bytes | 0 | | +| storage | type | fs | | +| storage | num_cpus | 0 | | +| storage | allow_insecure | false | | +| storage | fs.data_path | _data | | +| storage | gcs.endpoint_url | https://storage.googleapis.com | | +| storage | gcs.bucket | | | +| storage | gcs.root | | | +| storage | gcs.credential | | | +| storage | s3.region | | | +| storage | s3.endpoint_url | https://s3.amazonaws.com | | +| storage | s3.access_key_id | | | +| storage | s3.secret_access_key | | | +| storage | s3.security_token | | | +| storage | s3.bucket | | | +| storage | s3.root | | | +| storage | s3.master_key | | | +| storage | s3.enable_virtual_host_style | false | | +| storage | s3.role_arn | | | +| storage | s3.external_id | | | +| storage | azblob.account_name | | | +| storage | azblob.account_key | | | +| storage | azblob.container | | | +| storage | azblob.endpoint_url | | | +| storage | azblob.root | | | +| storage | obs.access_key_id | | | +| storage | obs.secret_access_key | | | +| storage | obs.bucket | | | +| storage | obs.endpoint_url | | | +| storage | obs.root | | | +| storage | oss.access_key_id | | | +| storage | oss.access_key_secret | | | +| storage | oss.bucket | | | +| storage | oss.endpoint_url | | | +| storage | oss.presign_endpoint_url | | | +| storage | oss.root | | | +| storage | cache.type | none | | +| storage | cache.num_cpus | 0 | | +| storage | cache.fs.data_path | _data | | +| storage | cache.moka.max_capacity | 1073741824 | | +| storage | cache.moka.time_to_live | 3600 | | +| storage | cache.moka.time_to_idle | 600 | | +| storage | cache.redis.endpoint_url | | | +| storage | cache.redis.username | | | +| storage | cache.redis.password | | | +| storage | cache.redis.root | | | +| storage | cache.redis.db | 0 | | +| storage | cache.redis.default_ttl | 0 | | ++---------+----------------------------------------+--------------------------------+-------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-contributors.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-contributors.md new file mode 100644 index 0000000000..ded5177d36 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-contributors.md @@ -0,0 +1,33 @@ +--- +title: system.contributors +--- + +包含贡献者的信息。 + +```sql +SELECT * FROM system.contributors LIMIT 20; ++-------------------------+ +| name | ++-------------------------+ +| artorias1024 | +| BohuTANG | +| dependabot[bot] | +| dependabot-preview[bot] | +| drdr xp | +| Eason | +| hulunbier | +| jyizheng | +| leiysky | +| smallfish | +| sundy-li | +| sundyli | +| taiyang-li | +| TLightSky | +| Winter Zhang | +| wubx | +| yizheng | +| Yizheng Jiao | +| zhang2014 | +| zhihanz | ++-------------------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-credits.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-credits.md new file mode 100644 index 0000000000..c84a1d9249 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-credits.md @@ -0,0 +1,33 @@ +--- +title: system.credits +--- + +包含有关 credits 的信息。 + +```sql +SELECT * FROM system.credits LIMIT 20; ++-------------------+---------+---------------------------+ +| name | version | license | ++-------------------+---------+---------------------------+ +| addr2line | 0.16.0 | Apache-2.0 OR MIT | +| adler | 1.0.2 | 0BSD OR Apache-2.0 OR MIT | +| ahash | 0.6.3 | Apache-2.0 OR MIT | +| ahash | 0.7.4 | Apache-2.0 OR MIT | +| aho-corasick | 0.7.18 | MIT OR Unlicense | +| ansi_term | 0.9.0 | MIT | +| ansi_term | 0.11.0 | MIT | +| ansi_term | 0.12.1 | MIT | +| anyhow | 1.0.43 | Apache-2.0 OR MIT | +| arbitrary | 1.0.1 | Apache-2.0 OR MIT | +| arrayvec | 0.4.12 | Apache-2.0 OR MIT | +| arrayvec | 0.5.2 | Apache-2.0 OR MIT | +| arrow-flight | 0.1.0 | Apache-2.0 | +| arrow2 | 0.4.0 | Apache-2.0 | +| assert_cmd | 2.0.1 | Apache-2.0 OR MIT | +| async-compat | 0.2.1 | Apache-2.0 OR MIT | +| async-raft | 0.6.1 | Apache-2.0 OR MIT | +| async-stream | 0.3.2 | MIT | +| async-stream-impl | 0.3.2 | MIT | +| async-trait | 0.1.51 | Apache-2.0 OR MIT | ++-------------------+---------+---------------------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases-with-history.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases-with-history.md new file mode 100644 index 0000000000..3e53bd1ba3 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases-with-history.md @@ -0,0 +1,24 @@ +--- +title: system.databases_with_history +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +记录所有数据库,包括活动的和已删除的数据库。它显示了每个数据库的 catalog、名称、唯一 ID、所有者(如果已指定)和删除时间戳(如果仍然处于活动状态,则为 NULL)。 + +另请参阅:[SHOW DROP DATABASES](../../10-sql-commands/00-ddl/00-database/show-drop-databases.md) + +```sql +SELECT * FROM system.databases_with_history; + +┌────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ catalog │ name │ database_id │ owner │ dropped_on │ +├─────────┼────────────────────┼─────────────────────┼──────────────────┼────────────────────────────┤ +│ default │ system │ 4611686018427387905 │ NULL │ NULL │ +│ default │ information_schema │ 4611686018427387906 │ NULL │ NULL │ +│ default │ default │ 1 │ NULL │ NULL │ +│ default │ my_db │ 114 │ NULL │ 2024-11-15 02:44:46.207120 │ +└────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases.md new file mode 100644 index 0000000000..2e894a1c3f --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-databases.md @@ -0,0 +1,36 @@ +--- +title: system.databases +--- + +提供了系统中所有数据库的元数据,包括它们的 catalogs、名称、唯一 ID、所有者和删除时间戳。 + +另请参阅:[SHOW DATABASES](../../10-sql-commands/00-ddl/00-database/show-databases.md) + +```sql title='Examples:' +SELECT * FROM system.databases; + +┌─────────────────────────────────────────────────────────────────────────────────────────────┐ +│ catalog │ name │ database_id │ owner │ dropped_on │ +├─────────┼────────────────────┼─────────────────────┼──────────────────┼─────────────────────┤ +│ default │ system │ 4611686018427387905 │ NULL │ NULL │ +│ default │ information_schema │ 4611686018427387906 │ NULL │ NULL │ +│ default │ default │ 1 │ NULL │ NULL │ +│ default │ doc │ 2597 │ account_admin │ NULL │ +└─────────────────────────────────────────────────────────────────────────────────────────────┘ +``` + +要显示 `system.databases` 的 schema,请使用 `DESCRIBE system.databases`: + +```sql +DESCRIBE system.databases; + +┌───────────────────────────────────────────────────────────┐ +│ Field │ Type │ Null │ Default │ Extra │ +├─────────────┼─────────────────┼────────┼─────────┼────────┤ +│ catalog │ VARCHAR │ NO │ '' │ │ +│ name │ VARCHAR │ NO │ '' │ │ +│ database_id │ BIGINT UNSIGNED │ NO │ 0 │ │ +│ owner │ VARCHAR │ YES │ NULL │ │ +│ dropped_on │ TIMESTAMP │ YES │ NULL │ │ +└───────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-functions.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-functions.md new file mode 100644 index 0000000000..3ccf6bc75c --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-functions.md @@ -0,0 +1,32 @@ +--- +title: system.functions +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含有关支持的内置标量和聚合的信息。 + +另请参阅:[SHOW FUNCTIONS](/sql/sql-commands/administration-cmds/show-functions)。 + +## 示例 + +```sql +SELECT * FROM system.functions LIMIT 10; + +┌──────────────────────────────────────────────────────────────┐ +│ name │ is_aggregate │ description │ syntax │ example │ +├──────────────┼──────────────┼─────────────┼────────┼─────────┤ +│ abs │ false │ │ │ │ +│ acos │ false │ │ │ │ +│ add │ false │ │ │ │ +│ add_days │ false │ │ │ │ +│ add_hours │ false │ │ │ │ +│ add_minutes │ false │ │ │ │ +│ add_months │ false │ │ │ │ +│ add_quarters │ false │ │ │ │ +│ add_seconds │ false │ │ │ │ +│ add_years │ false │ │ │ │ +└──────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-indexes.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-indexes.md new file mode 100644 index 0000000000..6cacab33cc --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-indexes.md @@ -0,0 +1,25 @@ +--- +title: system.indexes +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含有关已创建的聚合索引的信息。 + +另请参阅:[SHOW INDEXES](../../10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md) + +```sql +CREATE TABLE t1(a int,b int); + +CREATE AGGREGATING INDEX idx1 AS SELECT SUM(a), b FROM default.t1 WHERE b > 3 GROUP BY b; + +SELECT * FROM system.indexes; + ++----------+-------------+------------------------------------------------------------+----------------------------+ +| name | type | definition | created_on | ++----------+-------------+------------------------------------------------------------+----------------------------+ +| test_idx | AGGREGATING | SELECT b, SUM(a) FROM default.t1 WHERE (b > 3) GROUP BY b | 2023-05-17 11:53:54.474377 | ++----------+-------------+------------------------------------------------------------+----------------------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-locks.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-locks.md new file mode 100644 index 0000000000..c0092e3ad4 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-locks.md @@ -0,0 +1,22 @@ +--- +title: system.locks +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含系统中锁的信息。 + +另请参阅:[SHOW LOCKS](../../10-sql-commands/00-ddl/14-transaction/show-locks.md) + +```sql +SELECT * FROM system.locks; + ++----------+----------+-------+---------+------+------------------------+--------------------------------------+----------------------------+----------------------------+------------+ +| table_id | revision | type | status | user | node | query_id | created_on | acquired_on | extra_info | ++----------+----------+-------+---------+------+------------------------+--------------------------------------+----------------------------+----------------------------+------------+ +| 57 | 4517 | TABLE | HOLDING | root | xzi6pRbLUYasuA9QFB36m6 | d7989971-d5ec-4764-8e37-afe38ebc13e2 | 2023-12-13 09:56:47.295684 | 2023-12-13 09:56:47.310805 | | +| 57 | 4521 | TABLE | WAITING | zzq | xzi6pRbLUYasuA9QFB36m6 | 4bc78044-d4fc-4fe1-a5c5-ff6ab1e3e372 | 2023-12-13 09:56:48.419774 | NULL | | ++----------+----------+-------+---------+------+------------------------+--------------------------------------+----------------------------+----------------------------+------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-metrics.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-metrics.md new file mode 100644 index 0000000000..b579f1dcb6 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-metrics.md @@ -0,0 +1,27 @@ +--- +title: system.metrics +--- + +包含度量事件的信息。 + + +```sql +SELECT * FROM system.metrics; ++--------------------------------------+---------+---------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| metric | kind | labels | value | ++--------------------------------------+---------+---------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| session_connect_numbers | counter | {"tenant":"","cluster_name":""} | 1.0 | +| interpreter_usedtime_sum | untyped | {} | 0.8492673029999999 | +| interpreter_usedtime_count | untyped | {} | 20.0 | +| mysql_process_request_duration_sum | untyped | {} | 0.968661883 | +| mysql_process_request_duration_count | untyped | {} | 18.0 | +| optimizer_optimize_usedtime_sum | untyped | {} | 0.018182794999999998 | +| optimizer_optimize_usedtime_count | untyped | {} | 18.0 | +| parser_parse_usedtime_sum | untyped | {} | 0.012580696000000002 | +| parser_parse_usedtime_count | untyped | {} | 28.0 | +| interpreter_usedtime | summary | {} | [{"quantile":0.0,"count":0.00084899},{"quantile":0.5,"count":0.0034856524910821664},{"quantile":0.9,"count":0.007957025569656927},{"quantile":0.95,"count":0.012534129195898459},{"quantile":0.99,"count":0.012534129195898459},{"quantile":0.999,"count":0.012534129195898459},{"quantile":1.0,"count":0.2071939412186652}] | +| parser_parse_usedtime | summary | {} | [{"quantile":0.0,"count":0.000084647},{"quantile":0.5,"count":0.00035389863363880604},{"quantile":0.9,"count":0.0007507033214071675},{"quantile":0.95,"count":0.0007754244284730125},{"quantile":0.99,"count":0.0008191035653466911},{"quantile":0.999,"count":0.0008191035653466911},{"quantile":1.0,"count":0.0011012603000499356}] | +| mysql_process_request_duration | summary | {} | [{"quantile":0.0,"count":0.007505268},{"quantile":0.5,"count":0.0120836736849045},{"quantile":0.9,"count":0.015794397051616272},{"quantile":0.95,"count":0.01716470533994825},{"quantile":0.99,"count":0.01716470533994825},{"quantile":0.999,"count":0.01716470533994825},{"quantile":1.0,"count":0.21222674793285548}] | +| optimizer_optimize_usedtime | summary | {} | [{"quantile":0.0,"count":0.000398654},{"quantile":0.5,"count":0.0008709726097674335},{"quantile":0.9,"count":0.0013493935265982112},{"quantile":0.95,"count":0.0014431890055320044},{"quantile":0.99,"count":0.0014431890055320044},{"quantile":0.999,"count":0.0014431890055320044},{"quantile":1.0,"count":0.0015506206225213148}] | ++--------------------------------------+---------+---------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-numbers.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-numbers.md new file mode 100644 index 0000000000..26c2e1559e --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-numbers.md @@ -0,0 +1,20 @@ +--- +title: system.numbers +--- + +该表包含一个名为 number 的 UInt64 列,其中包含几乎所有从零开始的自然数。 + +您可以将此表用于测试,或者如果您需要进行暴力搜索。 + +从此表的读取也是并行化的。 + +用于测试。 + +```sql +SELECT avg(number) FROM numbers(100000000); ++-------------+ +| avg(number) | ++-------------+ +| 49999999.5 | ++-------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-cache.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-cache.md new file mode 100644 index 0000000000..38f2e2a7b4 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-cache.md @@ -0,0 +1,27 @@ +--- +title: system.query_cache +--- + +存储有关缓存查询结果的信息,包括查询 ID、结果大小和缓存数据位置等详细信息。 + +| 列 | 类型 | 描述 | +|----------------------|---------|-------------------------------------------------------------------| +| `sql` | String | 缓存查询的原始 SQL 文本。 | +| `query_id` | String | 缓存结果的查询的唯一标识符。 | +| `result_size` | UInt64 | 缓存结果文件的大小(以字节为单位)。 | +| `num_rows` | UInt64 | 缓存结果中的行数。 | +| `partitions_sha` | String | 用于验证缓存一致性的源分区哈希值。 | +| `location` | String | 缓存结果文件的存储位置(例如,内部路径)。 | +| `active_result_scan` | Boolean | 缓存结果当前是否被 `RESULT_SCAN` 使用。 | + +示例: + +```sql +SELECT * FROM system.query_cache; + +┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ sql │ query_id │ result_size │ num_rows │ partitions_sha │ location │ active_result_scan │ +├──────────────────┼──────────────────────────────────────┼─────────────┼──────────┼──────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────────────────┤ +│ SELECT * FROM t1 │ 39497827-9f20-464d-8389-0fb08129d9d4 │ 13 │ 3 │ 0756b2601aec1bccd8cc4b31f15692a993609364eead4595555933f2ec5f4f0d │ _result_cache/60050f5b0dc42f13b9803380b8dd576e582c66fcac68db0cdd1af915db166843/7468ce283bf9487fbc039b76c93047c1.parquet │ false │ +└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-log.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-log.md new file mode 100644 index 0000000000..34b47a973b --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-query-log.md @@ -0,0 +1,106 @@ +```md +--- +title: system.query_log +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +一个只读的内存表,用于存储所有查询日志。有关查询日志中可用的特定字段,请参阅[示例](#examples)部分。 + +## 设置会话标签 + +您可以选择为会话分配一个标签,以便更轻松地根据分配的会话标签在日志表中筛选日志。例如,以下代码将标签 `eric` 分配给当前会话: + +```sql +set session query_tag='eric'; +``` + +然后,我们可以运行一个查询,并使用分配的标签从日志表中检索日志: + +```sql +show users; + +select query_tag, query_text from system.query_log where query_tag='eric' limit 1; +``` + +在返回的结果中,您可以找到查询的记录,标记为 'eric': + +```sql +-[ RECORD 1 ]----------------------------------- + query_tag: eric + query_text: SHOW USERS +``` + +## 示例 + +`system.query_log` 表存储已执行查询的详细日志。以下是日志条目的示例: + +```sql +SELECT * FROM system.query_log; +``` + +```sql +... +-[ RECORD 10 ]----------------------------------- + log_type: 2 + log_type_name: Finish + handler_type: HTTPQuery + tenant_id: default + cluster_id: default + node_id: bwElPLZxWt0F8LOFAqsIL2 + sql_user: root + sql_user_quota: UserQuota + sql_user_privileges: GRANT ALL ON *.*, ROLES: ["account_admin"] + query_id: 1957f343-564e-471f-b036-a10ff853357e + query_kind: Other + query_text: SHOW USERS + query_hash: 63c65a28ae257d780e93a8a627190775 +query_parameterized_hash: 63c65a28ae257d780e93a8a627190775 + event_date: 2025-04-27 + event_time: 2025-04-27 16:59:31.681036 + query_start_time: 2025-04-27 16:59:31.671825 + query_duration_ms: 9 +query_queued_duration_ms: 0 + current_database: default + databases: + tables: + columns: + projections: + written_rows: 0 + written_bytes: 0 + join_spilled_rows: 0 + join_spilled_bytes: 0 + agg_spilled_rows: 0 + agg_spilled_bytes: 0 + group_by_spilled_rows: 0 + group_by_spilled_bytes: 0 + written_io_bytes: 0 +written_io_bytes_cost_ms: 0 + scan_rows: 1 + scan_bytes: 164 + scan_io_bytes: 0 + scan_io_bytes_cost_ms: 0 + scan_partitions: 0 + total_partitions: 0 + result_rows: 1 + result_bytes: 146 + cpu_usage: 8 + memory_usage: 0 + bytes_from_remote_disk: 0 + bytes_from_local_disk: 0 + bytes_from_memory: 0 + client_info: + client_address: 192.168.65.1 + user_agent: bendsql/0.24.1-f1f7de0 + exception_code: 0 + exception_text: + stack_trace: + server_version: v1.2.731-nightly-eb671da5e5(rust-1.88.0-nightly-2025-04-27T06:06:12.942159732Z) + query_tag: eric + session_settings: acquire_lock_timeout=30, aggregate_spilling_memory_ratio=60, auto_compaction_imperfect_blocks_threshold=25, auto_compaction_segments_limit=3, collation=utf8, compact_max_block_selection=10000, copy_dedup_full_path_by_default=0, cost_factor_aggregate_per_row=5, cost_factor_hash_table_per_row=10, cost_factor_network_per_row=50, create_query_flight_client_with_current_rt=1, data_retention_time_in_days=1, ddl_column_type_nullable=1, default_order_by_null=nulls_last, disable_join_reorder=0, disable_variant_check=0, dynamic_sample_time_budget_ms=0, efficiently_memory_group_by=0, enable_aggregating_index_scan=1, enable_analyze_histogram=0, enable_auto_fix_missing_bloom_index=0, enable_auto_vacuum=0, enable_block_stream_write=0, enable_bloom_runtime_filter=1, enable_cbo=1, enable_clickhouse_handler=0, enable_compact_after_multi_table_insert=0, enable_compact_after_write=1, enable_dio=1, enable_distributed_compact=0, enable_distributed_copy_into=1, enable_distributed_merge_into=1, enable_distributed_pruning=1, enable_distributed_recluster=0, enable_distributed_replace_into=0, enable_dphyp=1, enable_dst_hour_fix=0, enable_expand_roles=1, enable_experimental_aggregate_hashtable=1, enable_experimental_merge_into=1, enable_experimental_procedure=0, enable_experimental_queries_executor=0, enable_experimental_rbac_check=1, enable_geo_create_table=0, enable_hive_parquet_predict_pushdown=1, enable_last_snapshot_location_hint=1, enable_loser_tree_merge_sort=1, enable_materialized_cte=1, enable_merge_into_row_fetch=1, enable_new_copy_for_text_formats=1, enable_optimizer_trace=0, enable_parallel_multi_merge_sort=1, enable_parquet_page_index=1, enable_parquet_prewhere=0, enable_parquet_rowgroup_pruning=1, enable_planner_cache=1, enable_prune_cache=1, enable_prune_pipeline=1, enable_query_result_cache=0, enable_refresh_aggregating_index_after_write=1, enable_refresh_virtual_column_after_write=1, enable_replace_into_partitioning=1, enable_strict_datetime_parser=1, enable_table_lock=1, enforce_broadcast_join=0, enforce_shuffle_join=0, error_on_nondeterministic_update=1, external_server_connect_timeout_secs=10, external_server_request_batch_rows=65536, external_server_request_max_threads=256, external_server_request_retry_times=8, external_server_request_timeout_secs=180, flight_client_timeout=60, flight_connection_max_retry_times=0, flight_connection_retry_interval=1, force_aggregate_data_spill=0, force_join_data_spill=0, force_sort_data_spill=0, force_window_data_spill=0, format_null_as_str=1, geometry_output_format=GeoJSON, group_by_shuffle_mode=before_merge, group_by_two_level_threshold=20000, hide_options_in_show_create_table=1, hilbert_clustering_min_bytes=107374182400, hilbert_num_range_ids=1000, hilbert_sample_size_per_block=1000, hive_parquet_chunk_size=16384, http_handler_result_timeout_secs=60, idle_transaction_timeout_secs=14400, inlist_to_join_threshold=1024, input_read_buffer_size=4194304, join_spilling_buffer_threshold_per_proc_mb=512, join_spilling_memory_ratio=60, join_spilling_partition_bits=4, lazy_read_threshold=1000, load_file_metadata_expire_hours=24, max_block_size=65536, max_cte_recursive_depth=1000, max_execute_time_in_seconds=0, max_inlist_to_or=3, max_memory_usage=6574653440, max_push_down_limit=10000, max_query_memory_usage=0, max_result_rows=0, max_set_operator_count=18446744073709551615, max_spill_io_requests=48, max_storage_io_requests=48, max_threads=8, max_vacuum_temp_files_after_query=18446744073709551615, network_policy=, numeric_cast_option=rounding, optimizer_skip_list=, parquet_fast_read_bytes=16777216, parquet_max_block_size=8192, parse_datetime_ignore_remainder=1, persist_materialized_cte=0, prefer_broadcast_join=1, purge_duplicated_files_in_copy=0, query_flight_compression=LZ4, query_out_of_memory_behavior=throw, query_result_cache_allow_inconsistent=0, query_result_cache_max_bytes=1048576, query_result_cache_min_execute_secs=1, query_result_cache_ttl_secs=300, query_tag=eric, quoted_ident_case_sensitive=1, random_function_seed=0, recluster_block_size=1972396032, recluster_timeout_secs=43200, replace_into_bloom_pruning_max_column_number=4, replace_into_shuffle_strategy=0, sandbox_tenant=, script_max_steps=10000, short_sql_max_length=128, sort_spilling_batch_bytes=8388608, sort_spilling_memory_ratio=60, spilling_file_format=parquet, spilling_to_disk_vacuum_unknown_temp_dirs_limit=18446744073709551615, sql_dialect=PostgreSQL, statement_queue_ttl_in_seconds=15, statement_queued_timeout_in_seconds=0, storage_fetch_part_num=2, storage_io_max_page_bytes_for_read=524288, storage_io_min_bytes_for_seek=48, storage_read_buffer_size=1048576, stream_consume_batch_size_hint=0, table_lock_expire_secs=30, timezone=UTC, unquoted_ident_case_sensitive=0, use_parquet2=0, use_vacuum2_to_purge_transient_table_data=0, warehouse=, window_num_partitions=256, window_partition_sort_block_size=65536, window_partition_spilling_memory_ratio=60, window_partition_spilling_to_disk_bytes_limit=0, window_spill_unit_size_mb=256, scope: SESSION + extra: + has_profile: true + peek_memory_usage: {"bwElPLZxWt0F8LOFAqsIL2":576850} +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-settings.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-settings.md new file mode 100644 index 0000000000..4b0cd0e86e --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-settings.md @@ -0,0 +1,82 @@ +--- +title: system.settings +--- +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +存储当前会话的系统设置。 + +```sql +SELECT * FROM system.settings; +``` + +name |value |default |level |description |type | +--------------------------------------------+------------+------------+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+ +collation |binary |binary |SESSION|设置字符排序规则。可用值包括 "binary" 和 "utf8"。 |String| +ddl_column_type_nullable |1 |1 |SESSION|在创建或更改表时,如果列默认为可空。 |UInt64| +efficiently_memory_group_by |0 |0 |SESSION|高效地使用内存,但这可能会导致性能下降。 |UInt64| +enable_aggregating_index_scan |1 |1 |SESSION|查询时启用扫描聚合索引数据。 |UInt64| +enable_bushy_join |0 |0 |SESSION|允许优化器生成 bushy join 计划。 |UInt64| +enable_cbo |1 |1 |SESSION|启用基于成本的优化。 |UInt64| +enable_distributed_compact |0 |0 |SESSION|启用表的分布式压缩。 |UInt64| +enable_distributed_copy_into |0 |0 |SESSION|启用 COPY INTO 的分布式执行。 |UInt64| +enable_distributed_merge_into |0 |0 |SESSION|启用分布式 MERGE INTO。 |UInt64| +enable_distributed_recluster |0 |0 |SESSION|启用表的分布式重新聚类。 |UInt64| +enable_distributed_replace_into |0 |0 |SESSION|启用分布式 REPLACE INTO。 |UInt64| +enable_dphyp |1 |1 |SESSION|启用 dphyp join 顺序算法。 |UInt64| +enable_experimental_merge_into |0 |0 |SESSION|启用实验性的 MERGE INTO。 |UInt64| +enable_hive_parquet_predict_pushdown |1 |1 |SESSION|通过将此变量设置为 1 来启用 hive parquet predict 下推,默认值:1 |UInt64| +enable_parquet_page_index |1 |1 |SESSION|启用 parquet 页面索引。 |UInt64| +enable_parquet_prewhere |0 |0 |SESSION|启用 parquet prewhere。 |UInt64| +enable_parquet_rowgroup_pruning |1 |1 |SESSION|启用 parquet rowgroup 剪裁。 |UInt64| +enable_query_profiling |0 |0 |SESSION|启用查询 profile 记录。 |UInt64| +enable_query_result_cache |0 |0 |SESSION|启用缓存查询结果以提高相同查询的性能。 |UInt64| +enable_recluster_after_write |1 |1 |SESSION|启用写入后重新聚类(copy/replace-into)。 |UInt64| +enable_refresh_aggregating_index_after_write|0 |0 |SESSION|写入新数据后刷新聚合索引。 |UInt64| +enable_replace_into_bloom_pruning |1 |1 |SESSION|为 replace-into 语句启用 bloom 剪裁。 |UInt64| +enable_replace_into_partitioning |1 |1 |SESSION|为 replace-into 语句启用分区(如果表有 cluster keys)。 |UInt64| +enable_runtime_filter |0 |0 |SESSION|为 JOIN 启用运行时过滤器优化。 |UInt64| +enable_table_lock |1 |1 |SESSION|如果需要,启用表锁(默认启用)。 |UInt64| +flight_client_timeout |60 |60 |SESSION|设置 flight 客户端请求可以处理的最长时间(秒)。 |UInt64| +group_by_shuffle_mode |before_merge|before_merge|SESSION|Group by shuffle 模式,'before_partial' 更加平衡,但需要交换更多数据。 |String| +group_by_two_level_threshold |20000 |20000 |SESSION|设置 GROUP BY 操作中将触发两级聚合的键的数量。 |UInt64| +hide_options_in_show_create_table |1 |1 |SESSION|隐藏表相关信息,例如 SNAPSHOT_LOCATION 和 STORAGE_FORMAT,在 SHOW TABLE CREATE 结果的末尾。 |UInt64| +hive_parquet_chunk_size |16384 |16384 |SESSION|每次从 parquet 读取到 Databend 处理器的最大行数。 |UInt64| +input_read_buffer_size |4194304 |4194304 |SESSION|设置分配给缓冲读取器用于从存储读取数据的缓冲区的内存大小(以字节为单位)。 |UInt64| +join_spilling_threshold |0 |0 |SESSION|hash join 可以使用的最大内存量,0 表示无限制。 |UInt64| +lazy_read_threshold |1000 |1000 |SESSION|设置查询中启用延迟读取优化的最大 LIMIT。将其设置为 0 将禁用优化。 |UInt64| +load_file_metadata_expire_hours |12 |12 |SESSION|设置使用 COPY INTO 从中加载数据的文件的元数据过期的时长(以小时为单位)。 |UInt64| +max_block_size |65536 |65536 |SESSION|设置可以读取的单个数据块的最大字节大小。 |UInt64| +max_execute_time_in_seconds |0 |0 |SESSION|设置最大查询执行时间(以秒为单位)。将其设置为 0 表示没有限制。 |UInt64| +max_inlist_to_or |3 |3 |SESSION|设置可以包含在 IN 表达式中以转换为 OR 运算符的最大值数量。 |UInt64| +max_memory_usage |6871947673 |6871947673 |SESSION|设置处理单个查询的最大内存使用量(以字节为单位)。 |UInt64| +max_result_rows |0 |0 |SESSION|设置在未指定特定行数时,查询结果中可以返回的最大行数。将其设置为 0 表示没有限制。 |UInt64| +max_storage_io_requests |48 |48 |SESSION|设置最大并发 I/O 请求数。 |UInt64| +max_threads |8 |8 |SESSION|设置执行请求的最大线程数。 |UInt64| +numeric_cast_option |rounding |rounding |SESSION|将数字转换模式设置为 "rounding" 或 "truncating"。 |String| +parquet_fast_read_bytes |0 |0 |SESSION|较小的 Parquet 文件将被读取为整个文件,而不是逐列读取。 |UInt64| +parquet_uncompressed_buffer_size |2097152 |2097152 |SESSION|设置用于读取 Parquet 文件的缓冲区的大小(以字节为单位)。 |UInt64| +prefer_broadcast_join |1 |1 |SESSION|启用 broadcast join。 |UInt64| +query_result_cache_allow_inconsistent |0 |0 |SESSION|确定 Databend 是否将返回与底层数据不一致的缓存查询结果。 |UInt64| +query_result_cache_max_bytes |1048576 |1048576 |SESSION|设置单个查询结果的缓存的最大字节大小。 |UInt64| +query_result_cache_ttl_secs |300 |300 |SESSION|设置缓存查询结果的生存时间 (TTL)(以秒为单位)。一旦缓存结果的 TTL 过期,该结果将被视为陈旧,并且不会用于新查询。 |UInt64| +quoted_ident_case_sensitive |1 |1 |SESSION|确定 Databend 是否将带引号的标识符视为区分大小写。 |UInt64| +recluster_block_size |2405181685 |2405181685 |SESSION|设置重新聚类的最大块字节大小。 |UInt64| +recluster_timeout_secs |43200 |43200 |SESSION|设置重新聚类最终超时的时间(以秒为单位)。 |UInt64| +replace_into_bloom_pruning_max_column_number|4 |4 |SESSION|replace-into 语句的 bloom 剪裁使用的最大列数。 |UInt64| +replace_into_shuffle_strategy |0 |0 |SESSION|0 表示块级别 shuffle,1 表示段级别 shuffle。 |UInt64| +retention_period |12 |12 |SESSION|设置保留期限(以小时为单位)。 |UInt64| +sandbox_tenant | | |SESSION|将自定义 'sandbox_tenant' 注入到此会话中。这仅用于测试目的,并且仅当启用 'internal_enable_sandbox_tenant' 时才生效。 |String| +spilling_bytes_threshold_per_proc |0 |0 |SESSION|设置聚合器在查询执行期间将数据溢出到存储之前可以使用的最大内存量(以字节为单位)。 |UInt64| +spilling_memory_ratio |0 |0 |SESSION|设置聚合器在查询执行期间将数据溢出到存储之前可以使用的最大内存比率(以字节为单位)。 |UInt64| +sql_dialect |PostgreSQL |PostgreSQL |SESSION|设置 SQL 方言。可用值包括 "PostgreSQL"、"MySQL" 和 "Hive"。 |String| +storage_fetch_part_num |2 |2 |SESSION|设置在查询执行期间从存储并行提取的分区数。 |UInt64| +storage_io_max_page_bytes_for_read |524288 |524288 |SESSION|设置可以在单个 I/O 操作中从存储读取的数据页面的最大字节大小。 |UInt64| +storage_io_min_bytes_for_seek |48 |48 |SESSION|设置在数据文件中查找新位置时,必须在单个 I/O 操作中从存储读取的最小数据字节大小。 |UInt64| +storage_read_buffer_size |1048576 |1048576 |SESSION|设置用于将数据读入内存的缓冲区的大小(以字节为单位)。 |UInt64| +table_lock_expire_secs |5 |5 |SESSION|设置表锁过期的秒数。 |UInt64| +timezone |Japan |UTC |GLOBAL |设置时区。 |String| +unquoted_ident_case_sensitive |0 |0 |SESSION|确定 Databend 是否将未带引号的标识符视为区分大小写。 |UInt64| +use_parquet2 |1 |1 |SESSION|当 infer_schema() 时,使用 parquet2 而不是 parquet_rs。 |UInt64| +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-streams.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-streams.md new file mode 100644 index 0000000000..2f71555066 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-streams.md @@ -0,0 +1,20 @@ +--- +title: system.streams +--- +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +提供系统中 stream 的信息。此表中的每一行对应一个 stream,列包含 stream 的模式、注释(如果有)、关联的表名、表 ID、表版本、快照位置、无效原因(如果适用)和所有者等详细信息。 + +```sql title="Example:" +SELECT * FROM system.streams; + +┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ catalog │ database │ name │ stream_id │ created_on │ updated_on │ mode │ comment │ table_name │ table_id │ table_version │ snapshot_location │ invalid_reason │ owner │ +├─────────┼─────────────┼────────┼───────────┼────────────────────────────┼────────────────────────────┼─────────────┼──────────────────┼───────────────┼──────────┼───────────────┼───────────────────────────────────────────────────────┼────────────────┼──────────────────┤ +│ default │ test_stream │ s │ 3290 │ 2023-11-28 16:27:16.404667 │ 2023-11-28 16:37:15.761127 │ append_only │ │ test_stream.t │ 3272 │ 3342 │ 3264/3272/_ss/f143831d621c4c7a80f782a5f2ee2338_v4.mpk │ │ NULL │ +│ default │ test_stream │ s1 │ 3300 │ 2023-11-28 16:28:12.381539 │ 2023-11-28 16:28:12.381539 │ append_only │ │ test_stream.t │ 3272 │ 3298 │ 3264/3272/_ss/6e6a0a94b2a344e3b7eeff16fe6996dc_v4.mpk │ │ NULL │ +│ default │ test_stream │ s2 │ 3318 │ 2023-11-28 16:33:17.195153 │ 2023-11-28 16:33:17.195154 │ append_only │ this is a stream │ test_stream.t │ 3272 │ 3282 │ 3264/3272/_ss/a746f1cdbea84a3a91510a7118475b7d_v4.mpk │ │ NULL │ +└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-table-functions.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-table-functions.md new file mode 100644 index 0000000000..fe393ebc50 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-table-functions.md @@ -0,0 +1,37 @@ +--- +title: system.table_functions +--- + +获取所有表函数的名称。 + +```sql +SELECT * FROM system.table_functions; +``` + +```text ++----------------------------+ +| name | ++----------------------------+ +| numbers | +| numbers_mt | +| numbers_local | +| fuse_snapshot | +| fuse_segment | +| fuse_block | +| fuse_column | +| fuse_statistic | +| clustering_information | +| sync_crash_me | +| async_crash_me | +| infer_schema | +| list_stage | +| generate_series | +| range | +| ai_to_sql | +| execute_background_job | +| license_info | +| suggested_background_tasks | +| tenant_quota | +| fuse_encoding | ++----------------------------+ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables-with-history.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables-with-history.md new file mode 100644 index 0000000000..bd563b57d2 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables-with-history.md @@ -0,0 +1,52 @@ +--- +title: system.tables_with_history +--- + +提供所有表的元数据信息,包括历史表。它包括表属性、创建时间、行数、数据大小等详细信息。 + +另请参阅:[SHOW DROP TABLES](../../10-sql-commands/00-ddl/01-table/show-drop-tables.md) + +```sql title='Examples:' +SELECT * FROM system.tables_with_history LIMIT 3; + +┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ catalog │ database │ name │ table_id │ total_columns │ engine │ engine_full │ cluster_by │ is_transient │ is_attach │ created_on │ dropped_on │ updated_on │ num_rows │ data_size │ data_compressed_size │ index_size │ number_of_segments │ number_of_blocks │ owner │ comment │ table_type │ +├─────────┼──────────┼────────────────────┼─────────────────────┼───────────────┼───────────────────────┼───────────────────────┼────────────┼──────────────┼───────────┼────────────────────────────┼─────────────────────┼────────────────────────────┼──────────────────┼──────────────────┼──────────────────────┼──────────────────┼────────────────────┼──────────────────┼──────────────────┼─────────┼────────────┤ +│ default │ system │ metrics │ 4611686018427387919 │ 5 │ SystemMetrics │ SystemMetrics │ │ │ │ 2024-11-20 21:04:12.950815 │ NULL │ 2024-11-20 21:04:12.950815 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +│ default │ system │ clustering_history │ 4611686018427387925 │ 6 │ SystemLogTable │ SystemLogTable │ │ │ │ 2024-11-20 21:04:12.952439 │ NULL │ 2024-11-20 21:04:12.952439 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +│ default │ system │ queries_profiling │ 4611686018427387941 │ 7 │ QueriesProfilingTable │ QueriesProfilingTable │ │ │ │ 2024-11-20 21:04:12.952588 │ NULL │ 2024-11-20 21:04:12.952588 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` + +要显示 `system.tables_with_history` 的 schema,请使用 `DESCRIBE system.tables_with_history`: + +```sql +DESCRIBE system.tables_with_history; + +┌─────────────────────────────────────────────────────────────────────────────────────────┐ +│ Field │ Type │ Null │ Default │ Extra │ +├──────────────────────┼─────────────────┼────────┼──────────────────────────────┼────────┤ +│ catalog │ VARCHAR │ NO │ '' │ │ +│ database │ VARCHAR │ NO │ '' │ │ +│ name │ VARCHAR │ NO │ '' │ │ +│ table_id │ BIGINT UNSIGNED │ NO │ 0 │ │ +│ total_columns │ BIGINT UNSIGNED │ NO │ 0 │ │ +│ engine │ VARCHAR │ NO │ '' │ │ +│ engine_full │ VARCHAR │ NO │ '' │ │ +│ cluster_by │ VARCHAR │ NO │ '' │ │ +│ is_transient │ VARCHAR │ NO │ '' │ │ +│ is_attach │ VARCHAR │ NO │ '' │ │ +│ created_on │ TIMESTAMP │ NO │ '1970-01-01 00:00:00.000000' │ │ +│ dropped_on │ TIMESTAMP │ YES │ NULL │ │ +│ updated_on │ TIMESTAMP │ NO │ '1970-01-01 00:00:00.000000' │ │ +│ num_rows │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ data_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ data_compressed_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ index_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ number_of_segments │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ number_of_blocks │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ owner │ VARCHAR │ YES │ NULL │ │ +│ comment │ VARCHAR │ NO │ '' │ │ +│ table_type │ VARCHAR │ NO │ '' │ │ +└─────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables.md new file mode 100644 index 0000000000..8dfdf719ea --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-tables.md @@ -0,0 +1,55 @@ +--- +title: system.tables +--- + +提供所有表的元数据信息。它包括表属性、创建时间、行数、数据大小等详细信息。 + +另请参阅: + +- [information_schema.tables](information-schema-tables.md) +- [SHOW TABLES](../../10-sql-commands/00-ddl/01-table/show-tables.md) + +```sql title='Examples:' +SELECT * FROM system.tables LIMIT 3; + +┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ catalog │ database │ name │ table_id │ total_columns │ engine │ engine_full │ cluster_by │ is_transient │ is_attach │ created_on │ dropped_on │ updated_on │ num_rows │ data_size │ data_compressed_size │ index_size │ number_of_segments │ number_of_blocks │ owner │ comment │ table_type │ +├─────────┼──────────┼────────────────────┼─────────────────────┼───────────────┼───────────────────────┼───────────────────────┼────────────┼──────────────┼───────────┼────────────────────────────┼─────────────────────┼────────────────────────────┼──────────────────┼──────────────────┼──────────────────────┼──────────────────┼────────────────────┼──────────────────┼──────────────────┼─────────┼────────────┤ +│ default │ system │ metrics │ 4611686018427387919 │ 5 │ SystemMetrics │ SystemMetrics │ │ │ │ 2024-11-20 21:04:12.950815 │ NULL │ 2024-11-20 21:04:12.950815 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +│ default │ system │ clustering_history │ 4611686018427387925 │ 6 │ SystemLogTable │ SystemLogTable │ │ │ │ 2024-11-20 21:04:12.952439 │ NULL │ 2024-11-20 21:04:12.952439 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +│ default │ system │ queries_profiling │ 4611686018427387941 │ 7 │ QueriesProfilingTable │ QueriesProfilingTable │ │ │ │ 2024-11-20 21:04:12.952588 │ NULL │ 2024-11-20 21:04:12.952588 │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ NULL │ │ BASE TABLE │ +└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` + +要显示 `system.tables` 的 schema,请使用 `DESCRIBE system.tables`: + +```sql +DESCRIBE system.tables; + +┌─────────────────────────────────────────────────────────────────────────────────────────┐ +│ Field │ Type │ Null │ Default │ Extra │ +├──────────────────────┼─────────────────┼────────┼──────────────────────────────┼────────┤ +│ catalog │ VARCHAR │ NO │ '' │ │ +│ database │ VARCHAR │ NO │ '' │ │ +│ name │ VARCHAR │ NO │ '' │ │ +│ table_id │ BIGINT UNSIGNED │ NO │ 0 │ │ +│ total_columns │ BIGINT UNSIGNED │ NO │ 0 │ │ +│ engine │ VARCHAR │ NO │ '' │ │ +│ engine_full │ VARCHAR │ NO │ '' │ │ +│ cluster_by │ VARCHAR │ NO │ '' │ │ +│ is_transient │ VARCHAR │ NO │ '' │ │ +│ is_attach │ VARCHAR │ NO │ '' │ │ +│ created_on │ TIMESTAMP │ NO │ '1970-01-01 00:00:00.000000' │ │ +│ dropped_on │ TIMESTAMP │ YES │ NULL │ │ +│ updated_on │ TIMESTAMP │ NO │ '1970-01-01 00:00:00.000000' │ │ +│ num_rows │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ data_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ data_compressed_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ index_size │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ number_of_segments │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ number_of_blocks │ BIGINT UNSIGNED │ YES │ NULL │ │ +│ owner │ VARCHAR │ YES │ NULL │ │ +│ comment │ VARCHAR │ NO │ '' │ │ +│ table_type │ VARCHAR │ NO │ '' │ │ +└─────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-files.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-files.md new file mode 100644 index 0000000000..2a0a95cabc --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-files.md @@ -0,0 +1,68 @@ +--- +title: system.temp_files +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含有关 Databend 创建的临时文件的信息,例如溢出文件。要删除临时文件,请使用 [VACUUM TEMPORARY FILES](/sql/sql-commands/administration-cmds/vacuum-temp-files) 命令。 + +```sql +SELECT * FROM system.temp_files; +``` + +``` +┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ file_type │ file_name │ file_content_length │ file_last_modified_time │ +├───────────┼─────────────────────────────────────────────────────────────┼─────────────────────┼─────────────────────────┤ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/0tUE8EqsxxjO4ftZA8Zni6 │ 591239232 │ 2024-11-19 03:06:03 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/440NKJwbRrW8HCFfQuNmb4 │ 607193920 │ 2024-11-19 03:05:18 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/5oQxtCB58oRhTA7EgO3027 │ 787631104 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/7nBLkWobl4jaDQtROAIow1 │ 596923264 │ 2024-11-19 03:06:04 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/Dx1xSJ5kv5vZyoWdmSpe32 │ 780189824 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/KSHXnVch2KUbHCqE0rgpx7 │ 741196608 │ 2024-11-19 03:05:21 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/O3TvRQja41NrpME8qXjJE3 │ 792868608 │ 2024-11-19 03:06:06 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/OMXTEzXmvR5Zw3jk2BVlR5 │ 661675392 │ 2024-11-19 03:05:19 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/PkSYoCjNxwDqCwP3k0axs1 │ 797124864 │ 2024-11-19 03:04:33 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/Sdr4ew2l60k90e7zZs3mF │ 797046144 │ 2024-11-19 03:05:21 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/TPMpuE1ypRSwRiSx2bRhh6 │ 531469504 │ 2024-11-19 03:06:03 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/VdzW61PcSugFIGyCR4B6P6 │ 736063616 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/WcTI2vVUfyzy8XUyjQAhc2 │ 791146496 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/Y6cCfDUkIkeD7Mnm0Zut67 │ 738694976 │ 2024-11-19 03:06:05 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/eNACGmJy00y8Pr1xSPCT25 │ 790728256 │ 2024-11-19 03:05:21 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/fojFryduQMoru0kAwnzys5 │ 795929344 │ 2024-11-19 03:05:20 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/iWcusSG1zW0pnbo76j0vr7 │ 797382080 │ 2024-11-19 03:06:04 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/klWl0CxOZQ08IHymUdeHr │ 796787712 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/kyX7EzdFBOVEBDNwKtexC6 │ 743725184 │ 2024-11-19 03:05:20 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/qDOtHrPpdpPPxqqb2Ybht7 │ 794764672 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/qp7GnofqSBZXnJrFuuxqa6 │ 797497664 │ 2024-11-19 03:06:02 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/w2qbixYObBOaMlgk7IQms1 │ 716091520 │ 2024-11-19 03:06:05 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/xO0ozrAKnq0naJ85BO53I4 │ 779609664 │ 2024-11-19 03:04:34 │ +│ Spill │ 1e9411e1-3c2f-48ee-9712-9d3ce396d1b3/xpejvCn9HNGJOc0szcC5b4 │ 793506112 │ 2024-11-19 03:05:21 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/2Gk62gm2GgFSLjTIGJbWv6 │ 796510336 │ 2024-11-19 03:01:55 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/2iq9gvwTVgpyp4CSQhimY3 │ 613255680 │ 2024-11-19 03:01:52 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/6gwj6vG0FDPLbBPIfcBDK │ 750276224 │ 2024-11-19 03:02:36 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/FXMxh0kA9W6QM5gmJizr92 │ 640907328 │ 2024-11-19 03:02:35 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/JQgHSgRphfQNtty8iZYGV5 │ 592456704 │ 2024-11-19 03:02:35 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/KZh2d7Av3UgfFu63dLKyh7 │ 639652608 │ 2024-11-19 03:01:53 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/L2NiIGBOUUsWHwqhqYVni5 │ 767536768 │ 2024-11-19 03:01:07 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/MG1fGMXMqlX9x1iQhUHQr4 │ 750610560 │ 2024-11-19 03:01:54 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/ObfLpkM6boMbeZHvckEGy3 │ 796252032 │ 2024-11-19 03:02:36 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/QuP4XULWwFtw1eWN9wYPf4 │ 664751936 │ 2024-11-19 03:02:35 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/Qy1i0PXzXMmJQ5DYWqfhN5 │ 789342016 │ 2024-11-19 03:01:08 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/UblG5Do43sJ4eRrr7Jh2O5 │ 796920768 │ 2024-11-19 03:01:07 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/UvmDHe3hIAO1uGubX0O8K3 │ 789434112 │ 2024-11-19 03:01:55 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/UxEvvrlyQdZws1Ou8Qhy62 │ 595680768 │ 2024-11-19 03:02:34 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/ZrbdJT9xOcsHjExj6wCum3 │ 796707456 │ 2024-11-19 03:01:53 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/e2sY1RKXFsHtNFx4PkFxT1 │ 753388160 │ 2024-11-19 03:01:54 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/fnQanOb1s6OmwJtooi35K5 │ 796730688 │ 2024-11-19 03:01:08 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/hHSrrfmMjtnPP4gbKBuTc2 │ 764452672 │ 2024-11-19 03:01:07 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/pZ06OYSniDzOLp8vTtruQ6 │ 796523712 │ 2024-11-19 03:02:37 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/rPdRFZz6g3NlhIlbnk5b16 │ 790081408 │ 2024-11-19 03:01:07 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/sxOeDcjyKRrahkCHtAbeG2 │ 605099776 │ 2024-11-19 03:01:52 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/v1PU6oEHSSEieXL7mceqz5 │ 682199616 │ 2024-11-19 03:02:36 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/xRRUU3sKkjyFAKv4LDuUt2 │ 786471232 │ 2024-11-19 03:01:08 │ +│ Spill │ 71c1bdcd-8ebb-45c1-98af-2700df2f9e10/ys9yCVJNSRBY73ce46bCA6 │ 747938176 │ 2024-11-19 03:01:07 │ +└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-tables.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-tables.md new file mode 100644 index 0000000000..d0f2d88c47 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-temp-tables.md @@ -0,0 +1,18 @@ +--- +title: system.temporary_tables +--- +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +提供有关当前会话中所有现有临时表的信息。 + +```sql title='Examples:' +SELECT * FROM system.temporary_tables; + +┌────────────────────────────────────────────────────┐ +│ database │ name │ table_id │ engine │ +├──────────┼──────────┼─────────────────────┼────────┤ +│ default │ my_table │ 4611686018427407904 │ FUSE │ +└────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-user-functions.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-user-functions.md new file mode 100644 index 0000000000..7d9edd7fce --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-user-functions.md @@ -0,0 +1,24 @@ +--- +title: system.user_functions +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含系统中用户定义函数和外部函数的信息。 + +另请参阅:[SHOW USER FUNCTIONS](/sql/sql-commands/administration-cmds/show-user-functions)。 + +```sql +SELECT * FROM system.user_functions; + + +┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ name │ is_aggregate │ description │ arguments │ language │ definition │ +├────────────────┼───────────────────┼─────────────┼───────────────────────────────────────────────────────────┼──────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────┤ +│ binary_reverse │ NULL │ │ {"arg_types":["Binary NULL"],"return_type":"Binary NULL"} │ python │ (Binary NULL) RETURNS Binary NULL LANGUAGE python HANDLER = binary_reverse ADDRESS = http://0.0.0.0:8815 │ +│ echo │ NULL │ │ {"arg_types":["String NULL"],"return_type":"String NULL"} │ python │ (String NULL) RETURNS String NULL LANGUAGE python HANDLER = echo ADDRESS = http://0.0.0.0:8815 │ +│ isnotempty │ NULL │ │ {"parameters":["p"]} │ SQL │ (p) -> (NOT is_null(p)) │ +└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-virtual-columns.md b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-virtual-columns.md new file mode 100644 index 0000000000..8f4ee0cb52 --- /dev/null +++ b/docs/cn/sql-reference/00-sql-reference/31-system-tables/system-virtual-columns.md @@ -0,0 +1,21 @@ +--- +title: system.virtual_columns +--- + +import FunctionDescription from '@site/src/components/FunctionDescription'; + + + +包含系统中已创建的 virtual column 的信息。 + +另请参阅:[SHOW VIRTUAL COLUMNS](../../10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md) + +```sql +SELECT * FROM system.virtual_columns; + +┌───────────────────────────────────────────────────────────────────────────────────────────────┐ +│ database │ table │ virtual_columns │ created_on │ updated_on │ +├──────────┼────────┼─────────────────┼────────────────────────────┼────────────────────────────┤ +│ default │ test │ val['name'] │ 2023-12-25 21:24:26.127790 │ 2023-12-25 21:24:38.455268 │ +└───────────────────────────────────────────────────────────────────────────────────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md index 433010a6db..5d6c12e8fa 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; 显示实例上存在的数据库列表。 -另请参阅:[system.databases](../../../00-sql-reference/20-system-tables/system-databases.md) +另请参阅:[system.databases](../../../00-sql-reference/31-system-tables/system-databases.md) ## 语法 @@ -17,11 +17,11 @@ SHOW [ FULL ] DATABASES [ LIKE '' | WHERE ] ``` -| 参数 | 描述 | +| 参数 | 描述 | |-----------|-----------------------------------------------------------------------------------------------------------------------------| -| FULL | 列出包含附加信息的结果。 更多详细信息请参见 [示例](#examples)。 | -| LIKE | 使用区分大小写的模式匹配按名称筛选结果。 | -| WHERE | 使用 WHERE 子句中的表达式筛选结果。 | +| FULL | 列出包含附加信息的结果。 有关更多详细信息,请参见[示例](#examples)。 | +| LIKE | 使用区分大小写的模式匹配按名称过滤结果。 | +| WHERE | 使用 WHERE 子句中的表达式过滤结果。 | ## 示例 diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md index caf4611e69..0c370afd84 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md @@ -10,7 +10,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; - 只有在数据保留期内的已删除数据库才能被检索。 - 建议使用 `root` 等管理员用户。如果您使用的是 Databend Cloud,请使用具有 `account_admin` 角色的用户来查询已删除的数据库。 -另请参阅:[system.databases_with_history](../../../00-sql-reference/20-system-tables/system-databases-with-history.md) +另请参阅:[system.databases_with_history](../../../00-sql-reference/31-system-tables/system-databases-with-history.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md index de68592620..b3f12d303f 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md @@ -10,10 +10,10 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; - 临时表仅在创建它的会话中可见,并且在会话结束时自动删除,所有数据都会被清理。 - 如果临时表的自动清理失败(例如,由于查询节点崩溃),您可以使用 [FUSE_VACUUM_TEMPORARY_TABLE](../../../20-sql-functions/17-table-functions/fuse-vacuum-temporary-table.md) 函数手动清理临时表的剩余文件。 -- 要显示会话中现有的临时表,请查询 [system.temporary_tables](../../../00-sql-reference/20-system-tables/system-temp-tables.md) 系统表。请参见 [Example-1](#example-1)。 -- 与普通表同名的临时表优先,在删除之前会隐藏普通表。请参见 [Example-2](#example-2)。 +- 要显示会话中现有的临时表,请查询 [system.temporary_tables](../../../00-sql-reference/31-system-tables/system-temp-tables.md) 系统表。请参见 [Example-1](#example-1)。 +- 与普通表同名的临时表优先,隐藏普通表直到被删除。请参见 [Example-2](#example-2)。 - 创建或操作临时表不需要任何权限。 -- Databend 支持使用 [Fuse Engine](../../../00-sql-reference/30-table-engines/00-fuse.md) 和 [Memory Engine](../../../00-sql-reference/30-table-engines/01-memory.md) 创建临时表。 +- Databend 支持使用 [Fuse Engine](../../../00-sql-reference/30-table-engines/00-fuse.md) 创建临时表。 - 要使用 BendSQL 创建临时表,请确保您使用的是最新版本的 BendSQL。 ## 语法 @@ -31,7 +31,7 @@ CREATE [ OR REPLACE ] { TEMPORARY | TEMP } TABLE ### Example-1 -此示例演示如何创建一个临时表,并通过查询 [system.temporary_tables](../../../00-sql-reference/20-system-tables/system-temp-tables.md) 系统表来验证其存在: +此示例演示如何创建临时表,并通过查询 [system.temporary_tables](../../../00-sql-reference/31-system-tables/system-temp-tables.md) 系统表来验证其存在: ```sql CREATE TEMP TABLE my_table (id INT, description STRING); @@ -47,7 +47,7 @@ SELECT * FROM system.temporary_tables; ### Example-2 -此示例演示了与普通表同名的临时表如何优先。当两个表都存在时,操作目标是临时表,从而有效地隐藏了普通表。删除临时表后,可以再次访问普通表: +此示例演示了与普通表同名的临时表如何优先。当两个表都存在时,操作会针对临时表,从而有效地隐藏普通表。一旦临时表被删除,普通表就可以再次访问: ```sql -- Create a normal table diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md index 936131d0d6..35f3e201f1 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md @@ -4,11 +4,11 @@ sidebar_position: 11 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + 列出当前或指定数据库中已删除的表。 -另请参阅:[system.tables_with_history](../../../00-sql-reference/20-system-tables/system-tables-with-history.md) +另请参阅:[system.tables_with_history](../../../00-sql-reference/31-system-tables/system-tables-with-history.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md index 92889bd2a5..8d5a28959e 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md @@ -12,7 +12,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; 从 1.2.415 版本开始,SHOW TABLES 命令不再在其结果中包含视图。要显示视图,请改用 [SHOW VIEWS](../05-view/show-views.md)。 ::: -另请参阅:[system.tables](../../../00-sql-reference/20-system-tables/system-tables.md) +另请参阅:[system.tables](../../../00-sql-reference/31-system-tables/system-tables.md) ## 语法 @@ -23,5 +23,97 @@ SHOW [ FULL ] TABLES [ LIKE '' | WHERE ] ``` -| 参数 | 描述 -| FULL | 列出包含附加信息的结果。 请参阅[示例](#examples)以获取更多详细信息。 \ No newline at end of file +| 参数 | 描述 | +|-----------|-----------------------------------------------------------------------------------------------------------------------------| +| FULL | 列出包含附加信息的结果。 有关更多详细信息,请参见 [示例](#examples)。 | +| FROM / IN | 指定数据库。 如果省略,则该命令将返回当前数据库的结果。 | +| HISTORY | 显示保留期内(默认为 24 小时)表删除的时间戳。 如果尚未删除表,则 `drop_time` 的值为 NULL。 | +| LIKE | 使用区分大小写的模式匹配按名称过滤结果。 | +| WHERE | 使用 WHERE 子句中的表达式过滤结果。 | + +## 示例 + +以下示例列出了当前数据库(默认)中所有表的名称: + +```sql +SHOW TABLES; + +┌───────────────────┐ +│ Tables_in_default │ +├───────────────────┤ +│ books │ +│ mytable │ +│ ontime │ +│ products │ +└───────────────────┘ +``` + +以下示例列出了包含附加信息的所有表: + +```sql +SHOW FULL TABLES; + +┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ +│ tables │ table_type │ database │ catalog │ owner │ engine │ cluster_by │ create_time │ num_rows │ data_size │ data_compressed_size │ index_size │ +├──────────┼────────────┼──────────┼─────────┼──────────────────┼────────┼────────────┼────────────────────────────┼──────────────────┼──────────────────┼──────────────────────┼──────────────────┤ +│ books │ BASE TABLE │ default │ default │ account_admin │ FUSE │ │ 2024-01-16 03:53:15.354132 │ 0 │ 0 │ 0 │ 0 │ +│ mytable │ BASE TABLE │ default │ default │ account_admin │ FUSE │ │ 2024-01-16 03:53:27.968505 │ 0 │ 0 │ 0 │ 0 │ +│ ontime │ BASE TABLE │ default │ default │ account_admin │ FUSE │ │ 2024-01-16 03:53:42.052399 │ 0 │ 0 │ 0 │ 0 │ +│ products │ BASE TABLE │ default │ default │ account_admin │ FUSE │ │ 2024-01-16 03:54:00.883985 │ 0 │ 0 │ 0 │ 0 │ +└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ +``` + +以下示例演示了当存在可选参数 HISTORY 时,结果将包括已删除的表: + +```sql +DROP TABLE products; + +SHOW TABLES; + +┌───────────────────┐ +│ Tables_in_default │ +├───────────────────┤ +│ books │ +│ mytable │ +│ ontime │ +└───────────────────┘ + +SHOW TABLES HISTORY; + +┌────────────────────────────────────────────────┐ +│ Tables_in_default │ drop_time │ +├───────────────────┼────────────────────────────┤ +│ books │ NULL │ +│ mytable │ NULL │ +│ ontime │ NULL │ +│ products │ 2024-01-16 03:55:47.900362 │ +└────────────────────────────────────────────────┘ +``` + +以下示例列出了名称末尾包含字符串“time”的表: + +```sql +SHOW TABLES LIKE '%time'; + +┌───────────────────┐ +│ Tables_in_default │ +├───────────────────┤ +│ ontime │ +└───────────────────┘ + +-- CASE-SENSITIVE pattern matching. +-- No results will be returned if you code the previous statement like this: +SHOW TABLES LIKE '%TIME'; +``` + +以下示例列出了数据大小大于 1,000 字节的表: + +```sql +SHOW TABLES WHERE data_size > 1000 ; + +┌───────────────────┐ +│ Tables_in_default │ +├───────────────────┤ +│ ontime │ +└───────────────────┘ +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md index abff93cd3f..06129aaacc 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; 显示已创建的聚合索引。等效于 `SELECT * FROM system.indexes`。 -另请参阅:[system.indexes](../../../00-sql-reference/20-system-tables/system-indexes.md) +另请参见:[system.indexes](../../../00-sql-reference/31-system-tables/system-indexes.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md index a419a07535..de4959637b 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md @@ -10,9 +10,9 @@ import EEFeature from '@site/src/components/EEFeature'; -显示系统中创建的 virtual column。等效于 `SELECT * FROM system.virtual_columns`。 +显示系统中已创建的 virtual column。等效于 `SELECT * FROM system.virtual_columns`。 -另请参阅:[system.virtual_columns](../../../00-sql-reference/20-system-tables/system-virtual-columns.md) +另请参阅:[system.virtual_columns](../../../00-sql-reference/31-system-tables/system-virtual-columns.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md index 3d33f3b513..163eeda7e5 100644 --- a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md +++ b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md @@ -1,17 +1,16 @@ --- title: SHOW SETTINGS --- - import FunctionDescription from '@site/src/components/FunctionDescription'; -Databend 提供了多种系统设置,使您可以控制 Databend 的工作方式。此命令显示当前值和默认值,以及可用系统设置的 [Setting Levels](#setting-levels)。要更新设置,请使用 [SET](02-set-global.md) 或 [UNSET](02-unset.md) 命令。 +Databend 提供了多种系统设置,使您可以控制 Databend 的工作方式。此命令显示当前值和默认值,以及可用系统设置的 [设置级别](#setting-levels)。要更新设置,请使用 [SET](02-set-global.md) 或 [UNSET](02-unset.md) 命令。 -- 某些 Databend 行为无法通过系统设置更改;您必须在使用 Databend 时考虑到这些行为。例如, - - Databend 将字符串编码为 UTF-8 字符集。 - - Databend 对数组使用从 1 开始的编号约定。 -- Databend 将系统设置存储在系统表 [system.settings](../../00-sql-reference/20-system-tables/system-settings.md) 中。 +- 某些 Databend 行为无法通过系统设置进行更改;在使用 Databend 时,您必须考虑到这些行为。例如, + - Databend 将字符串编码为 UTF-8 字符集。 + - Databend 对数组使用从 1 开始的编号约定。 +- Databend 将系统设置存储在系统表 [system.settings](../../00-sql-reference/31-system-tables/system-settings.md) 中。 ## 语法 @@ -21,18 +20,18 @@ SHOW SETTINGS [LIKE '' | WHERE ] | [LIMIT ] ## 设置级别 -每个 Databend 设置都有一个级别,可以是 Global、Default 或 Session。下表说明了每个级别之间的区别: +每个 Databend 设置都带有一个级别,可以是 Global、Default 或 Session。下表说明了每个级别之间的区别: -| Level | Description | -| ------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| Global | 具有此级别的设置将写入元服务,并影响同一租户中的所有计算集群。此级别的更改具有全局影响,并适用于多个计算集群共享的整个数据库环境。 | -| Default | 具有此级别的设置通过 `databend-query.toml` 配置文件进行配置。此级别的更改仅影响单个查询实例,并且特定于配置文件。此级别为单个查询实例提供默认设置。 | -| Session | 具有此级别的设置仅限于单个请求或会话。它们具有最窄的范围,仅适用于正在进行的特定会话或请求,从而提供了一种在每个会话的基础上自定义设置的方法。 | +| Level | Description | +|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Global | 具有此级别的设置将写入元数据服务,并影响同一租户中的所有计算集群。此级别的更改具有全局影响,并适用于多个计算集群共享的整个数据库环境。 | +| Default | 具有此级别的设置通过 `databend-query.toml` 配置文件进行配置。此级别的更改仅影响单个查询实例,并且特定于配置文件。此级别为单个查询实例提供默认设置。 | +| Session | 具有此级别的设置仅限于单个请求或会话。它们具有最窄的范围,并且仅适用于正在进行的特定会话或请求,从而提供了一种在每个会话的基础上自定义设置的方法。 | ## 示例 :::note -由于 Databend 会不时更新系统设置,因此此示例可能不会显示最新的结果。要查看 Databend 中的最新系统设置,请在您的 Databend 实例中执行 `SHOW SETTINGS;`。 +由于 Databend 会不时更新系统设置,因此此示例可能不会显示最新的结果。要查看 Databend 中的最新系统设置,请在 Databend 实例中执行 `SHOW SETTINGS;`。 ::: ```sql @@ -47,4 +46,4 @@ SHOW SETTINGS LIMIT 5; │ auto_compaction_imperfect_blocks_threshold │ 50 │ 50 │ None │ DEFAULT │ Threshold for triggering auto compaction. This occurs when the number of imperfect blocks in a snapshot exceeds this value after write operations. │ UInt64 │ │ collation │ utf8 │ utf8 │ ["utf8"] │ DEFAULT │ Sets the character collation. Available values include "utf8". │ String │ └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` +``` \ No newline at end of file diff --git a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md index 54a1614639..7fc80b8b63 100644 --- a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md +++ b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; 列出当前支持的内置标量函数和聚合函数。 -另请参阅:[system.functions](../../00-sql-reference/20-system-tables/system-functions.md) +另请参阅:[system.functions](../../00-sql-reference/31-system-tables/system-functions.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md index 067c46d29b..fc06dcdc80 100644 --- a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md +++ b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; 列出系统中已存在的用户自定义函数和外部函数。等效于 `SELECT name, is_aggregate, description, arguments, language FROM system.user_functions ...`。 -另请参阅:[system.user_functions](../../00-sql-reference/20-system-tables/system-user-functions.md) +另请参阅:[system.user_functions](../../00-sql-reference/31-system-tables/system-user-functions.md) ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md index 0d78c6258d..9438550894 100644 --- a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md +++ b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md @@ -3,9 +3,9 @@ title: SHOW METRICS --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + -显示 [系统指标](../../00-sql-reference/20-system-tables/system-metrics.md) 列表。 +显示 [系统指标](../../00-sql-reference/31-system-tables/system-metrics.md) 列表。 ## 语法 diff --git a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md index 3e74cf640d..ab73fcd78d 100644 --- a/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md +++ b/docs/cn/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md @@ -12,7 +12,7 @@ import EEFeature from '@site/src/components/EEFeature'; 删除 Databend 创建的临时文件,例如溢出文件。 -另请参阅:[system.temp_files](../../00-sql-reference/20-system-tables/system-temp-files.md) +另请参阅:[system.temp_files](../../00-sql-reference/31-system-tables/system-temp-files.md) ## 语法 @@ -20,9 +20,9 @@ import EEFeature from '@site/src/components/EEFeature'; VACUUM TEMPORARY FILES [ LIMIT ] ``` -| 参数 | 描述 | -| ----- | ---------------------------- | -| LIMIT | 要删除的临时文件的最大数量。 | +| 参数 | 描述 | +|-----------|------------------------------------------------------| +| LIMIT | 要删除的临时文件的最大数量。 | ## 输出 @@ -32,7 +32,7 @@ VACUUM TEMPORARY FILES [ LIMIT ] ```sql SELECT * FROM system.temp_files; -``` + ``` ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ @@ -89,7 +89,7 @@ SELECT * FROM system.temp_files; └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -``` +```sql VACUUM TEMPORARY FILES; ┌────────┐ @@ -97,4 +97,4 @@ VACUUM TEMPORARY FILES; ├────────┤ │ 48 │ └────────┘ -``` +``` \ No newline at end of file diff --git a/docs/en/guides/51-access-data-lake/02-iceberg.md b/docs/en/guides/51-access-data-lake/02-iceberg.md index e194936017..65a181afc8 100644 --- a/docs/en/guides/51-access-data-lake/02-iceberg.md +++ b/docs/en/guides/51-access-data-lake/02-iceberg.md @@ -1,15 +1,15 @@ --- -title: Apache Iceberg +title: Apache Iceberg™ --- import FunctionDescription from '@site/src/components/FunctionDescription'; -Databend supports the integration of an [Apache Iceberg](https://iceberg.apache.org/) catalog, enhancing its compatibility and versatility for data management and analytics. This extends Databend's capabilities by seamlessly incorporating the powerful metadata and storage management capabilities of Apache Iceberg into the platform. +Databend supports the integration of an [Apache Iceberg™](https://iceberg.apache.org/) catalog, enhancing its compatibility and versatility for data management and analytics. This extends Databend's capabilities by seamlessly incorporating the powerful metadata and storage management capabilities of Apache Iceberg™ into the platform. -## Quick Start with Apache Iceberg +## Quick Start with Iceberg -If you want to quickly try out Apache Iceberg and experiment with table operations locally, a [Docker-based starter project](https://github.com/databendlabs/iceberg-quick-start) is available. This setup allows you to: +If you want to quickly try out Iceberg and experiment with table operations locally, a [Docker-based starter project](https://github.com/databendlabs/iceberg-quick-start) is available. This setup allows you to: - Run Spark with Iceberg support - Use a REST catalog (Iceberg REST Fixture) @@ -205,9 +205,9 @@ ORDER BY ## Datatype Mapping -This table maps data types between Apache Iceberg and Databend. Please note that Databend does not currently support Iceberg data types that are not listed in the table. +This table maps data types between Apache Iceberg™ and Databend. Please note that Databend does not currently support Iceberg data types that are not listed in the table. -| Apache Iceberg | Databend | +| Apache Iceberg™ | Databend | | ------------------------------- | ----------------------- | | BOOLEAN | [BOOLEAN](/sql/sql-reference/data-types/boolean) | | INT | [INT32](/sql/sql-reference/data-types/numeric#integer-data-types) | @@ -254,7 +254,7 @@ CONNECTION=( | Parameter | Required? | Description | |------------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `` | Yes | The name of the catalog you want to create. | -| `TYPE` | Yes | Specifies the catalog type. For Iceberg, set to `ICEBERG`. | +| `TYPE` | Yes | Specifies the catalog type. For Apache Iceberg™, set to `ICEBERG`. | | `CONNECTION` | Yes | The connection parameters for the Iceberg catalog. | | `TYPE` (inside `CONNECTION`) | Yes | The connection type. For Iceberg, it is typically set to `rest` for REST-based connection. | | `ADDRESS` | Yes | The address or URL of the Iceberg service (e.g., `http://127.0.0.1:8181`). | @@ -336,7 +336,7 @@ iceberg_table_meta_count = 0 In addition to metadata caching, Databend also supports table data caching for Iceberg catalog tables, similar to Fuse tables. For more information on data caching, refer to the `[cache] Section` in the [Query Configurations](../10-deploy/04-references/02-node-config/02-query-config.md) reference. -## Iceberg Table Functions +## Apache Iceberg™ Table Functions Databend provides the following table functions for querying Iceberg metadata, allowing users to inspect snapshots and manifests efficiently: diff --git a/docs/en/guides/54-query/03-udf.md b/docs/en/guides/54-query/03-udf.md index 85e33b7211..349a6fa780 100644 --- a/docs/en/guides/54-query/03-udf.md +++ b/docs/en/guides/54-query/03-udf.md @@ -22,32 +22,47 @@ A lambda UDF allows users to define custom operations using anonymous functions This example creates UDFs to extract specific values from JSON data within a table using SQL queries. ```sql --- Define UDFs -CREATE FUNCTION get_v1 AS (input_json) -> input_json['v1']; -CREATE FUNCTION get_v2 AS (input_json) -> input_json['v2']; +CREATE OR REPLACE TABLE sale_items ( + item_id INT, + details VARIANT +); + +INSERT INTO sale_items VALUES + (1, PARSE_JSON('{"name": "T-Shirt", "price": 20.00, "discount_pct": 10}')), -- 10% discount + (2, PARSE_JSON('{"name": "Jeans", "price": 50.00, "discount_pct": 25}')), -- 25% discount + (3, PARSE_JSON('{"name": "Jacket", "price": 100.00, "discount_pct": 0}')), -- No discount + (4, PARSE_JSON('{"name": "Socks", "price": 5.00, "discount_pct": 50}')); -- 50% discount + +-- Define a Lambda UDF to calculate the final price after discount +-- WITH EXPLICIT CASTING +CREATE OR REPLACE FUNCTION calculate_final_price AS (item_info) -> + (item_info['price']::FLOAT) * (1 - (item_info['discount_pct']::FLOAT) / 100.0); SHOW USER FUNCTIONS; +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ +--| name | is_aggregate | description | arguments | language | created_on | +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ +--| calculate_final_price | 0 | | {"parameters":["item_info"]} | SQL | YYYY-MM-DD HH:MM:SS.ffffff | +--+-----------------------+----------------+-------------+---------------------------------+----------+----------------------------+ -┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ name │ is_aggregate │ description │ arguments │ language │ created_on │ -├────────┼───────────────────┼─────────────┼───────────────────────────────┼──────────┼────────────────────────────┤ -│ get_v1 │ NULL │ │ {"parameters":["input_json"]} │ SQL │ 2024-11-18 23:20:28.432842 │ -│ get_v2 │ NULL │ │ {"parameters":["input_json"]} │ SQL │ 2024-11-18 23:21:46.838744 │ -└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ - --- Create a table -CREATE TABLE json_table(time TIMESTAMP, data JSON); - --- Insert a time event -INSERT INTO json_table VALUES('2022-06-01 00:00:00.00000', PARSE_JSON('{"v1":1.5, "v2":20.5}')); - --- Get v1 and v2 value from the event -SELECT get_v1(data), get_v2(data) FROM json_table; -+------------+------------+ -| data['v1'] | data['v2'] | -+------------+------------+ -| 1.5 | 20.5 | -+------------+------------+ +-- Use the Lambda UDF to get item names and their final prices +SELECT + item_id, + details['name']::STRING AS item_name, + details['price']::FLOAT AS original_price, + calculate_final_price(details) AS final_price +FROM sale_items +ORDER BY item_id; + +-- Expected output for the SELECT query (final_price should now have values): +--+---------+-----------+----------------+-------------+ +--| item_id | item_name | original_price | final_price | +--+---------+-----------+----------------+-------------+ +--| 1 | T-Shirt | 20.00 | 18.00 | +--| 2 | Jeans | 50.00 | 37.50 | +--| 3 | Jacket | 100.00 | 100.00 | +--| 4 | Socks | 5.00 | 2.50 | +--+---------+-----------+----------------+-------------+ ``` ## Embedded UDFs @@ -75,126 +90,117 @@ The Python UDF must use only Python's standard library; third-party imports are #### Data Type Mappings -See [Data Type Mappings](/developer/drivers/python#data-type-mappings) in the Developer Guide. +| Databend | Python | +| --------- | ----------------- | +| BOOLEAN | bool | +| TINYINT | int | +| SMALLINT | int | +| INT | int | +| BIGINT | int | +| FLOAT | float | +| DOUBLE | float | +| DECIMAL | decimal.Decimal | +| DATE | datetime.date | +| TIMESTAMP | datetime.datetime | +| VARCHAR | str | +| BINARY | bytes | +| ARRAY | list | +| TUPLE | tuple | +| MAP | dict | +| VARIANT | str | +| BITMAP | str | +| GEOMETRY | str | #### Usage Examples -This example defines a Python UDF for sentiment analysis, creates a table, inserts sample data, and performs sentiment analysis on the text data. - -1. Define a Python UDF named `sentiment_analysis`. - ```sql --- Create the sentiment analysis function -CREATE OR REPLACE FUNCTION sentiment_analysis(STRING) RETURNS STRING -LANGUAGE python HANDLER = 'sentiment_analysis_handler' -AS $$ -def remove_stop_words(text, stop_words): - """ - Removes common stop words from the text. - - Args: - text (str): The input text. - stop_words (set): A set of stop words to remove. - - Returns: - str: Text with stop words removed. - """ - return ' '.join([word for word in text.split() if word.lower() not in stop_words]) - -def calculate_sentiment(text, positive_words, negative_words): - """ - Calculates the sentiment score of the text. - - Args: - text (str): The input text. - positive_words (set): A set of positive words. - negative_words (set): A set of negative words. - - Returns: - int: Sentiment score. - """ - words = text.split() - score = sum(1 for word in words if word in positive_words) - sum(1 for word in words if word in negative_words) - return score - -def get_sentiment_label(score): - """ - Determines the sentiment label based on the sentiment score. - - Args: - score (int): The sentiment score. - - Returns: - str: Sentiment label ('Positive', 'Negative', 'Neutral'). - """ - if score > 0: - return 'Positive' - elif score < 0: - return 'Negative' - else: - return 'Neutral' - -def sentiment_analysis_handler(text): - """ - Analyzes the sentiment of the input text. - - Args: - text (str): The input text. - - Returns: - str: Sentiment analysis result including the score and label. - """ - stop_words = set(["a", "an", "the", "and", "or", "but", "if", "then", "so"]) - positive_words = set(["good", "happy", "joy", "excellent", "positive", "love"]) - negative_words = set(["bad", "sad", "pain", "terrible", "negative", "hate"]) - - clean_text = remove_stop_words(text, stop_words) - sentiment_score = calculate_sentiment(clean_text, positive_words, negative_words) - sentiment_label = get_sentiment_label(sentiment_score) - - return f'Sentiment Score: {sentiment_score}; Sentiment Label: {sentiment_label}' -$$; -``` - -2. Perform sentiment analysis on the text data using the `sentiment_analysis` function. - -```sql -CREATE OR REPLACE TABLE texts ( - original_text STRING +-- Create a table with user interaction logs +CREATE TABLE user_interaction_logs ( + log_id INT, + log_data VARIANT -- JSON interaction log ); --- Insert sample data -INSERT INTO texts (original_text) -VALUES -('The quick brown fox feels happy and joyful'), -('A hard journey, but it was painful and sad'), -('Uncertain outcomes leave everyone unsure and hesitant'), -('The movie was excellent and everyone loved it'), -('A terrible experience that made me feel bad'); - +-- Insert sample interaction log data +INSERT INTO user_interaction_logs VALUES + (1, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:00:00Z", "action": "view_product", "details": {"product_id": "p789", "category": "electronics", "price": 99.99}}')), + (2, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:05:10Z", "action": "add_to_cart", "details": {"product_id": "p789", "quantity": 1, "category": "electronics"}}')), + (3, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:02:30Z", "action": "search", "details": {"query": "wireless headphones", "results_count": 15}}')), + (4, PARSE_JSON('{"user_id": "u789", "timestamp": "2023-01-15T10:08:00Z", "action": "purchase", "details": {"order_id": "o555", "total_amount": 125.50, "item_count": 2}}')), + (5, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:10:00Z", "action": "view_page", "details": {"page_name": "homepage"}}')), + (6, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:12:00Z", "action": "purchase", "details": {"order_id": "o556", "total_amount": 25.00, "item_count": 1}}')); + +-- Create a Python UDF to extract features from interaction logs +CREATE OR REPLACE FUNCTION extract_interaction_features_py(VARIANT) +RETURNS VARCHAR +LANGUAGE python HANDLER = 'extract_features' +AS $$ +import json + +def extract_features(log): + log_dict = log if isinstance(log, dict) else {} + action = log_dict.get('action', '').lower() + details = log_dict.get('details', {}) + if not isinstance(details, dict): + details = {} + + is_search_action = False + has_product_interaction = False + product_category_if_any = None + search_query_length = 0 + purchase_value_bucket = None + + if action == 'search': + is_search_action = True + search_query_length = len(details.get('query', '')) + + if action in ['view_product', 'add_to_cart', 'remove_from_cart']: + has_product_interaction = True + product_category_if_any = details.get('category') + + if action == 'purchase': + has_product_interaction = True + + if action == 'purchase': + total_amount = details.get('total_amount', 0.0) + if not isinstance(total_amount, (int, float)): + total_amount = 0.0 + + if total_amount < 50: + purchase_value_bucket = 'Low' + elif total_amount < 200: + purchase_value_bucket = 'Medium' + else: + purchase_value_bucket = 'High' + + result_dict = { + "is_search_action": is_search_action, + "has_product_interaction": has_product_interaction, + "product_category_if_any": product_category_if_any, + "search_query_length": search_query_length, + "purchase_value_bucket": purchase_value_bucket + } + return json.dumps(result_dict) +$$; +-- Use the Python UDF to extract features SELECT - original_text, - sentiment_analysis(original_text) AS processed_text + log_id, + log_data['user_id']::STRING AS user_id, + log_data['action']::STRING AS action, + extract_interaction_features_py(log_data) AS extracted_features FROM - texts; - -| original_text | processed_text | -|----------------------------------------------------------|---------------------------------------------------| -| The quick brown fox feels happy and joyful | Sentiment Score: 1; Sentiment Label: Positive | -| A hard journey, but it was painful and sad | Sentiment Score: -1; Sentiment Label: Negative | -| Uncertain outcomes leave everyone unsure and hesitant | Sentiment Score: 0; Sentiment Label: Neutral | -| The movie was excellent and everyone loved it | Sentiment Score: 1; Sentiment Label: Positive | -| A terrible experience that made me feel bad | Sentiment Score: -2; Sentiment Label: Negative | + user_interaction_logs +ORDER BY + log_id; ``` + ### JavaScript A JavaScript UDF allows you to invoke JavaScript code from a SQL query via Databend's built-in handler, enabling seamless integration of JavaScript logic within your SQL queries. #### Data Type Mappings -The following table shows the type mapping between Databend and JavaScript: | Databend Type | JS Type | | ----------------- | ---------- | @@ -210,62 +216,77 @@ The following table shows the type mapping between Databend and JavaScript: | BIGINT UNSIGNED | Number | | FLOAT | Number | | DOUBLE | Number | -| STRING | String | +| VARCHAR | String | | DATE / TIMESTAMP | Date | | DECIMAL | BigDecimal | | BINARY | Uint8Array | #### Usage Examples -This example defines a JavaScript UDF named "gcd_js" to calculate the greatest common divisor (GCD) of two integers, and applies it within a SQL query: - ```sql -CREATE FUNCTION gcd_js (INT, INT) RETURNS BIGINT LANGUAGE javascript HANDLER = 'gcd_js' AS $$ -export function gcd_js(a, b) { - while (b != 0) { - let t = b; - b = a % b; - a = t; - } - return a; -} -$$ +-- Create a table with user interaction logs +CREATE TABLE user_interaction_logs ( + log_id INT, + log_data VARIANT -- JSON interaction log +); -SELECT - number, - gcd_js((number * 3), (number * 6)) -FROM - numbers(5) -WHERE - (number > 0) -ORDER BY 1; -``` +-- Insert sample interaction log data +INSERT INTO user_interaction_logs VALUES + (1, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:00:00Z", "action": "view_product", "details": {"product_id": "p789", "category": "electronics", "price": 99.99}}')), + (2, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:05:10Z", "action": "add_to_cart", "details": {"product_id": "p789", "quantity": 1, "category": "electronics"}}')), + (3, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:02:30Z", "action": "search", "details": {"query": "wireless headphones", "results_count": 15}}')), + (4, PARSE_JSON('{"user_id": "u789", "timestamp": "2023-01-15T10:08:00Z", "action": "purchase", "details": {"order_id": "o555", "total_amount": 125.50, "item_count": 2}}')), + (5, PARSE_JSON('{"user_id": "u123", "timestamp": "2023-01-15T10:10:00Z", "action": "view_page", "details": {"page_name": "homepage"}}')), + (6, PARSE_JSON('{"user_id": "u456", "timestamp": "2023-01-15T10:12:00Z", "action": "purchase", "details": {"order_id": "o556", "total_amount": 25.00, "item_count": 1}}')); -This example defines an aggregate UDF that calculates the weighted average of a set of values by aggregating them based on their corresponding weights: -```sql -CREATE FUNCTION weighted_avg (INT, INT) STATE {sum INT, weight INT} RETURNS FLOAT -LANGUAGE javascript AS $$ -export function create_state() { - return {sum: 0, weight: 0}; -} -export function accumulate(state, value, weight) { - state.sum += value * weight; - state.weight += weight; - return state; -} -export function retract(state, value, weight) { - state.sum -= value * weight; - state.weight -= weight; - return state; -} -export function merge(state1, state2) { - state1.sum += state2.sum; - state1.weight += state2.weight; - return state1; -} -export function finish(state) { - return state.sum / state.weight; +-- Create a JavaScript UDF to extract features from interaction logs +CREATE FUNCTION extract_interaction_features_js(VARIANT) +RETURNS VARIANT +LANGUAGE javascript HANDLER = 'extractFeatures' +AS $$ +export function extractFeatures(log) { + const action = (log.action || '').toLowerCase(); + const details = log.details || {}; + + let isSearchAction = false; + let hasProductInteraction = false; + let productCategoryIfAny = null; + let searchQueryLength = 0; + let purchaseValueBucket = null; + + if (action === 'search') { + isSearchAction = true; + searchQueryLength = (details.query || '').length; + } + + if (['view_product', 'add_to_cart', 'remove_from_cart'].includes(action)) { + hasProductInteraction = true; + productCategoryIfAny = details.category || null; + } + + if (action === 'purchase' && !productCategoryIfAny) { + hasProductInteraction = true; + } + + if (action === 'purchase') { + const totalAmount = details.total_amount || 0.0; + if (totalAmount < 50) { + purchaseValueBucket = 'Low'; + } else if (totalAmount < 200) { + purchaseValueBucket = 'Medium'; + } else { + purchaseValueBucket = 'High'; + } + } + + return { + is_search_action: isSearchAction, + has_product_interaction: hasProductInteraction, + product_category_if_any: productCategoryIfAny, + search_query_length: searchQueryLength, + purchase_value_bucket: purchaseValueBucket + }; } $$; ``` @@ -281,7 +302,7 @@ In this example, the "wasm_gcd" function is created to compute the greatest comm Prior to its execution, the function implementation undergoes a series of steps. First, it is compiled into a binary file, followed by compression into 'test10_udf_wasm_gcd.wasm.zst'. Finally, the compressed file is uploaded to a stage in advance. :::note -The function can be implemented in Rust, as demonstrated in the example available at https://github.com/risingwavelabs/arrow-udf/blob/main/arrow-udf-wasm/examples/wasm.rs +The function can be implemented in Rust, as demonstrated in the example available at https://github.com/arrow-udf/arrow-udf/blob/main/arrow-udf-example/src/lib.rs ::: ```sql diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/00-fuse.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/00-fuse.md index 15ec0e1fbf..979f65faf6 100644 --- a/docs/en/sql-reference/00-sql-reference/30-table-engines/00-fuse.md +++ b/docs/en/sql-reference/00-sql-reference/30-table-engines/00-fuse.md @@ -1,12 +1,28 @@ --- -title: Fuse Engine +title: Fuse Engine Tables --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + -Databend utilizes the Fuse Engine as its default engine, offering a data management system with a user-friendly interface reminiscent of Git. Users have the ability to effortlessly query data at any given moment and effortlessly restore data to any desired point in time. +## Overview + +Databend uses the Fuse Engine as its default storage engine, providing a Git-like data management system with: + +- **Snapshot-based Architecture**: Query and restore data at any point in time, with history of data changes for recovery +- **High Performance**: Optimized for analytical workloads with automatic indexing and bloom filters +- **Efficient Storage**: Uses Parquet format with high compression for optimal storage efficiency +- **Flexible Configuration**: Customizable compression, indexing, and storage options +- **Data Maintenance**: Automatic data retention, snapshot management, and change tracking capabilities + +## When to Use Fuse Engine + +Ideal for: +- **Analytics**: OLAP queries with columnar storage +- **Data Warehousing**: Large volumes of historical data +- **Time-Travel**: Access to historical data versions +- **Cloud Storage**: Optimized for object storage ## Syntax @@ -20,25 +36,168 @@ CREATE TABLE ( For more details about the `CREATE TABLE` syntax, see [CREATE TABLE](../../10-sql-commands/00-ddl/01-table/10-ddl-create-table.md). -| Parameter | Description | -| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `ENGINE` | If an engine is not explicitly specified, Databend will automatically default to using the Fuse Engine to create tables, which is equivalent to `ENGINE = FUSE`. | -| `CLUSTER BY` | Specifies the sorting method for data that consists of multiple expressions. For more information, see [Cluster Key](/guides/performance/cluster-key). | -| `` | The Fuse Engine offers various options (case-insensitive) that allow you to customize the table's properties. See [Fuse Engine Options](#fuse-engine-options) for details.
- Separate multiple options with a space.
- Use [ALTER TABLE OPTION](../../10-sql-commands/00-ddl/01-table/90-alter-table-option.md) to modify a table's options.
- Use [SHOW CREATE TABLE](../../10-sql-commands/00-ddl/01-table/show-create-table.md) to show a table's options. | +## Parameters + +Below are the main parameters for creating a Fuse Engine table: + +#### `ENGINE` +- **Description:** + If an engine is not explicitly specified, Databend will automatically default to using the Fuse Engine to create tables, which is equivalent to `ENGINE = FUSE`. + +--- + +#### `CLUSTER BY` +- **Description:** + Specifies the sorting method for data that consists of multiple expressions. For more information, see [Cluster Key](/guides/performance/cluster-key). + +--- + +#### `` +- **Description:** + The Fuse Engine offers various options (case-insensitive) that allow you to customize the table's properties. + - See [Fuse Engine Options](#fuse-engine-options) for details. + - Separate multiple options with a space. + - Use [ALTER TABLE OPTION](../../10-sql-commands/00-ddl/01-table/90-alter-table-option.md) to modify a table's options. + - Use [SHOW CREATE TABLE](../../10-sql-commands/00-ddl/01-table/show-create-table.md) to show a table's options. + +--- ## Fuse Engine Options -The following are the available Fuse Engine options: - -| Option | Syntax | Description | -| ------------------------------ | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| bloom_index_columns | `bloom_index_columns = ' [, ...]'` | Specifies the columns to be used for the bloom index. The data type of these columns can be Map, Number, String, Date, or Timestamp. If no specific columns are specified, the bloom index is created by default on all supported columns. `bloom_index_columns=''` disables the bloom indexing. | -| compression | `compression = ''` | Specifies the compression method for the engine. Compression options include lz4, zstd, snappy, or none. The compression method defaults to zstd in object storage and lz4 in file system (fs) storage. | -| storage_format | `storage_format = ''` | Specifies how data is stored. By default, the storage_format is set to **Parquet**, which offers high compression and is ideal for cloud-native object storage. Additionally, the experimental **Native** format is supported, optimizing memory copy overhead for storage devices like file systems. | -| snapshot_loc | `snapshot_loc = ''` | Specifies a location parameter in string format, allowing easy sharing of a table without data copy. | -| block_size_threshold | `block_size_threshold = ` | Specifies the maximum block size in bytes. Defaults to 104,857,600 bytes. | -| block_per_segment | `block_per_segment = ` | Specifies the maximum number of blocks in a segment. Defaults to 1,000. | -| row_per_block | `row_per_block = ` | Specifies the maximum number of rows in a file. Defaults to 1,000,000. | -| change_tracking | `change_tracking = True / False` | Setting this option to `True` in the Fuse Engine allows for tracking changes for a table.
Creating a stream for a table will automatically set `change_tracking` to `True` and introduce additional hidden columns to the table as change tracking metadata. For more information, see [How Stream Works](/guides/load-data/continuous-data-pipelines/stream#how-stream-works). | -| data_retention_period_in_hours | `data_retention_period_in_hours = ` | Specifies the number of hours to retain table data. The minimum value is 1 hour. The maximum value is defined by the `data_retention_time_in_days_max` setting in the [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) configuration file , or defaults to 2,160 hours (90 days x 24 hours) if not specified. | -| data_retention_num_snapshots_to_keep | `data_retention_num_snapshots_to_keep = ` | Specifies the number of snapshots to retain for a table. This option works in conjunction with the `enable_auto_vacuum` setting to provide granular control over snapshot retention policies on a per-table basis. When set, only the specified number of most recent snapshots will be kept after vacuum operations. | +Below are the available Fuse Engine options, grouped by their purpose: + +--- + +### `compression` +- **Syntax:** + `compression = ''` +- **Description:** + Specifies the compression method for the engine. Compression options include lz4, zstd, snappy, or none. The compression method defaults to zstd in object storage and lz4 in file system (fs) storage. + +--- + +### `snapshot_loc` +- **Syntax:** + `snapshot_loc = ''` +- **Description:** + Specifies a location parameter in string format, allowing easy sharing of a table without data copy. + +--- + + +### `block_size_threshold` +- **Syntax:** + `block_size_threshold = ` +- **Description:** + Specifies the maximum block size in bytes. Defaults to 104,857,600 bytes. + +--- + +### `block_per_segment` +- **Syntax:** + `block_per_segment = ` +- **Description:** + Specifies the maximum number of blocks in a segment. Defaults to 1,000. + +--- + +### `row_per_block` +- **Syntax:** + `row_per_block = ` +- **Description:** + Specifies the maximum number of rows in a file. Defaults to 1,000,000. + +--- + +### `bloom_index_columns` +- **Syntax:** + `bloom_index_columns = ' [, ...]'` +- **Description:** + Specifies the columns to be used for the bloom index. The data type of these columns can be Map, Number, String, Date, or Timestamp. If no specific columns are specified, the bloom index is created by default on all supported columns. `bloom_index_columns=''` disables the bloom indexing. + +--- + +### `change_tracking` +- **Syntax:** + `change_tracking = True / False` +- **Description:** + Setting this option to `True` in the Fuse Engine allows for tracking changes for a table. Creating a stream for a table will automatically set `change_tracking` to `True` and introduce additional hidden columns to the table as change tracking metadata. For more information, see [How Stream Works](/guides/load-data/continuous-data-pipelines/stream#how-stream-works). + +--- + +### `data_retention_period_in_hours` +- **Syntax:** + `data_retention_period_in_hours = ` +- **Description:** + Specifies the number of hours to retain table data. The minimum value is 1 hour. The maximum value is defined by the `data_retention_time_in_days_max` setting in the [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) configuration file, or defaults to 2,160 hours (90 days x 24 hours) if not specified. + +--- + +### `enable_auto_vacuum` +- **Syntax:** + `enable_auto_vacuum = 0 / 1` +- **Description:** + Controls whether a table automatically triggers vacuum operations during mutations. This can be set globally as a setting for all tables or configured at the table level. The table-level option has a higher priority than the session/global setting of the same name. When enabled (set to 1), vacuum operations will be automatically triggered after mutations like INSERT or ALTER TABLE, cleaning up the table data according to the configured retention policy. + + **Examples:** + ```sql + -- Set enable_auto_vacuum globally for all tables across all sessions + SET GLOBAL enable_auto_vacuum = 1; + + -- Create a table with auto vacuum disabled (overrides global setting) + CREATE OR REPLACE TABLE t1 (id INT) ENABLE_AUTO_VACUUM = 0; + INSERT INTO t1 VALUES(1); -- Won't trigger vacuum despite global setting + + -- Create another table that inherits the global setting + CREATE OR REPLACE TABLE t2 (id INT); + INSERT INTO t2 VALUES(1); -- Will trigger vacuum due to global setting + + -- Enable auto vacuum for an existing table + ALTER TABLE t1 SET OPTIONS(ENABLE_AUTO_VACUUM = 1); + INSERT INTO t1 VALUES(2); -- Now will trigger vacuum + + -- Table option takes precedence over global settings + SET GLOBAL enable_auto_vacuum = 0; -- Turn off globally + -- t1 will still vacuum because table setting overrides global + INSERT INTO t1 VALUES(3); -- Will still trigger vacuum + INSERT INTO t2 VALUES(2); -- Won't trigger vacuum anymore + ``` + +--- + +### `data_retention_num_snapshots_to_keep` +- **Syntax:** + `data_retention_num_snapshots_to_keep = ` +- **Description:** + Specifies the number of snapshots to retain during vacuum operations. This can be set globally as a setting for all tables or configured at the table level. The table-level option has a higher priority than the session/global setting of the same name. When set, only the specified number of most recent snapshots will be kept after vacuum operations. Overrides the `data_retention_time_in_days` setting. If set to 0, this setting will be ignored. This option works in conjunction with the `enable_auto_vacuum` setting to provide granular control over snapshot retention policies. + + **Examples:** + ```sql + -- Set global retention to 10 snapshots for all tables across all sessions + SET GLOBAL data_retention_num_snapshots_to_keep = 10; + + -- Create a table with custom snapshot retention (overrides global setting) + CREATE OR REPLACE TABLE t1 (id INT) + enable_auto_vacuum = 1 + data_retention_num_snapshots_to_keep = 5; + + -- Create another table that inherits the global setting + CREATE OR REPLACE TABLE t2 (id INT) enable_auto_vacuum = 1; + + -- When vacuum is triggered: + -- t1 will keep 5 snapshots (table setting) + -- t2 will keep 10 snapshots (global setting) + + -- Change global setting + SET GLOBAL data_retention_num_snapshots_to_keep = 20; + + -- Table options still take precedence: + -- t1 will still keep only 5 snapshots + -- t2 will now keep 20 snapshots + + -- Modify snapshot retention for an existing table + ALTER TABLE t1 SET OPTIONS(data_retention_num_snapshots_to_keep = 3); + -- Now t1 will keep 3 snapshots when vacuum is triggered + ``` + +--- diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/01-memory.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/01-memory.md deleted file mode 100644 index ec4c52cb7a..0000000000 --- a/docs/en/sql-reference/00-sql-reference/30-table-engines/01-memory.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Memory Engine ---- - -## Syntax - -```sql -CREATE TABLE table_name ( - column_name1 column_type1, - column_name2 column_type2, - ... -) ENGINE = Memory; -``` - -## Use cases - -This engine is only for development and testing purposes. It is not recommended to use it in production. - -While the Memory Engine provides several advantages, there are also some limitations: - -- Limited storage capacity: The amount of data that can be stored is limited by the amount of memory available on the server. This makes the Memory Engine less suitable for large datasets. - -- Data loss on server failure: Since all the data is stored in memory, if the server hosting the Databend instance fails, all the data stored in memory will be lost. - diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md new file mode 100644 index 0000000000..250b4626f6 --- /dev/null +++ b/docs/en/sql-reference/00-sql-reference/30-table-engines/02-iceberg.md @@ -0,0 +1,9 @@ +--- +id: iceberg +title: Apache Iceberg™ Tables +sidebar_label: Apache Iceberg™ Tables +--- + +import Content from '../../../guides/51-access-data-lake/02-iceberg.md'; + + diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/02-random.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/02-random.md deleted file mode 100644 index 8ac8733b83..0000000000 --- a/docs/en/sql-reference/00-sql-reference/30-table-engines/02-random.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Random Engine ---- - -## Syntax - -```sql -CREATE TABLE table_name ( - column_name1 column_type1, - column_name2 column_type2, - ... -) ENGINE = Random; -``` - -## Use cases - -Random engine is used to generate random data for testing purposes. If limit is not specified in query, it will use `max_block_size` as limit value. - -Example: - -```sql -> create table r (a int, b timestamp, c String, d Variant) Engine = Random; - -> select * from r limit 3; - -+-------------+----------------------------+-------+-----------------------------------------------------------------------------------------+ -| a | b | c | d | -+-------------+----------------------------+-------+-----------------------------------------------------------------------------------------+ -| 1515222693 | 5644-08-08 08:18:02.153307 | Cd4EL | false | -| -1721583430 | 2801-10-02 21:10:08.239880 | bdREv | {"2CbjY":5566179425126492090,"FtfVz":9111809212512345709,"VQuVv":-9022907531714095185} | -| -1865622072 | 6158-08-16 19:03:47.563392 | wKcpg | {"7dZAZ":-6513853114735328596,"nkZOZ":-400420074545948308,"px9BO":-8839157034206468334} | -+-------------+----------------------------+-------+-----------------------------------------------------------------------------------------+ -``` diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/03-hive.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/03-hive.md new file mode 100644 index 0000000000..997dcdbe08 --- /dev/null +++ b/docs/en/sql-reference/00-sql-reference/30-table-engines/03-hive.md @@ -0,0 +1,9 @@ +--- +id: hive +title: Apache Hive Tables +sidebar_label: Apache Hive Tables +--- + +import Content from '../../../guides/51-access-data-lake/01-hive.md'; + + diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/04-delta.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/04-delta.md deleted file mode 100644 index 44e0fc1d74..0000000000 --- a/docs/en/sql-reference/00-sql-reference/30-table-engines/04-delta.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Delta Lake Engine ---- - -See [Delta Lake Engine](/guides/access-data-lake/delta). \ No newline at end of file diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/_04-delta.md b/docs/en/sql-reference/00-sql-reference/30-table-engines/_04-delta.md new file mode 100644 index 0000000000..14e2789266 --- /dev/null +++ b/docs/en/sql-reference/00-sql-reference/30-table-engines/_04-delta.md @@ -0,0 +1,9 @@ +--- +id: delta +title: Delta Lake Engine +sidebar_label: Delta Lake Engine +--- + +import Content from '../../../guides/51-access-data-lake/03-delta.md'; + + diff --git a/docs/en/sql-reference/00-sql-reference/30-table-engines/_category_.json b/docs/en/sql-reference/00-sql-reference/30-table-engines/_category_.json index 2b6433a0e0..e43e5670b2 100644 --- a/docs/en/sql-reference/00-sql-reference/30-table-engines/_category_.json +++ b/docs/en/sql-reference/00-sql-reference/30-table-engines/_category_.json @@ -1,3 +1,3 @@ { - "label": "Table Engines" + "label": "Tables" } \ No newline at end of file diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/_category_.json b/docs/en/sql-reference/00-sql-reference/31-system-tables/_category_.json similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/_category_.json rename to docs/en/sql-reference/00-sql-reference/31-system-tables/_category_.json diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/information-schema-tables.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/information-schema-tables.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/information-schema-tables.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/information-schema-tables.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-build-options.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-build-options.md similarity index 99% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-build-options.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-build-options.md index 098af83b95..52959731c6 100644 --- a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-build-options.md +++ b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-build-options.md @@ -18,4 +18,4 @@ SELECT * FROM system.build_options; | | sse2 | +----------------+---------------------+ 4 rows in set (0.031 sec) -``` \ No newline at end of file +``` diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-caches.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-caches.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-caches.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-caches.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-clusters.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-clusters.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-clusters.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-clusters.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-columns.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-columns.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-columns.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-columns.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-configs.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-configs.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-configs.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-configs.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-contributors.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-contributors.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-contributors.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-contributors.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-credits.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-credits.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-credits.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-credits.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-databases-with-history.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-databases-with-history.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-databases-with-history.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-databases-with-history.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-databases.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-databases.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-databases.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-databases.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-functions.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-functions.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-functions.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-functions.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-indexes.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-indexes.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-indexes.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-indexes.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-locks.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-locks.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-locks.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-locks.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-metrics.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-metrics.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-metrics.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-metrics.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-numbers.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-numbers.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-numbers.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-numbers.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-query-cache.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-query-cache.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-query-cache.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-query-cache.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-query-log.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-query-log.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-query-log.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-query-log.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-settings.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-settings.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-settings.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-settings.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-streams.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-streams.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-streams.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-streams.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-table-functions.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-table-functions.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-table-functions.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-table-functions.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-tables-with-history.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-tables-with-history.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-tables-with-history.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-tables-with-history.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-tables.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-tables.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-tables.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-tables.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-temp-files.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-temp-files.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-temp-files.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-temp-files.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-temp-tables.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-temp-tables.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-temp-tables.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-temp-tables.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-user-functions.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-user-functions.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-user-functions.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-user-functions.md diff --git a/docs/en/sql-reference/00-sql-reference/20-system-tables/system-virtual-columns.md b/docs/en/sql-reference/00-sql-reference/31-system-tables/system-virtual-columns.md similarity index 100% rename from docs/en/sql-reference/00-sql-reference/20-system-tables/system-virtual-columns.md rename to docs/en/sql-reference/00-sql-reference/31-system-tables/system-virtual-columns.md diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md b/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md index 452100becc..68cff7c60f 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-databases.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; Shows the list of databases that exist on the instance. -See also: [system.databases](../../../00-sql-reference/20-system-tables/system-databases.md) +See also: [system.databases](../../../00-sql-reference/31-system-tables/system-databases.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md b/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md index 0ed5f7f8c6..22a5fa7899 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/00-database/show-drop-databases.md @@ -10,7 +10,7 @@ Lists all databases along with their deletion timestamps if they have been dropp - Dropped databases can only be retrieved if they are within the data retention period. - It is recommended to use an admin user, such as `root`. If you are using Databend Cloud, use a user with the `account_admin` role to query dropped databases. -See also: [system.databases_with_history](../../../00-sql-reference/20-system-tables/system-databases-with-history.md) +See also: [system.databases_with_history](../../../00-sql-reference/31-system-tables/system-databases-with-history.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md index 1106c54c10..b4b24d356e 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/10-ddl-create-temp-table.md @@ -10,10 +10,10 @@ Creates a temporary table that is automatically dropped at the end of the sessio - A temporary table is visible only within the session that created it and is automatically dropped, with all data vacuumed, when the session ends. - In cases where automatic cleanup of temporary tables fails—for example, due to a query node crash—you can use the [FUSE_VACUUM_TEMPORARY_TABLE](../../../20-sql-functions/17-table-functions/fuse-vacuum-temporary-table.md) function to manually clean up leftover files from temporary tables. -- To show the existing temporary tables in the session, query the [system.temporary_tables](../../../00-sql-reference/20-system-tables/system-temp-tables.md) system table. See [Example-1](#example-1). +- To show the existing temporary tables in the session, query the [system.temporary_tables](../../../00-sql-reference/31-system-tables/system-temp-tables.md) system table. See [Example-1](#example-1). - A temporary table with the same name as a normal table takes precedence, hiding the normal table until dropped. See [Example-2](#example-2). - No privileges are required to create or operate on a temporary table. -- Databend supports creating temporary tables with both the [Fuse Engine](../../../00-sql-reference/30-table-engines/00-fuse.md) and [Memory Engine](../../../00-sql-reference/30-table-engines/01-memory.md). +- Databend supports creating temporary tables with the [Fuse Engine](../../../00-sql-reference/30-table-engines/00-fuse.md). - To create temporary tables using BendSQL, ensure you are using the latest version of BendSQL. ## Syntax @@ -31,7 +31,7 @@ The omitted parts follow the syntax of [CREATE TABLE](10-ddl-create-table.md). ### Example-1 -This example demonstrates how to create a temporary table and verify its existence by querying the [system.temporary_tables](../../../00-sql-reference/20-system-tables/system-temp-tables.md) system table: +This example demonstrates how to create a temporary table and verify its existence by querying the [system.temporary_tables](../../../00-sql-reference/31-system-tables/system-temp-tables.md) system table: ```sql CREATE TEMP TABLE my_table (id INT, description STRING); diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md index a126e65a53..7ba46a3cf8 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-drop-tables.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; Lists the dropped tables in the current or a specified database. -See also: [system.tables_with_history](../../../00-sql-reference/20-system-tables/system-tables-with-history.md) +See also: [system.tables_with_history](../../../00-sql-reference/31-system-tables/system-tables-with-history.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md index 1e7c67df15..8e39018cb5 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/01-table/show-tables.md @@ -12,7 +12,7 @@ Lists the tables in the current or a specified database. Starting from version 1.2.415, the SHOW TABLES command no longer includes views in its results. To display views, use [SHOW VIEWS](../05-view/show-views.md) instead. ::: -See also: [system.tables](../../../00-sql-reference/20-system-tables/system-tables.md) +See also: [system.tables](../../../00-sql-reference/31-system-tables/system-tables.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md b/docs/en/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md index 4a0983b7cc..2323ba9a5a 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/07-aggregating-index/show-indexes.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; Shows the created aggregating indexes. Equivalent to `SELECT * FROM system.indexes`. -See also: [system.indexes](../../../00-sql-reference/20-system-tables/system-indexes.md) +See also: [system.indexes](../../../00-sql-reference/31-system-tables/system-indexes.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md b/docs/en/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md index 3b9e51ef2a..67c35d3a7a 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/07-virtual-column/show-virtual-columns.md @@ -12,7 +12,7 @@ import EEFeature from '@site/src/components/EEFeature'; Shows the created virtual columns in the system. Equivalent to `SELECT * FROM system.virtual_columns`. -See also: [system.virtual_columns](../../../00-sql-reference/20-system-tables/system-virtual-columns.md) +See also: [system.virtual_columns](../../../00-sql-reference/31-system-tables/system-virtual-columns.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md index 2f2f5c73c1..28f48b6181 100644 --- a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md +++ b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/03-show-settings.md @@ -10,7 +10,7 @@ Databend provides a variety of system settings that enable you to control how Da - Some Databend behaviors cannot be changed through the system settings; you must take them into consideration while working with Databend. For example, - Databend encodes strings to the UTF-8 charset. - Databend uses a 1-based numbering convention for arrays. -- Databend stores the system settings in the system table [system.settings](../../00-sql-reference/20-system-tables/system-settings.md). +- Databend stores the system settings in the system table [system.settings](../../00-sql-reference/31-system-tables/system-settings.md). ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md index 9ae079506d..18000c9cf2 100644 --- a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md +++ b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/04-show-functions.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; Lists the currently supported built-in scalar and aggregate functions. -See also: [system.functions](../../00-sql-reference/20-system-tables/system-functions.md) +See also: [system.functions](../../00-sql-reference/31-system-tables/system-functions.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md index aee22a0345..8d32489505 100644 --- a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md +++ b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/05-show-user-functions.md @@ -8,7 +8,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; Lists the existing user-defined functions and external functions in the system. Equivalent to `SELECT name, is_aggregate, description, arguments, language FROM system.user_functions ...`. -See also: [system.user_functions](../../00-sql-reference/20-system-tables/system-user-functions.md) +See also: [system.user_functions](../../00-sql-reference/31-system-tables/system-user-functions.md) ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md index 924537c8ac..d97d936e7f 100644 --- a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md +++ b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/08-show-metrics.md @@ -5,7 +5,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; -Shows the list of [system metrics](../../00-sql-reference/20-system-tables/system-metrics.md). +Shows the list of [system metrics](../../00-sql-reference/31-system-tables/system-metrics.md). ## Syntax diff --git a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md index 52cb55eaf5..a09f587dc8 100644 --- a/docs/en/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md +++ b/docs/en/sql-reference/10-sql-commands/50-administration-cmds/09-vacuum-temp-files.md @@ -12,7 +12,7 @@ import EEFeature from '@site/src/components/EEFeature'; Removes temporary files created by Databend, such as spill files. -See also: [system.temp_files](../../00-sql-reference/20-system-tables/system-temp-files.md) +See also: [system.temp_files](../../00-sql-reference/31-system-tables/system-temp-files.md) ## Syntax