Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 102 additions & 68 deletions docs/en/docs/lakehouse/multi-catalog/jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,108 @@ The transaction mechanism ensures the atomicity of data writing to JDBC External

## Guide

### View the JDBC Catalog

You can query all Catalogs in the current Doris cluster through SHOW CATALOGS:

```sql
SHOW CATALOGS;
```

Query the creation statement of a Catalog through SHOW CREATE CATALOG:

```sql
SHOW CREATE CATALOG <catalog_name>;
```

### Drop the JDBC Catalog

A Catalog can be deleted via DROP CATALOG:

```sql
DROP CATALOG <catalog_name>;
```

### Query the JDBC Catalog

1. Use SWITCH to switch the Catalog in effect for the current session:

```sql
SWITCH <catalog_name>;
```

2. Query all libraries under the current Catalog through SHOW DATABASES:

```sql
SHOW DATABASES FROM <catalog_name>;
```

```sql
SHOW DATABASES;
```

3. Use USE to switch the Database that takes effect in the current session:

```sql
USE <database_name>;
```

Or directly use `USE <catalog_name>.<database_name>;` to switch the Database that takes effect in the current session

4. Query all tables under the current Catalog through SHOW TABLES:

```sql
SHOW TABLES FROM <catalog_name>.<database_name>;
```

```sql
SHOW TABLES FROM <database_name>;
```

```sql
SHOW TABLES;
```

5. Query the data of a table under the current Catalog through SELECT:

```sql
SELECT * FROM <table_name>;
```

## SQL Passthrough

In versions prior to Doris 2.0.3, users could only perform query operations (SELECT) through the JDBC Catalog.
Starting from version Doris 2.0.4, users can perform DDL (Data Definition Language) and DML (Data Manipulation Language) operations on JDBC data sources using the `CALL` command.

```
CALL EXECUTE_STMT("catalog_name", "raw_stmt_string");
```

The `EXECUTE_STMT()` procedure involves two parameters:

- Catalog Name: Currently, only the Jdbc Catalog is supported.
- Execution Statement: Currently, only DDL and DML statements are supported. These statements must use the syntax specific to the JDBC data source.

```
CALL EXECUTE_STMT("jdbc_catalog", "insert into db1.tbl1 values(1,2), (3, 4)");

CALL EXECUTE_STMT("jdbc_catalog", "delete from db1.tbl1 where k1 = 2");

CALL EXECUTE_STMT("jdbc_catalog", "create table db1.tbl2 (k1 int)");
```

### Principles and Limitations

Through the `CALL EXECUTE_STMT()` command, Doris directly sends the SQL statements written by the user to the JDBC data source associated with the Catalog for execution. Therefore, this operation has the following limitations:

- The SQL statements must be in the syntax specific to the data source, as Doris does not perform syntax and semantic checks.
- It is recommended that table names in SQL statements be fully qualified, i.e., in the `db.tbl` format. If the `db` is not specified, the db name specified in the JDBC Catalog's JDBC URL will be used.
- SQL statements cannot reference tables outside of the JDBC data source, nor can they reference Doris's tables. However, they can reference tables within the JDBC data source that have not been synchronized to the Doris JDBC Catalog.
- When executing DML statements, it is not possible to obtain the number of rows inserted, updated, or deleted; success of the command execution can only be confirmed.
- Only users with LOAD permissions on the Catalog can execute this command.

## Supported Datasoures

### MySQL

#### Example
Expand Down Expand Up @@ -602,74 +704,6 @@ CREATE CATALOG jdbc_oceanbase PROPERTIES (
When Doris connects to OceanBase, it will automatically recognize that OceanBase is in MySQL or Oracle mode. Hierarchical correspondence and type mapping refer to [MySQL](#MySQL) and [Oracle](#Oracle)
:::

### View the JDBC Catalog

You can query all Catalogs in the current Doris cluster through SHOW CATALOGS:

```sql
SHOW CATALOGS;
```

Query the creation statement of a Catalog through SHOW CREATE CATALOG:

```sql
SHOW CREATE CATALOG <catalog_name>;
```

### Drop the JDBC Catalog

A Catalog can be deleted via DROP CATALOG:

```sql
DROP CATALOG <catalog_name>;
```

### Query the JDBC Catalog

1. Use SWITCH to switch the Catalog in effect for the current session:

```sql
SWITCH <catalog_name>;
```

2. Query all libraries under the current Catalog through SHOW DATABASES:

```sql
SHOW DATABASES FROM <catalog_name>;
```

```sql
SHOW DATABASES;
```

3. Use USE to switch the Database that takes effect in the current session:

```sql
USE <database_name>;
```

Or directly use `USE <catalog_name>.<database_name>;` to switch the Database that takes effect in the current session

4. Query all tables under the current Catalog through SHOW TABLES:

```sql
SHOW TABLES FROM <catalog_name>.<database_name>;
```

```sql
SHOW TABLES FROM <database_name>;
```

```sql
SHOW TABLES;
```

5. Query the data of a table under the current Catalog through SELECT:

```sql
SELECT * FROM <table_name>;
```

## JDBC Drivers

It is recommended to use the following versions of Driver to connect to the corresponding database. Other versions of the Driver have not been tested and may cause unexpected problems.
Expand Down
170 changes: 102 additions & 68 deletions docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,108 @@ set enable_odbc_transcation = true;

## 使用指南

### 查看 JDBC Catalog

可以通过 SHOW CATALOGS 查询当前所在 Doris 集群里所有 Catalog:

```sql
SHOW CATALOGS;
```

通过 SHOW CREATE CATALOG 查询某个 Catalog 的创建语句:

```sql
SHOW CREATE CATALOG <catalog_name>;
```

### 删除 JDBC Catalog

可以通过 DROP CATALOG 删除某个 Catalog:

```sql
DROP CATALOG <catalog_name>;
```

### 查询 JDBC Catalog

1. 通过 SWITCH 切换当前会话生效的 Catalog:

```sql
SWITCH <catalog_name>;
```

2. 通过 SHOW DATABASES 查询当前 Catalog 下的所有库:

```sql
SHOW DATABASES FROM <catalog_name>;
```

```sql
SHOW DATABASES;
```

3. 通过 USE 切换当前会话生效的 Database:

```sql
USE <database_name>;
```

或者直接通过 `USE <catalog_name>.<database_name>;` 切换当前会话生效的 Database

4. 通过 SHOW TABLES 查询当前 Catalog 下的所有表:

```sql
SHOW TABLES FROM <catalog_name>.<database_name>;
```

```sql
SHOW TABLES FROM <database_name>;
```

```sql
SHOW TABLES;
```

5. 通过 SELECT 查询当前 Catalog 下的某个表的数据:

```sql
SELECT * FROM <table_name>;
```

### SQL 透传

在 Doris 2.0.3 之前的版本中,用户只能通过 JDBC Catalog 进行查询操作(SELECT)。
在 Doris 2.0.4 版本之后,用户可以通过 `CALL` 命令,对 JDBC 数据源进行 DDL 和 DML 操作。

```
CALL EXECUTE_STMT("catalog_name", "raw_stmt_string");
```

`EXECUTE_STMT()` 过程有两个参数:

- Catalog Name:目前仅支持 Jdbc Catalog。
- 执行语句:目前仅支持 DDL 和 DML 语句。并且需要直接使用 JDBC 数据源对应的语法。

```
CALL EXECUTE_STMT("jdbc_catalog", "insert into db1.tbl1 values(1,2), (3, 4)");

CALL EXECUTE_STMT(jdbc_catalog", "delete from db1.tbl1 where k1 = 2");

CALL EXECUTE_STMT(jdbc_catalog", "create table dbl1.tbl2 (k1 int)");
```

#### 原理和限制

通过 `CALL EXECUTE_STMT()` 命令,Doris 会直接将用户编写的 SQL 语句发送给 Catalog 对应的 JDBC 数据源进行执行。因此,这个操作有如下限制:

- SQL 语句必须是数据源对应的语法,Doris 不会做语法和语义检查。
- SQL 语句中引用的表名建议是全限定名,即 `db.tbl` 这种格式。如果未指定 db,则会使用 JDBC Catalog 的 JDBC url 中指定的 db 名称。
- SQL 语句中不可引用 JDBC 数据源之外的库表,也不可以引用 Doris 的库表。但可以引用在 JDBC 数据源内的,但是没有同步到 Doris JDBC Catalog 的库表。
- 执行 DML 语句,无法获取插入、更新或删除的行数,只能获取命令是否执行成功。
- 只有对 Catalog 有 LOAD 权限的用户,才能执行这个命令。

## 支持的数据源

### MySQL

#### 创建示例
Expand Down Expand Up @@ -603,74 +705,6 @@ CREATE CATALOG jdbc_oceanbase PROPERTIES (
Doris 在连接 OceanBase 时,会自动识别 OceanBase 处于 MySQL 或者 Oracle 模式,层级对应和类型映射参考 [MySQL](#MySQL) 与 [Oracle](#Oracle)
:::

### 查看 JDBC Catalog

可以通过 SHOW CATALOGS 查询当前所在 Doris 集群里所有 Catalog:

```sql
SHOW CATALOGS;
```

通过 SHOW CREATE CATALOG 查询某个 Catalog 的创建语句:

```sql
SHOW CREATE CATALOG <catalog_name>;
```

### 删除 JDBC Catalog

可以通过 DROP CATALOG 删除某个 Catalog:

```sql
DROP CATALOG <catalog_name>;
```

### 查询 JDBC Catalog

1. 通过 SWITCH 切换当前会话生效的 Catalog:

```sql
SWITCH <catalog_name>;
```

2. 通过 SHOW DATABASES 查询当前 Catalog 下的所有库:

```sql
SHOW DATABASES FROM <catalog_name>;
```

```sql
SHOW DATABASES;
```

3. 通过 USE 切换当前会话生效的 Database:

```sql
USE <database_name>;
```

或者直接通过 `USE <catalog_name>.<database_name>;` 切换当前会话生效的 Database

4. 通过 SHOW TABLES 查询当前 Catalog 下的所有表:

```sql
SHOW TABLES FROM <catalog_name>.<database_name>;
```

```sql
SHOW TABLES FROM <database_name>;
```

```sql
SHOW TABLES;
```

5. 通过 SELECT 查询当前 Catalog 下的某个表的数据:

```sql
SELECT * FROM <table_name>;
```

## JDBC Driver 列表

推荐使用以下版本的 Driver 连接对应的数据库。其他版本的 Driver 未经测试,可能导致非预期的问题。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ BUILD: 'BUILD';
BUILTIN: 'BUILTIN';
BY: 'BY';
CACHED: 'CACHED';
CALL: 'CALL';
CANCEL: 'CANCEL';
CASE: 'CASE';
CAST: 'CAST';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ statement
constraint #addConstraint
| ALTER TABLE table=relation
DROP CONSTRAINT constraintName=errorCapturingIdentifier #dropConstraint
| CALL functionName=identifier LEFT_PAREN (expression (COMMA expression)*)? RIGHT_PAREN #callProcedure
;

constraint
Expand All @@ -117,6 +118,7 @@ partitionSpec
// TODO: support analyze external table partition spec https://github.com/apache/doris/pull/24154
// | PARTITIONS LEFT_PAREN ASTERISK RIGHT_PAREN
// | PARTITIONS WITH RECENT
LEFT_PAREN referenceSlots+=errorCapturingIdentifier (COMMA referenceSlots+=errorCapturingIdentifier)* RIGHT_PAREN
;

dataDesc
Expand Down Expand Up @@ -906,6 +908,7 @@ nonReserved
| BUILD
| BUILTIN
| CACHED
| CALL
| CATALOG
| CATALOGS
| CHAIN
Expand Down
Loading