Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion distribution/proxy-native/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -718,7 +718,7 @@
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<classifier>http</classifier>
<classifier>all</classifier>
<scope>runtime</scope>
</dependency>
</dependencies>
Expand Down
2 changes: 1 addition & 1 deletion distribution/proxy/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -753,7 +753,7 @@
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<classifier>http</classifier>
<classifier>all</classifier>
<scope>runtime</scope>
</dependency>
</dependencies>
Expand Down
2 changes: 1 addition & 1 deletion distribution/proxy/src/main/release-docs/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -324,7 +324,7 @@ BSD licenses
The following components are provided under a BSD license. See project link for details.
The text of each license is also included at licenses/LICENSE-[project].txt.

antlr4-runtime 4.10.1: https://github.com/antlr/antlr4, BSD-3-Clause
antlr4-runtime 4.13.2: https://github.com/antlr/antlr4, BSD-3-Clause
asm 9.3: https://github.com/llbit/ow2-asm, BSD-3-Clause
commons-compiler 3.1.11: https://github.com/janino-compiler/janino, BSD-3-Clause
janino 3.1.11: https://github.com/janino-compiler/janino, BSD-3-Clause
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,42 +280,13 @@ Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supporte
}
```

6. 当需要通过 ShardingSphere JDBC 使用 ClickHouse 方言时,
用户需要手动引入相关的可选模块和 classifier 为 `http` 的 ClickHouse JDBC 驱动。
原则上,ShardingSphere 的 GraalVM Native Image 集成不希望使用 classifier 为 `all` 的 `com.clickhouse:clickhouse-jdbc`,
因为 Uber Jar 会导致采集重复的 GraalVM Reachability Metadata。
可能的配置例子如下,

```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-dialect-clickhouse</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.6.3</version>
<classifier>http</classifier>
</dependency>
</dependencies>
</project>
```

7. ShardingSphere 的单元测试仅使用 Maven 模块 `io.github.linghengqian:hive-server2-jdbc-driver-thin` 来在 GraalVM Native Image 下验证 HiveServer2 集成的可用性。
6. ShardingSphere 的单元测试仅使用 Maven 模块 `io.github.linghengqian:hive-server2-jdbc-driver-thin` 来在 GraalVM Native Image 下验证 HiveServer2 集成的可用性。
如果开发者直接使用 `org.apache.hive:hive-jdbc`,则应自行处理依赖冲突和提供额外的 GraalVM Reachability Metadata 。

8. 由于 https://github.com/oracle/graal/issues/7979 的影响,
7. 由于 https://github.com/oracle/graal/issues/7979 的影响,
对应 `com.oracle.database.jdbc:ojdbc8` Maven 模块的 Oracle JDBC Driver 无法在 GraalVM Native Image 下使用。

9. 包括但不限于来自第三方依赖的 `com.mysql.cj.LocalizedErrorMessages`,
8. 包括但不限于来自第三方依赖的 `com.mysql.cj.LocalizedErrorMessages`,
`com.microsoft.sqlserver.jdbc.SQLServerResource`,
`org.postgresql.translation.messages`,
`org.opengauss.translation.messages` 等 `Resource Bundles` 在默认情况下会根据系统的默认语言环境加载 L10N 资源,
Expand All @@ -340,11 +311,11 @@ without it being registered as reachable. Add it to the resource metadata to sol
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:186)
```

10. 受 `apache/calcite` 使用的 `janino-compiler/janino` 的影响,
9. 受 `apache/calcite` 使用的 `janino-compiler/janino` 的影响,
ShardingSphere 的 `SQL Federation` 功能在 GraalVM Native Image 下不可用。
这同样导致 ShardingSphere Proxy Native 无法使用 OpenGauss 集成。

11. 受 https://github.com/oracle/graal/issues/11280 影响,
10. 受 https://github.com/oracle/graal/issues/11280 影响,
Etcd 的 Cluster 模式集成无法在通过 Windows 11 编译的 GraalVM Native Image 下使用,
且 Etcd 的 Cluster 模式会与 GraalVM Tracing Agent 产生冲突。
若开发者需要在通过 Linux 编译的 GraalVM Native Image 下使用 Etcd 的 Cluster 模式,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -291,42 +291,13 @@ Users need to define the following JSON in the `reachability-metadata.json` file
}
```

6. When using the ClickHouse dialect through ShardingSphere JDBC,
users need to manually introduce the relevant optional modules and the ClickHouse JDBC driver with the classifier `http`.
In principle, ShardingSphere's GraalVM Native Image integration does not want to use `com.clickhouse:clickhouse-jdbc` with classifier `all`,
because Uber Jar will cause the collection of duplicate GraalVM Reachability Metadata.
Possible configuration examples are as follows,

```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-dialect-clickhouse</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.6.3</version>
<classifier>http</classifier>
</dependency>
</dependencies>
</project>
```

7. ShardingSphere's unit test only uses the Maven module `io.github.linghengqian:hive-server2-jdbc-driver-thin` to verify the availability of HiveServer2 integration under GraalVM Native Image.
6. ShardingSphere's unit test only uses the Maven module `io.github.linghengqian:hive-server2-jdbc-driver-thin` to verify the availability of HiveServer2 integration under GraalVM Native Image.
If developers use `org.apache.hive:hive-jdbc` directly, they should handle dependency conflicts and provide additional GraalVM Reachability Metadata by themselves.

8. Due to https://github.com/oracle/graal/issues/7979 ,
7. Due to https://github.com/oracle/graal/issues/7979 ,
the Oracle JDBC Driver corresponding to the `com.oracle.database.jdbc:ojdbc8` Maven module cannot be used under GraalVM Native Image.

9. Including but not limited to `com.mysql.cj.LocalizedErrorMessages`,
8. Including but not limited to `com.mysql.cj.LocalizedErrorMessages`,
`com.microsoft.sqlserver.jdbc.SQLServerResource`,
`org.postgresql.translation.messages`,
`org.opengauss.translation.messages` from third-party dependencies. By default, L10N resources are loaded according to the system's default locale,
Expand All @@ -351,11 +322,11 @@ without it being registered as reachable. Add it to the resource metadata to sol
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:186)
```

10. Due to the use of `janino-compiler/janino` by `apache/calcite`,
9. Due to the use of `janino-compiler/janino` by `apache/calcite`,
ShardingSphere's `SQL Federation` feature is unavailable in the GraalVM Native Image.
This also prevents ShardingSphere Proxy Native from integrating with OpenGauss.

11. Due to the issue at https://github.com/oracle/graal/issues/11280,
10. Due to the issue at https://github.com/oracle/graal/issues/11280,
Etcd's Cluster mode integration cannot be used on GraalVM Native Images compiled via Windows 11,
and Etcd's Cluster mode will conflict with the GraalVM Tracing Agent.
If developers need to use Etcd's Cluster mode on GraalVM Native Images compiled via Linux,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ ShardingSphere 对 ClickHouse JDBC Driver 的支持位于可选模块中。
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<classifier>http</classifier>
<version>0.6.3</version>
<classifier>all</classifier>
<version>0.9.4</version>
</dependency>
</dependencies>
```
Expand All @@ -43,7 +43,7 @@ ShardingSphere 对 ClickHouse JDBC Driver 的支持位于可选模块中。
```yaml
services:
clickhouse-server:
image: clickhouse/clickhouse-server:25.9.6.117
image: clickhouse/clickhouse-server:25.10.3.100
environment:
CLICKHOUSE_SKIP_USER_SETUP: "1"
ports:
Expand Down Expand Up @@ -167,24 +167,14 @@ public class ExampleUtils {
### SQL 限制

ShardingSphere JDBC DataSource 尚不支持执行 ClickHouse 的 `create table`,`truncate table` 和 `drop table` 语句。
用户应考虑为 ShardingSphere 提交包含单元测试的 PR。
当前 ShardingSphere 对 ClickHouse 的 `INNER JOIN` 语法解析存在不足,
对 `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id` 这类 SQL,它可能返回错误的查询结果。

### 分布式序列限制

ClickHouse 自身的,对应分布式序列功能的列类型是 `UUID`,`UUID` 在 ClickHouse JDBC Driver 中接收为 `java.util.UUID`,
参考 https://github.com/ClickHouse/ClickHouse/issues/56228 。
而 ShardingSphere 的 `SNOWFLAKE` 的分布式序列 SPI 实现对应的列类型是 `UInt64`,
在 ShardingSphere JDBC Driver 中接收为 `java.lang.Long`。

当为 ShardingSphere 配置连接至 ClickHouse 时, 若同时配置了 ShardingSphere 使用 `SNOWFLAKE` 的分布式序列 SPI 实现,
ShardingSphere 的分布式序列功能使用的 ClickHouse 真实数据库中的列类型不应该被设置为 `UUID`。

由于 `com.clickhouse:clickhouse-jdbc:0.6.3:http` Maven 模块的 `com.clickhouse.jdbc.ClickHouseConnection#prepareStatement(String, int)`
故意在 `autoGeneratedKeys` 为 `java.sql.Statement.RETURN_GENERATED_KEYS` 时抛出异常,
以阻止 ShardingSphere 正常代理 `com.clickhouse.jdbc.internal.ClickHouseConnectionImpl`,
因此如果用户需要从 JDBC 业务代码获取 ShardingSphere 生成的分布式序列,需要将 `autoGeneratedKeys` 置为 `java.sql.Statement.NO_GENERATED_KEYS`。

一个可能的示例如下,
受 https://github.com/ClickHouse/ClickHouse/issues/21697 影响,
由于 ClickHouse 不支持 `INSERT ... RETURNING` 语法,
开发者无法在向 ShardingSphere 的逻辑数据源执行 `INSERT` SQL 后获得分布式序列。即,如下操作是不允许的,

```java
import com.zaxxer.hikari.HikariConfig;
Expand All @@ -199,7 +189,7 @@ public class ExampleTest {
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement = connection.prepareStatement(
"INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')",
Statement.NO_GENERATED_KEYS
Statement.RETURN_GENERATED_KEYS
)) {
preparedStatement.executeUpdate();
try (ResultSet resultSet = preparedStatement.getGeneratedKeys()) {
Expand Down Expand Up @@ -227,9 +217,3 @@ ClickHouse 不支持 ShardingSphere 集成级别的本地事务,XA 事务或 S
嵌入式 ClickHouse `chDB` 尚未发布 Java 客户端,
ShardingSphere 不针对 SNAPSHOT 版本的 https://github.com/chdb-io/chdb-java 做集成测试。
参考 https://github.com/chdb-io/chdb/issues/243 。

### ClickHouse JDBC Driver V2 限制

ClickHouse JDBC Driver V2 自 https://github.com/ClickHouse/clickhouse-java/pull/2368 所在的 `0.8.6` 里程碑开始,
使用 `org.antlr:antlr4-maven-plugin:4.13.2`。这与 ShardingSphere 使用的 `org.antlr:antlr4-runtime:4.10.1` 产生冲突。
ShardingSphere 仅使用 `com.clickhouse:clickhouse-jdbc:0.6.3:http` 测试 ClickHouse 集成。
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ the possible Maven dependencies are as follows,
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<classifier>http</classifier>
<version>0.6.3</version>
<classifier>all</classifier>
<version>0.9.4</version>
</dependency>
</dependencies>
```
Expand All @@ -43,7 +43,7 @@ Write a Docker Compose file to start ClickHouse.
```yaml
services:
clickhouse-server:
image: clickhouse/clickhouse-server:25.9.6.117
image: clickhouse/clickhouse-server:25.10.3.100
environment:
CLICKHOUSE_SKIP_USER_SETUP: "1"
ports:
Expand Down Expand Up @@ -170,27 +170,15 @@ public class ExampleUtils {

ShardingSphere JDBC DataSource does not yet support executing ClickHouse's `create table`, `truncate table`,
and `drop table` statements.
Users should consider submitting a PR containing unit tests for ShardingSphere.
The current ShardingSphere parsing of ClickHouse's `INNER JOIN` syntax has shortcomings,
and it may return incorrect query results for SQL statements such as `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id`.

### Key Generate restrictions

The column type corresponding to the Key Generate function of ClickHouse itself is `UUID`,
and `UUID` is received as `java.util.UUID` in ClickHouse JDBC Driver,
refer to https://github.com/ClickHouse/ClickHouse/issues/56228 .
The column type corresponding to the Key Generate SPI implementation of ShardingSphere's `SNOWFLAKE` is `UInt64`,
which is received as `java.lang.Long` in ShardingSphere JDBC Driver.

When configuring ShardingSphere to connect to ClickHouse,
if ShardingSphere is also configured to use the Key Generate SPI implementation of `SNOWFLAKE`,
the column type in the ClickHouse real database used by ShardingSphere's Key Generate function should not be set to `UUID`.

Because `com.clickhouse.jdbc.ClickHouseConnection#prepareStatement(String, int)` of `com.clickhouse:clickhouse-jdbc:0.6.3:http`
Maven module intentionally throws an exception when `autoGeneratedKeys` is `java.sql.Statement.RETURN_GENERATED_KEYS`,
to prevent ShardingSphere from proxying `com.clickhouse.jdbc.internal.ClickHouseConnectionImpl` normally,
therefore, if users need to obtain the Key generated by ShardingSphere from the JDBC business code,
they need to set `autoGeneratedKeys` to `java.sql.Statement.NO_GENERATED_KEYS`.

A possible example is as follows,
Due to the issue mentioned in https://github.com/ClickHouse/ClickHouse/issues/21697 ,
because ClickHouse does not support the `INSERT ... RETURNING` syntax,
developers cannot obtain distributed sequences after executing `INSERT` SQL into ShardingSphere's logical data source.
Specifically, the following operations are not allowed:

```java
import com.zaxxer.hikari.HikariConfig;
Expand All @@ -205,7 +193,7 @@ public class ExampleTest {
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement = connection.prepareStatement(
"INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')",
Statement.NO_GENERATED_KEYS
Statement.RETURN_GENERATED_KEYS
)) {
preparedStatement.executeUpdate();
try (ResultSet resultSet = preparedStatement.getGeneratedKeys()) {
Expand Down Expand Up @@ -233,10 +221,3 @@ See https://github.com/ClickHouse/clickhouse-java/issues/2023 .
The embedded ClickHouse `chDB` Java client has not been released yet.
ShardingSphere does not do integration testing for the SNAPSHOT version of https://github.com/chdb-io/chdb-java .
Refer to https://github.com/chdb-io/chdb/issues/243 .

### Limitations of ClickHouse JDBC Driver V2

Starting from the `0.8.6` milestone at https://github.com/ClickHouse/clickhouse-java/pull/2368 ,
ClickHouse JDBC Driver V2 uses `org.antlr:antlr4-maven-plugin:4.13.2`.
This conflicts with `org.antlr:antlr4-runtime:4.10.1` used by ShardingSphere.
ShardingSphere only uses `com.clickhouse:clickhouse-jdbc:0.6.3:http` to test ClickHouse integration.
Original file line number Diff line number Diff line change
Expand Up @@ -366,9 +366,9 @@ ShardingSphere 仅针对 HiveServer2 `4.0.1` 进行集成测试。

### SQL 限制

HiveServer2 并不能保证每一条 `insert` 相关的 DML SQL 都能成功执行,尽管可能没有任何异常被抛出。

ShardingSphere JDBC DataSource 尚不支持执行 HiveServer2 的 `set` 语句。
当前 ShardingSphere 对 HiveServer2 的 `INNER JOIN` 语法解析存在不足,
对 `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id` 这类 SQL,它可能返回错误的查询结果。

#### 使用 `initFile` 参数部分绕开 SQL 限制

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -372,9 +372,9 @@ Reference https://issues.apache.org/jira/browse/HIVE-28418 .

### SQL Limitations

HiveServer2 does not guarantee that every `insert` related DML SQL can be executed successfully, although no exception may be thrown.

ShardingSphere JDBC DataSource does not yet support executing the `set` statement of HiveServer2.
The current ShardingSphere parsing of HiveServer2's `INNER JOIN` syntax has shortcomings,
and it may return incorrect query results for SQL statements such as `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id`.

#### Use `initFile` parameter to partially bypass SQL restrictions

Expand Down

This file was deleted.

Loading
Loading