Skip to content

Commit

Permalink
[KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
Browse files Browse the repository at this point in the history
# 🔍 Description
## Issue References 🔗

I get reported by a user that using Kyuubi JDBC driver 1.8.0 to access Spark Thrift Server 2.4 (with Hive 1.2.1) with getting an error on opening the session even with `clientProtocol=7`

```
org.apache.hive.service.cli.HiveSQLException: java.lang.IllegalArgumentException: hive configuration hive.server2.thrift.resultset.default.fetch.size does not exists.
        at org.apache.hive.service.cli.session.HiveSessionImpl.configureSession(HiveSessionImpl.java:220)
        at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:154)
        at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258)
        ... 13 more
```

## Describe Your Solution 🔧

When `hive.conf.validation` is `true` (it also is the default value), an IllegalArgumentException will be thrown if the provided configurations start with `hive.` and can not be recognized by the server.

One solution is to disable `hive.conf.validation` on the server side, but we can also address it by avoiding passing this configuration if we know that the server does not support it.

HIVE-14901 (2.3.0, HIVE_CLI_SERVICE_PROTOCOL_V10) introduces `hive.server2.thrift.resultset.default.fetch.size`, so we can set this configuration only when protocol >= HIVE_CLI_SERVICE_PROTOCOL_V10

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Verified locally by connecting to HS2 2.1.1 and HS2 2.3.9

#### Behavior Without This Pull Request ⚰️

IllegalArgumentException throws when using  `jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1

Everything is OK when using `jdbc:hive2://localhost:10000/default` to connect HS2 2.3.9

#### Behavior With This Pull Request 🎉

Everything is OK when using `jdbc:hive2://localhost:10000/default;clientProtocol=8` to connect HS2 2.1.1

Everything is OK when using `jdbc:hive2://localhost:10000/default` to connect HS2 2.3.9

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6036 from pan3793/jdbc-fetchsize.

Closes #6036

7ea91f1 [Cheng Pan] nit
e6ea829 [Cheng Pan] fix
1dbecbb [Cheng Pan] JDBC driver conditional sets fetchSize on opening session

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
  • Loading branch information
pan3793 committed Jan 31, 2024
1 parent 208354c commit 4bd259a
Showing 1 changed file with 6 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -723,10 +723,6 @@ private void openSession() throws SQLException {
}
// switch the database
openConf.put("use:database", connParams.getDbName());
// set the fetchSize
openConf.put(
"set:hiveconf:hive.server2.thrift.resultset.default.fetch.size",
Integer.toString(fetchSize));
if (wmPool != null) {
openConf.put("set:hivevar:wmpool", wmPool);
}
Expand All @@ -751,6 +747,12 @@ private void openSession() throws SQLException {
clientProtocolStr, CLIENT_PROTOCOL_VERSION));
}
openReq.setClient_protocol(clientProtocol);
// HIVE-14901: set the fetchSize
if (clientProtocol.compareTo(TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V10) >= 0) {
openConf.put(
"set:hiveconf:hive.server2.thrift.resultset.default.fetch.size",
Integer.toString(fetchSize));
}
try {
openConf.put("kyuubi.client.ipAddress", InetAddress.getLocalHost().getHostAddress());
} catch (UnknownHostException e) {
Expand Down

0 comments on commit 4bd259a

Please sign in to comment.