Skip to content

Commit 88c8cc3

Browse files
zzzzming95bowenliang123
authored andcommitted
[KYUUBI #5807] Simple norm the use of kyuubi.operation.result.arrow.timestampAsString and kyuubi.operation.result.format
# 🔍 Description ## Issue References 🔗 This pull request norm the arrow conf. ## Describe Your Solution 🔧 Using `kyuubi.operation.result.arrow.timestampAsString` and `kyuubi.operation.result.format` string to get conf . To replace as `ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING.key` and `OPERATION_RESULT_FORMAT`. Make it more norm. ## Types of changes 🔖 - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Other ## Test Plan 🧪 No need test --- # Checklists ## 📝 Author Self Checklist - [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project - [x] I have performed a self-review - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) ## 📝 Committer Pre-Merge Checklist - [ ] Pull request title is okay. - [ ] No license issues. - [ ] Milestone correctly set? - [ ] Test coverage is ok - [ ] Assignees are selected. - [ ] Minimum number of approvals - [ ] No changes are requested **Be nice. Be informative.** Closes #5807 from zzzzming95/norm-conf. Closes #5807 9e65bfa [佘志铭] norm the arrow conf Authored-by: 佘志铭 <shezhiming@corp.netease.com> Signed-off-by: liangbowen <liangbowen@gf.com.cn>
1 parent 13af6ae commit 88c8cc3

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ import org.apache.spark.sql.types.StructType
2929

3030
import org.apache.kyuubi.{KyuubiSQLException, Utils}
3131
import org.apache.kyuubi.config.KyuubiConf
32-
import org.apache.kyuubi.config.KyuubiConf.{OPERATION_SPARK_LISTENER_ENABLED, SESSION_PROGRESS_ENABLE, SESSION_USER_SIGN_ENABLED}
32+
import org.apache.kyuubi.config.KyuubiConf.{ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING, OPERATION_SPARK_LISTENER_ENABLED, SESSION_PROGRESS_ENABLE, SESSION_USER_SIGN_ENABLED}
3333
import org.apache.kyuubi.config.KyuubiReservedKeys.{KYUUBI_SESSION_SIGN_PUBLICKEY, KYUUBI_SESSION_USER_KEY, KYUUBI_SESSION_USER_SIGN, KYUUBI_STATEMENT_ID_KEY}
3434
import org.apache.kyuubi.engine.spark.KyuubiSparkUtil.{getSessionConf, SPARK_SCHEDULER_POOL_KEY}
3535
import org.apache.kyuubi.engine.spark.events.SparkOperationEvent
@@ -271,7 +271,7 @@ abstract class SparkOperation(session: Session)
271271
protected def resultFormat: String = "thrift"
272272

273273
protected def timestampAsString: Boolean = {
274-
spark.conf.get("kyuubi.operation.result.arrow.timestampAsString", "false").toBoolean
274+
spark.conf.get(ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING.key, "false").toBoolean
275275
}
276276

277277
protected def setSessionUserSign(): Unit = {

externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkSQLOperationManager.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ class SparkSQLOperationManager private (name: String) extends OperationManager(n
8686
val incrementalCollect = spark.conf.getOption(OPERATION_INCREMENTAL_COLLECT.key)
8787
.map(_.toBoolean).getOrElse(operationIncrementalCollectDefault)
8888
// TODO: respect the config of the operation ExecuteStatement, if it was set.
89-
val resultFormat = spark.conf.get("kyuubi.operation.result.format", "thrift")
89+
val resultFormat = spark.conf.get(OPERATION_RESULT_FORMAT.key, "thrift")
9090
resultFormat.toLowerCase match {
9191
case "arrow" =>
9292
new ArrowBasedExecuteStatement(

0 commit comments

Comments
 (0)