Commit b21fb41
[SPARK-54565][CORE] SparkBuildInfo should load
### What changes were proposed in this pull request?
Change SparkBuildInfo to use its own classloader instead of thread context classloader to load `spark-version-info.properties`.
### Why are the changes needed?
I hit an issue during the Connect JDBC driver & JetBrains DataGrip integration.
```
2025-11-25 18:48:09,475 [ 55114] WARN - #c.i.d.d.BaseDatabaseErrorHandler$MissingDriverClassErrorInfo - Exception org.apache.spark.SparkException: Could not find spark-version-info.properties [in thread "RMI TCP Connection(3)-127.0.0.1"]
java.lang.ExceptionInInitializerError: Exception org.apache.spark.SparkException: Could not find spark-version-info.properties [in thread "RMI TCP Connection(3)-127.0.0.1"]
at org.apache.spark.SparkBuildInfo$.<clinit>(SparkBuildInfo.scala:35)
at org.apache.spark.sql.connect.client.SparkConnectClient$.org$apache$spark$sql$connect$client$SparkConnectClient$$genUserAgent(SparkConnectClient.scala:978)
at org.apache.spark.sql.connect.client.SparkConnectClient$Configuration$.apply$default$8(SparkConnectClient.scala:999)
at org.apache.spark.sql.connect.client.SparkConnectClient$Builder.<init>(SparkConnectClient.scala:683)
at org.apache.spark.sql.connect.client.SparkConnectClient$.builder(SparkConnectClient.scala:676)
at org.apache.spark.sql.connect.client.jdbc.SparkConnectConnection.<init>(SparkConnectConnection.scala:31)
at org.apache.spark.sql.connect.client.jdbc.NonRegisteringSparkConnectDriver.connect(NonRegisteringSparkConnectDriver.scala:36)
at com.intellij.database.remote.jdbc.helpers.JdbcHelperImpl.connect(JdbcHelperImpl.java:786)
at com.intellij.database.remote.jdbc.impl.RemoteDriverImpl.connect(RemoteDriverImpl.java:47)
```
After adding some debug messages, I found it was caused by using wrong classloader.
```
c.i.e.r.RemoteProcessSupport - ContextClassLoader: com.intellij.database.remote.jdbc.impl.JdbcClassLoader$1559cc356
c.i.e.r.RemoteProcessSupport - SparkBuildInfo ClassLoader: com.intellij.database.remote.jdbc.impl.JdbcClassLoader$JdbcClassLoaderImpl62e93ea8
```
Similar issue that affects Hive JDBC driver and Spark's Isolated Classloader (see SPARK-32256) was fixed by [HADOOP-14067](https://issues.apache.org/jira/browse/HADOOP-14067)
### Does this PR introduce _any_ user-facing change?
This fixes corner issues that the application uses multiple classloaders with Spark libs.
### How was this patch tested?
Pass GHA to ensure the change breaks nothing, also manually verified the Connect JDBC driver & JetBrains DataGrip integration.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53279 from pan3793/SPARK-54565.
Lead-authored-by: Cheng Pan <chengpan@apache.org>
Co-authored-by: Cheng Pan <pan3793@gmail.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
(cherry picked from commit 095c2c3)
Signed-off-by: yangjie01 <yangjie01@baidu.com>spark-version-info.properties from its own classloader1 parent ddc020a commit b21fb41
File tree
1 file changed
+2
-2
lines changed- common/utils/src/main/scala/org/apache/spark
1 file changed
+2
-2
lines changedLines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
33 | | - | |
| 32 | + | |
| 33 | + | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| |||
0 commit comments