address comments

sunchao · May 29, 2024 · d80e5dd · d80e5dd
1 parent e1ff4cc
commit d80e5dd
Show file tree

Hide file tree

Showing 2 changed files with 2 additions and 2 deletions.
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
@@ -88,7 +88,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
   /** Default properties present in the currently defined defaults file. */
   lazy val defaultSparkProperties: HashMap[String, String] = {
     val defaultProperties = new HashMap[String, String]()
-    if (verbose) {
+    if (verbose && propertiesFile != null) {
       logInfo(log"Using properties file: ${MDC(PATH, propertiesFile)}")
     }
     Option(propertiesFile).foreach { filename =>

diff --git a/docs/core-migration-guide.md b/docs/core-migration-guide.md
@@ -50,7 +50,7 @@ license: |
 
 - Since Spark 4.0, Spark performs speculative executions less aggressively with `spark.speculation.multiplier=3` and `spark.speculation.quantile=0.9`. To restore the legacy behavior, you can set `spark.speculation.multiplier=1.5` and `spark.speculation.quantile=0.75`.
 
-- Since Spark 4.0, when configurations are provided to `spark-submit` or `spark-shell` via `--properties-file` parameter, Spark will also load extra configurations from `SPARK_CONF_DIR/spark-defaults.conf`. Those configurations defined in the former take higher precedence over those from the latter.
+- Since Spark 4.0, when configurations are provided to `spark-submit` or `spark-shell` via `--properties-file` parameter, Spark will also load extra configurations from `SPARK_CONF_DIR/spark-defaults.conf`. Those configurations defined in the former take higher precedence over those from the latter. Other components such as Spark thrift server, connect server, SQL shell that call `spark-submit` internally will also have the same behavior change.
 
 ## Upgrading from Core 3.4 to 3.5