diff --git a/docs/core-migration-guide.md b/docs/core-migration-guide.md index 28a9dd0f43715..76ccfc66746ef 100644 --- a/docs/core-migration-guide.md +++ b/docs/core-migration-guide.md @@ -48,7 +48,9 @@ license: | - Since Spark 4.0, the MDC (Mapped Diagnostic Context) key for Spark task names in Spark logs has been changed from `mdc.taskName` to `task_name`. To use the key `mdc.taskName`, you can set `spark.log.legacyTaskNameMdc.enabled` to `true`. -- Since Spark 4.0, Spark performs speculative executions less agressively with `spark.speculation.multiplier=3` and `spark.speculation.quantile=0.9`. To restore the legacy behavior, you can set `spark.speculation.multiplier=1.5` and `spark.speculation.quantile=0.75`. +- Since Spark 4.0, Spark performs speculative executions less aggressively with `spark.speculation.multiplier=3` and `spark.speculation.quantile=0.9`. To restore the legacy behavior, you can set `spark.speculation.multiplier=1.5` and `spark.speculation.quantile=0.75`. + +- Since Spark 4.0, when configurations are provided to `spark-submit` or `spark-shell` via `--properties-file` parameter, Spark will also load extra configurations from `SPARK_CONF_DIR/spark-defaults.conf`. Those configurations defined in the former take higher precedence over those from the latter. ## Upgrading from Core 3.4 to 3.5