You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-23941][MESOS] Mesos task failed on specific spark app name
## What changes were proposed in this pull request?
Shell escaped the name passed to spark-submit and change how conf attributes are shell escaped.
## How was this patch tested?
This test has been tested manually with Hive-on-spark with mesos or with the use case described in the issue with the sparkPi application with a custom name which contains illegal shell characters.
With this PR, hive-on-spark on mesos works like a charm with hive 3.0.0-SNAPSHOT.
I state that this contribution is my original work and that I license the work to the project under the project’s open source license
Author: Bounkong Khamphousone <bounkong.khamphousone@ebiznext.com>
Closes#21014 from tiboun/fix/SPARK-23941.
(cherry picked from commit 6782359)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
Copy file name to clipboardExpand all lines: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -530,9 +530,9 @@ private[spark] class MesosClusterScheduler(
530
530
.filter { case (key, _) =>!replicatedOptionsBlacklist.contains(key) }
531
531
.toMap
532
532
(defaultConf ++ driverConf).foreach { case (key, value) =>
0 commit comments