Skip to content

Commit 273f2c8

Browse files
author
Andrew Or
committed
[SPARK-4771][Docs] Document standalone cluster supervise mode
tdas looks like streaming already refers to the supervise mode. The link from there is broken though. Author: Andrew Or <andrew@databricks.com> Closes #3627 from andrewor14/document-supervise and squashes the following commits: 9ca0908 [Andrew Or] Wording changes 2b55ed2 [Andrew Or] Document standalone cluster supervise mode
1 parent 6dcafa7 commit 273f2c8

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

docs/spark-standalone.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,7 @@ To run an interactive Spark shell against the cluster, run the following command
244244

245245
You can also pass an option `--total-executor-cores <numCores>` to control the number of cores that spark-shell uses on the cluster.
246246

247-
# Launching Compiled Spark Applications
247+
# Launching Spark Applications
248248

249249
The [`spark-submit` script](submitting-applications.html) provides the most straightforward way to
250250
submit a compiled Spark application to the cluster. For standalone clusters, Spark currently
@@ -259,6 +259,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. `
259259
To control the application's configuration or execution environment, see
260260
[Spark Configuration](configuration.html).
261261

262+
Additionally, standalone `cluster` mode supports restarting your application automatically if it
263+
exited with non-zero exit code. To use this feature, you may pass in the `--supervise` flag to
264+
`spark-submit` when launching your application. Then, if you wish to kill an application that is
265+
failing repeatedly, you may do so through:
266+
267+
./bin/spark-class org.apache.spark.deploy.Client kill <master url> <driver ID>
268+
269+
You can find the driver ID through the standalone Master web UI at `http://<master url>:8080`.
270+
262271
# Resource Scheduling
263272

264273
The standalone cluster mode currently only supports a simple FIFO scheduler across applications.

0 commit comments

Comments
 (0)