Skip to content

Commit 9cbc072

Browse files
author
Neelesh Srinivas Salian
committed
Updated a few lines in the Launching Spark on YARN Section
1 parent 8e8db7f commit 9cbc072

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

docs/running-on-yarn.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,10 @@ Java system properties or environment variables not managed by YARN, they should
1717
Spark application's configuration (driver, executors, and the AM when running in client mode).
1818

1919
There are two deploy modes that can be used to launch Spark applications on YARN. In yarn-cluster mode, the Spark driver runs inside an application master process which is managed by YARN on the cluster, and the client can go away after initiating the application. In yarn-client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN.
20+
(Default: --deploy-mode client)
21+
22+
Unlike in Spark standalone and Mesos mode, in which the master's address is specified in the "master" parameter, in YARN mode the ResourceManager's address is picked up from the Hadoop configuration. Thus, the master parameter is yarn.
2023

21-
Unlike in Spark standalone and Mesos mode, in which the master's address is specified in the "master" parameter, in YARN mode the ResourceManager's address is picked up from the Hadoop configuration. Thus, the master parameter is simply “yarn-client” or “yarn-cluster”.
22-
2324
To launch a Spark application in yarn-cluster mode:
2425

2526
./bin/spark-submit --class path.to.your.Class --master yarn-cluster [options] <app jar> [app options]

0 commit comments

Comments
 (0)