Skip to content

Commit 332bb0e

Browse files
committed
[SPARK-49049][DOCS] Document MasterPage custom title conf and REST API server-side env variable replacements
### What changes were proposed in this pull request? This PR aims to document the following three recent improvements. - #47491 - #47509 - #47511 ### Why are the changes needed? To provide an updated documentation. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs and check the HTML manually. <img width="926" alt="Screenshot 2024-07-29 at 14 10 40" src="https://github.com/user-attachments/assets/6c904ec0-0ece-432a-8e41-aeb88f7baab8"> <img width="932" alt="Screenshot 2024-07-29 at 13 52 20" src="https://github.com/user-attachments/assets/ca3afe9a-dcfe-4258-b455-9ff4781cb4e5"> <img width="940" alt="Screenshot 2024-07-29 at 13 52 29" src="https://github.com/user-attachments/assets/ad9635d4-c66f-4320-8b93-005443d4df2e"> ### Was this patch authored or co-authored using generative AI tooling? No. Closes #47523 from dongjoon-hyun/SPARK-49049. Lead-authored-by: Dongjoon Hyun <dhyun@apple.com> Co-authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
1 parent 80c4432 commit 332bb0e

File tree

1 file changed

+28
-1
lines changed

1 file changed

+28
-1
lines changed

docs/spark-standalone.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,15 @@ SPARK_MASTER_OPTS supports the following system properties:
210210
</td>
211211
<td>1.1.0</td>
212212
</tr>
213+
<tr>
214+
<td><code>spark.master.ui.title</code></td>
215+
<td>(None)</td>
216+
<td>
217+
Specifies the title of the Master UI page. If unset, <code>Spark Master at 'master url'</code>
218+
is used by default.
219+
</td>
220+
<td>4.0.0</td>
221+
</tr>
213222
<tr>
214223
<td><code>spark.master.ui.decommission.allow.mode</code></td>
215224
<td><code>LOCAL</code></td>
@@ -601,7 +610,9 @@ via <code>http://[host:port]/[version]/submissions/[action]</code> where
601610
<thead><tr><th>Command</th><th>Description</th><th>HTTP METHOD</th><th>Since Version</th></tr></thead>
602611
<tr>
603612
<td><code>create</code></td>
604-
<td>Create a Spark driver via <code>cluster</code> mode.</td>
613+
<td>Create a Spark driver via <code>cluster</code> mode. Since 4.0.0, Spark master supports server-side
614+
variable replacements for the values of Spark properties and environment variables.
615+
</td>
605616
<td>POST</td>
606617
<td>1.3.0</td>
607618
</tr>
@@ -665,6 +676,22 @@ The following is the response from the REST API for the above <code>create</code
665676
}
666677
```
667678

679+
For <code>sparkProperties</code> and <code>environmentVariables</code>, users can use place
680+
holders for server-side environment variables like the following.
681+
682+
```bash
683+
{% raw %}
684+
...
685+
"sparkProperties": {
686+
"spark.hadoop.fs.s3a.endpoint": "{{AWS_ENDPOINT_URL}}",
687+
"spark.hadoop.fs.s3a.endpoint.region": "{{AWS_REGION}}"
688+
},
689+
"environmentVariables": {
690+
"AWS_CA_BUNDLE": "{{AWS_CA_BUNDLE}}"
691+
},
692+
...
693+
{% endraw %}
694+
```
668695

669696
# Resource Scheduling
670697

0 commit comments

Comments
 (0)