Skip to content

Commit

Permalink
[apache#1124] docs: Add the document for tez-client (apache#1125)
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Add the document for tez-client.

### Why are the changes needed?

Fix: apache#1124 

### Does this PR introduce _any_ user-facing change?
No.
  • Loading branch information
bin41215 authored Aug 9, 2023
1 parent 3a6a9e3 commit 2f46358
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions docs/client_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ and Continuous partition assignment mechanism.
# Default value is 1.0, used to estimate task concurrency, how likely is this part of the resource between spark.dynamicAllocation.minExecutors and spark.dynamicAllocation.maxExecutors to be allocated
--conf spark.rss.estimate.task.concurrency.dynamic.factor=1.0
```

Since v0.8.0, `RssShuffleManager` would disable local shuffle reader(`set spark.sql.adaptive.localShuffleReader.enabled=false`) optimization by default.

Local shuffle reader as its name indicates is suitable and optimized for spark's external shuffle service, and shall not be used for remote shuffle service. It would cause many random small IOs and network connections with Uniffle's shuffle server
Expand Down Expand Up @@ -164,7 +164,7 @@ Other configuration:
|spark.rss.access.timeout.ms|10000|The timeout to access Uniffle coordinator|
|spark.rss.client.access.retry.interval.ms|20000|The interval between retries fallback to SortShuffleManager|
|spark.rss.client.access.retry.times|0|The number of retries fallback to SortShuffleManager|


### Client Quorum Setting

Expand Down Expand Up @@ -241,7 +241,8 @@ Notice: this feature requires the MEMORY_LOCAL_HADOOP mode.
| Property Name | Default | Description |
|--------------------------------|---------|-------------------------------------------------------------------------|
| tez.rss.avoid.recompute.succeeded.task | false | Whether to avoid recompute succeeded task when node is unhealthy or black-listed |

| tez.rss.client.max.buffer.size | 3k | The max buffer size in map side |

### Netty Setting
| Property Name | Default | Description |
|-----------------------------------------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand Down

0 comments on commit 2f46358

Please sign in to comment.