Commit 403c8d5
[SPARK-26287][CORE] Don't need to create an empty spill file when memory has no records
## What changes were proposed in this pull request?
If there are no records in memory, then we don't need to create an empty temp spill file.
## How was this patch tested?
Existing tests
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.
Closes apache#23225 from wangjiaochun/ShufflSorter.
Authored-by: 10087686 <wang.jiaochun@zte.com.cn>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>1 parent ec506bd commit 403c8d5
File tree
2 files changed
+10
-4
lines changed- core/src
- main/java/org/apache/spark/shuffle/sort
- test/java/org/apache/spark/shuffle/sort
2 files changed
+10
-4
lines changedLines changed: 9 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
148 | 157 | | |
149 | 158 | | |
150 | 159 | | |
| |||
157 | 166 | | |
158 | 167 | | |
159 | 168 | | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | 169 | | |
165 | 170 | | |
166 | 171 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
235 | 235 | | |
236 | 236 | | |
237 | 237 | | |
| 238 | + | |
238 | 239 | | |
239 | 240 | | |
240 | 241 | | |
| |||
0 commit comments