Skip to content

Commit 08b9601

Browse files
author
Fangshi Li
committed
[SPARK-23815]follow up. further improve comment
1 parent a9342cc commit 08b9601

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -190,8 +190,8 @@ class HadoopMapReduceCommitProtocol(
190190
// According to the official hadoop FileSystem API spec, delete op should assume
191191
// the destination is no longer present regardless of return value, thus we do not
192192
// need to double check if finalPartPath exists before rename.
193-
// Also, in our case, delete should return false only when finalPartPath does not
194-
// exist. When this happens, we need to take action only if parent of finalPartPath
193+
// Also in our case, based on the spec, delete returns false only when finalPartPath
194+
// does not exist. When this happens, we need to take action if parent of finalPartPath
195195
// also does not exist(e.g. the scenario described on SPARK-23815), because
196196
// FileSystem API spec on rename op says the rename dest(finalPartPath) must have
197197
// a parent that exists, otherwise we may get unexpected result on the rename.

0 commit comments

Comments
 (0)