Skip to content

Commit 25db5da

Browse files
committed
Change-Id: I423f052ca48915502f182cb4f1c67cdf04838a99
1 parent 82372d0 commit 25db5da

File tree

1 file changed

+2
-4
lines changed
  • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown

1 file changed

+2
-4
lines changed

hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/manifest_committer.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -317,7 +317,7 @@ None of the S3A committers support this. Condition (1) is not met by
317317
the staging committers, while (2) is not met by S3 itself.
318318

319319
To use the manifest committer with dynamic partition overwrites, the
320-
spark version must contain
320+
spark version must contain
321321
[SPARK-40034](https://issues.apache.org/jira/browse/SPARK-40034)
322322
_PathOutputCommitters to work with dynamic partition overwrite_.
323323

@@ -339,8 +339,6 @@ for SQL queries/Spark DataSet operations where many thousands of files are creat
339339
The fact that these will suffer from performance problems before
340340
throttling scale issues surface, should be considered a warning.
341341

342-
343-
344342
# <a name="SUCCESS"></a> Job Summaries in `_SUCCESS` files
345343

346344
The original hadoop committer creates a zero byte `_SUCCESS` file in the root of the output directory
@@ -673,7 +671,7 @@ For this to work, a number of conditions must be met:
673671
* All jobs/tasks must create files with unique filenames.
674672
* All jobs must create output with the same directory partition structure.
675673
* The job/queries MUST NOT be using Spark Dynamic Partitioning "INSERT OVERWRITE TABLE"; data may be lost.
676-
This holds for *all* committers, not just the manifest committer.
674+
This holds for *all* committers, not just the manifest committer.
677675
* Remember to delete the `_temporary` directory later!
678676

679677
This has *NOT BEEN TESTED*

0 commit comments

Comments
 (0)