MAPREDUCE-7448. Skipping cleanup with FileOutputCommitter V1 can corrupt output: warn and document (#6038)

* update documentation for mapreduce committer
* add warning if the user attempts to use FileOutputCommiter V1 with skipping cleanup

Contributed by ConfX

Author: teamconfx

hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java

@@ -158,6 +158,11 @@ public FileOutputCommitter(Path outputPath,
         "output directory:" + skipCleanup + ", ignore cleanup failures: " +
         ignoreCleanupFailures);
 
+    if (algorithmVersion == 1 && skipCleanup) {
+        LOG.warn("Skip cleaning up when using FileOutputCommitter V1 can lead to unexpected behaviors. " +
+                "For example, committing several times may be allowed falsely.");
+    }
+
     if (outputPath != null) {
       FileSystem fs = outputPath.getFileSystem(context.getConfiguration());
       this.outputPath = fs.makeQualified(outputPath);

hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/manifest_committer.md

@@ -202,6 +202,7 @@ Here are the main configuration options of the committer.
 
 There are some more, as covered in the (Advanced)[#advanced] section.
 
+WARNING: setting `mapreduce.fileoutputcommitter.cleanup.skipped` to `true` is not compatible with version 1 of the committer and can cause unexpected behaviors.
 
 ## <a name="scaling"></a> Scaling jobs `mapreduce.manifest.committer.io.threads`