HADOOP-16147: Allow CopyListing sequence file keys and values to be m…

[noslowerdna]

…ore easily customized

HADOOP-16147

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	86	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	1290	trunk passed
+1	compile	32	trunk passed
+1	checkstyle	29	trunk passed
+1	mvnsite	33	trunk passed
+1	shadedclient	832	branch has no errors when building and testing our client artifacts.
+1	findbugs	48	trunk passed
+1	javadoc	27	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	28	the patch passed
+1	compile	25	the patch passed
+1	javac	25	the patch passed
-0	checkstyle	18	hadoop-tools/hadoop-distcp: The patch generated 6 new + 42 unchanged - 0 fixed = 48 total (was 42)
+1	mvnsite	29	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	900	patch has no errors when building and testing our client artifacts.
+1	findbugs	48	the patch passed
+1	javadoc	21	the patch passed
		_ Other Tests _
+1	unit	902	hadoop-distcp in the patch passed.
+1	asflicense	32	The patch does not generate ASF License warnings.
		4505

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-517/1/artifact/out/Dockerfile
GITHUB PR	#517
JIRA Issue	HADOOP-16147
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 01c252a2f85e 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 9de34d2
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-517/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-distcp.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-517/1/testReport/
Max. process+thread count	294 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-517/1/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

[noslowerdna]

Responding to @hadoop-yetus,

Please justify why no new tests are needed for this patch.

Only minor refactoring of current code was done, allowing for the more specific behavior override. No functionality was modified.

Also please list what manual steps were performed to verify this patch.

In a separate project, we created a custom CopyListing implementation with the getFileListingKey method overridden to return a different key, and then successfully ran a distributed copy producing the desired alternative target paths by setting distcp.copy.listing.class to the name of that custom class.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	24	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	973	trunk passed
+1	compile	27	trunk passed
+1	checkstyle	21	trunk passed
+1	mvnsite	31	trunk passed
+1	shadedclient	717	branch has no errors when building and testing our client artifacts.
+1	findbugs	34	trunk passed
+1	javadoc	16	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	25	the patch passed
+1	compile	22	the patch passed
+1	javac	22	the patch passed
+1	checkstyle	13	the patch passed
+1	mvnsite	24	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	714	patch has no errors when building and testing our client artifacts.
+1	findbugs	42	the patch passed
+1	javadoc	19	the patch passed
		_ Other Tests _
+1	unit	742	hadoop-distcp in the patch passed.
+1	asflicense	28	The patch does not generate ASF License warnings.
		3554

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-517/2/artifact/out/Dockerfile
GITHUB PR	#517
JIRA Issue	HADOOP-16147
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 32c6e768a325 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 6c8c422
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-517/2/testReport/
Max. process+thread count	445 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-517/2/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

[noslowerdna]

This was committed here,

• 3.3.0: faba359
• 3.2.1: ade3af6