HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries … #802

bgaborg · 2019-05-08T11:19:03Z

…(and tombstones)

hadoop-yetus · 2019-05-08T13:24:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	39	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 4 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	71	Maven dependency ordering for branch
+1	mvninstall	1050	trunk passed
+1	compile	1017	trunk passed
+1	checkstyle	144	trunk passed
+1	mvnsite	131	trunk passed
+1	shadedclient	1003	branch has no errors when building and testing our client artifacts.
+1	javadoc	93	trunk passed
0	spotbugs	65	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	179	trunk passed
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	80	the patch passed
+1	compile	966	the patch passed
+1	javac	966	the patch passed
-0	checkstyle	139	root: The patch generated 1 new + 18 unchanged - 2 fixed = 19 total (was 20)
-1	mvnsite	49	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	739	patch has no errors when building and testing our client artifacts.
+1	javadoc	89	the patch passed
+1	findbugs	193	the patch passed
		_ Other Tests _
+1	unit	512	hadoop-common in the patch passed.
-1	unit	279	hadoop-aws in the patch failed.
+1	asflicense	53	The patch does not generate ASF License warnings.
		6930

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestNullMetadataStore
	hadoop.fs.s3a.s3guard.TestLocalMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 72ea7fc7de89 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `96dc5ce`
Default Java	1.8.0_191
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/testReport/
Max. process+thread count	1381 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

ajfabbri

Will comment on the bigger picture stuff on the JIRA.

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java

hadoop-common-project/hadoop-common/src/main/resources/core-default.xml

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/MetadataStore.java

hadoop-yetus · 2019-05-22T21:17:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	51	Docker mode activated.
		_ Prechecks _
+1	dupname	1	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 4 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	23	Maven dependency ordering for branch
+1	mvninstall	1172	trunk passed
+1	compile	1190	trunk passed
+1	checkstyle	150	trunk passed
+1	mvnsite	136	trunk passed
+1	shadedclient	1087	branch has no errors when building and testing our client artifacts.
+1	javadoc	108	trunk passed
0	spotbugs	76	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	220	trunk passed
		_ Patch Compile Tests _
0	mvndep	25	Maven dependency ordering for patch
-1	mvninstall	32	hadoop-aws in the patch failed.
-1	compile	1345	root in the patch failed.
-1	javac	1345	root in the patch failed.
-0	checkstyle	172	root: The patch generated 1 new + 33 unchanged - 2 fixed = 34 total (was 35)
-1	mvnsite	46	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	2	The patch has no ill-formed XML file.
+1	shadedclient	747	patch has no errors when building and testing our client artifacts.
+1	javadoc	102	the patch passed
-1	findbugs	43	hadoop-aws in the patch failed.
		_ Other Tests _
+1	unit	616	hadoop-common in the patch passed.
-1	unit	50	hadoop-aws in the patch failed.
+1	asflicense	56	The patch does not generate ASF License warnings.
		7632

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux b59acd366d03 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 9c61494
Default Java	1.8.0_212
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-compile-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-compile-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/testReport/
Max. process+thread count	1382 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-27T19:11:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	47	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 6 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	75	Maven dependency ordering for branch
+1	mvninstall	1076	trunk passed
+1	compile	1031	trunk passed
+1	checkstyle	143	trunk passed
+1	mvnsite	128	trunk passed
+1	shadedclient	986	branch has no errors when building and testing our client artifacts.
+1	javadoc	107	trunk passed
0	spotbugs	68	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	188	trunk passed
		_ Patch Compile Tests _
0	mvndep	24	Maven dependency ordering for patch
-1	mvninstall	30	hadoop-aws in the patch failed.
-1	compile	940	root in the patch failed.
-1	javac	940	root in the patch failed.
-0	checkstyle	141	root: The patch generated 8 new + 46 unchanged - 3 fixed = 54 total (was 49)
-1	mvnsite	46	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	2	The patch has no ill-formed XML file.
+1	shadedclient	679	patch has no errors when building and testing our client artifacts.
+1	javadoc	102	the patch passed
-1	findbugs	44	hadoop-aws in the patch failed.
		_ Other Tests _
+1	unit	506	hadoop-common in the patch passed.
-1	unit	45	hadoop-aws in the patch failed.
-1	asflicense	52	The patch generated 17 ASF License warnings.
		6676

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 162ab21d8a6d 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `83549db`
Default Java	1.8.0_212
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-compile-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-compile-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/testReport/
asflicense	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/artifact/out/patch-asflicense-problems.txt
Max. process+thread count	1463 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-29T11:50:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	34	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 6 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	26	Maven dependency ordering for branch
+1	mvninstall	1156	trunk passed
+1	compile	1142	trunk passed
+1	checkstyle	141	trunk passed
+1	mvnsite	123	trunk passed
+1	shadedclient	971	branch has no errors when building and testing our client artifacts.
+1	javadoc	90	trunk passed
0	spotbugs	64	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	189	trunk passed
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	81	the patch passed
+1	compile	1146	the patch passed
+1	javac	1146	the patch passed
-0	checkstyle	138	root: The patch generated 10 new + 46 unchanged - 3 fixed = 56 total (was 49)
-1	mvnsite	47	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	4	The patch has no ill-formed XML file.
+1	shadedclient	664	patch has no errors when building and testing our client artifacts.
+1	javadoc	90	the patch passed
+1	findbugs	197	the patch passed
		_ Other Tests _
+1	unit	536	hadoop-common in the patch passed.
-1	unit	286	hadoop-aws in the patch failed.
+1	asflicense	39	The patch does not generate ASF License warnings.
		7174

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestLocalMetadataStore
	hadoop.fs.s3a.s3guard.TestNullMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 20e6b109ed7e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `544876f`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/testReport/
Max. process+thread count	1361 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/5/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-29T17:54:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	62	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 6 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	30	Maven dependency ordering for branch
+1	mvninstall	1043	trunk passed
+1	compile	1021	trunk passed
+1	checkstyle	144	trunk passed
+1	mvnsite	114	trunk passed
+1	shadedclient	958	branch has no errors when building and testing our client artifacts.
+1	javadoc	95	trunk passed
0	spotbugs	61	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	174	trunk passed
		_ Patch Compile Tests _
0	mvndep	21	Maven dependency ordering for patch
+1	mvninstall	72	the patch passed
+1	compile	962	the patch passed
+1	javac	962	the patch passed
-0	checkstyle	139	root: The patch generated 9 new + 46 unchanged - 3 fixed = 55 total (was 49)
-1	mvnsite	48	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	2	The patch has no ill-formed XML file.
+1	shadedclient	685	patch has no errors when building and testing our client artifacts.
+1	javadoc	102	the patch passed
+1	findbugs	206	the patch passed
		_ Other Tests _
+1	unit	541	hadoop-common in the patch passed.
-1	unit	295	hadoop-aws in the patch failed.
+1	asflicense	46	The patch does not generate ASF License warnings.
		6868

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestLocalMetadataStore
	hadoop.fs.s3a.s3guard.TestNullMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 177afef00d36 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `544876f`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/testReport/
Max. process+thread count	1424 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/6/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-30T14:33:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	26	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	82	Maven dependency ordering for branch
+1	mvninstall	1122	trunk passed
+1	compile	995	trunk passed
+1	checkstyle	146	trunk passed
+1	mvnsite	123	trunk passed
+1	shadedclient	1043	branch has no errors when building and testing our client artifacts.
+1	javadoc	94	trunk passed
0	spotbugs	63	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	182	trunk passed
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	76	the patch passed
+1	compile	949	the patch passed
+1	javac	949	the patch passed
-0	checkstyle	148	root: The patch generated 23 new + 49 unchanged - 3 fixed = 72 total (was 52)
-1	mvnsite	44	hadoop-aws in the patch failed.
+1	whitespace	1	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	729	patch has no errors when building and testing our client artifacts.
+1	javadoc	94	the patch passed
+1	findbugs	207	the patch passed
		_ Other Tests _
+1	unit	568	hadoop-common in the patch passed.
-1	unit	300	hadoop-aws in the patch failed.
+1	asflicense	47	The patch does not generate ASF License warnings.
		7076

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestNullMetadataStore
	hadoop.fs.s3a.s3guard.TestLocalMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 2f7898e48f62 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `18c1eeb`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/testReport/
Max. process+thread count	1578 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/7/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-30T14:43:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	33	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	108	Maven dependency ordering for branch
+1	mvninstall	1184	trunk passed
+1	compile	1045	trunk passed
+1	checkstyle	144	trunk passed
+1	mvnsite	123	trunk passed
+1	shadedclient	1055	branch has no errors when building and testing our client artifacts.
+1	javadoc	90	trunk passed
0	spotbugs	67	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	195	trunk passed
		_ Patch Compile Tests _
0	mvndep	23	Maven dependency ordering for patch
+1	mvninstall	90	the patch passed
+1	compile	1026	the patch passed
+1	javac	1026	the patch passed
-0	checkstyle	145	root: The patch generated 23 new + 49 unchanged - 3 fixed = 72 total (was 52)
-1	mvnsite	41	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	715	patch has no errors when building and testing our client artifacts.
+1	javadoc	92	the patch passed
+1	findbugs	213	the patch passed
		_ Other Tests _
+1	unit	520	hadoop-common in the patch passed.
-1	unit	272	hadoop-aws in the patch failed.
+1	asflicense	43	The patch does not generate ASF License warnings.
		7246

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestLocalMetadataStore
	hadoop.fs.s3a.s3guard.TestNullMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 6afc1c03219d 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `18c1eeb`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/testReport/
Max. process+thread count	1344 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/8/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-05-31T16:08:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	34	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	19	Maven dependency ordering for branch
+1	mvninstall	1043	trunk passed
+1	compile	1047	trunk passed
+1	checkstyle	130	trunk passed
+1	mvnsite	117	trunk passed
+1	shadedclient	946	branch has no errors when building and testing our client artifacts.
+1	javadoc	96	trunk passed
0	spotbugs	64	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	178	trunk passed
		_ Patch Compile Tests _
0	mvndep	21	Maven dependency ordering for patch
+1	mvninstall	75	the patch passed
+1	compile	963	the patch passed
+1	javac	963	the patch passed
-0	checkstyle	148	root: The patch generated 23 new + 49 unchanged - 3 fixed = 72 total (was 52)
-1	mvnsite	42	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	656	patch has no errors when building and testing our client artifacts.
+1	javadoc	93	the patch passed
+1	findbugs	198	the patch passed
		_ Other Tests _
+1	unit	533	hadoop-common in the patch passed.
-1	unit	281	hadoop-aws in the patch failed.
+1	asflicense	43	The patch does not generate ASF License warnings.
		6752

Reason	Tests
Failed junit tests	hadoop.fs.s3a.s3guard.TestNullMetadataStore

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux a0aa837af83d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `1ae062c`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/testReport/
Max. process+thread count	1573 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/9/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ITtlTimeProvider.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/MetadataStore.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/NullMetadataStore.java

...tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardOutOfBandOperations.java

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java

...doop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStoreScale.java

...p-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/MetadataStoreTestBase.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java

steveloughran · 2019-06-06T13:21:53Z

Production code looks OK, with one exception.

The exception: switch to Configuration.getTimeDuration() to read the time in, and make the default value seconds. Users can switch to any other unit they want by changing the suffix. Doing this now avoids the problems encountered when retrofitting other bits of the code (block size) to using more usable configuration values.

test-wise we need to add one which verifies a delete of an expired metadata entry deletes any new raw file.

hadoop-yetus · 2019-06-06T14:50:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	35	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	70	Maven dependency ordering for branch
-1	mvninstall	52	root in trunk failed.
-1	compile	48	root in trunk failed.
+1	checkstyle	127	trunk passed
-1	mvnsite	32	hadoop-common in trunk failed.
-1	shadedclient	270	branch has errors when building and testing our client artifacts.
+1	javadoc	65	trunk passed
0	spotbugs	50	Used deprecated FindBugs config; considering switching to SpotBugs.
-1	findbugs	24	hadoop-common in trunk failed.
		_ Patch Compile Tests _
0	mvndep	20	Maven dependency ordering for patch
-1	mvninstall	24	hadoop-common in the patch failed.
-1	compile	44	root in the patch failed.
-1	javac	44	root in the patch failed.
-0	checkstyle	129	root: The patch generated 4 new + 49 unchanged - 3 fixed = 53 total (was 52)
-1	mvnsite	27	hadoop-common in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	2	The patch has no ill-formed XML file.
-1	shadedclient	41	patch has errors when building and testing our client artifacts.
+1	javadoc	68	the patch passed
-1	findbugs	25	hadoop-common in the patch failed.
		_ Other Tests _
-1	unit	30	hadoop-common in the patch failed.
+1	unit	267	hadoop-aws in the patch passed.
+1	asflicense	24	The patch does not generate ASF License warnings.
		1630

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 41f82093297d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `36757ad`
Default Java	1.8.0_212
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/branch-mvninstall-root.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/branch-compile-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/branch-mvnsite-hadoop-common-project_hadoop-common.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/branch-findbugs-hadoop-common-project_hadoop-common.txt
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-compile-root.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-compile-root.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/diff-checkstyle-root.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/testReport/
Max. process+thread count	341 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/16/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-06-07T13:12:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	50	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 10 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	62	Maven dependency ordering for branch
+1	mvninstall	1147	trunk passed
+1	compile	1133	trunk passed
+1	checkstyle	142	trunk passed
+1	mvnsite	115	trunk passed
+1	shadedclient	960	branch has no errors when building and testing our client artifacts.
+1	javadoc	87	trunk passed
0	spotbugs	63	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	184	trunk passed
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	80	the patch passed
+1	compile	1070	the patch passed
+1	javac	1070	the patch passed
-0	checkstyle	140	root: The patch generated 4 new + 49 unchanged - 3 fixed = 53 total (was 52)
+1	mvnsite	122	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	xml	1	The patch has no ill-formed XML file.
+1	shadedclient	665	patch has no errors when building and testing our client artifacts.
+1	javadoc	95	the patch passed
+1	findbugs	220	the patch passed
		_ Other Tests _
+1	unit	565	hadoop-common in the patch passed.
+1	unit	286	hadoop-aws in the patch passed.
+1	asflicense	44	The patch does not generate ASF License warnings.
		7187

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-802/17/artifact/out/Dockerfile
GITHUB PR	#802
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle
uname	Linux 85e8f81c0b46 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `a91d24f`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/17/artifact/out/diff-checkstyle-root.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/17/testReport/
Max. process+thread count	1387 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-802/17/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

steveloughran · 2019-06-10T10:30:11Z

...tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardOutOfBandOperations.java

+
+    try {
+      guardedFs.setTtlTimeProvider(mockTimeProvider);
+      when(mockTimeProvider.getMetadataTtl()).thenReturn(ttl);


be aware that using the doAnswer() mockito API may be less "elegant" but it backports to the older mockito (and hence hadoop) versions more easily. I'd recommend going with the older API for now, to make that backport lower-stress

steveloughran · 2019-06-10T10:38:15Z

as far as I can see, patch is done; looks like last bug was squashed.

I'm going to D/L and retest locally, do a final review and then give a vote on it. If I can merge in then it gives me the "opportunity" to reconcile my rename patch with it :)

steveloughran · 2019-06-10T13:07:19Z

Got a fair few failures on the OOB tests on my run of the merged patch. This was with s3guard + dynamo, but not auth

[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.02 s - in org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat
[ERROR] Tests run: 20, Failures: 4, Errors: 4, Skipped: 0, Time elapsed: 183.22 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations
[ERROR] testOutOfBandDeletes[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 8.071 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/OutOfBandDelete-ba80dd3b-0d53-4768-9be2-f7473c0c3afe
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.expectExceptionWhenReadingOpenFileAPI(ITestS3GuardOutOfBandOperations.java:902)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.outOfBandDeletes(ITestS3GuardOutOfBandOperations.java:597)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testOutOfBandDeletes(ITestS3GuardOutOfBandOperations.java:267)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/OutOfBandDelete-ba80dd3b-0d53-4768-9be2-f7473c0c3afe
	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2660)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2549)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2450)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:899)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$19(S3AFileSystem.java:3988)
	at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$20(S3AFileSystem.java:3987)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

[ERROR] testLongerLengthOverwrite[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 24.282 s  <<< FAILURE!
java.lang.AssertionError: 
File length in authoritative table with 
Raw: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/OverwriteFileTest-c68b356a-6d5e-4e3a-90ef-4bd1552d4175; isDirectory=false; length=27; replication=1; blocksize=33554432; modification_time=1560167320000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=fd902fb718bd01bcddabcd77a2e3d547 versionId=KGqiYuMuYDAuAejU6b.cAjgxERJ02nft
Guarded: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/OverwriteFileTest-c68b356a-6d5e-4e3a-90ef-4bd1552d4175; isDirectory=false; length=27; replication=1; blocksize=33554432; modification_time=1560167320000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=fd902fb718bd01bcddabcd77a2e3d547 versionId=KGqiYuMuYDAuAejU6b.cAjgxERJ02nft. Actual: 27
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failEquals(Assert.java:185)
	at org.junit.Assert.assertNotEquals(Assert.java:199)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.verifyFileStatusAsExpected(ITestS3GuardOutOfBandOperations.java:771)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.lambda$overwriteFile$4(ITestS3GuardOutOfBandOperations.java:631)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:985)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:976)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:252)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:304)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:350)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.overwriteFile(ITestS3GuardOutOfBandOperations.java:626)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testLongerLengthOverwrite(ITestS3GuardOutOfBandOperations.java:255)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)

[ERROR] testListingSameLengthOverwrite[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 25.874 s  <<< FAILURE!
java.lang.AssertionError: 
Authoritative is enabled, so metadata is not updated in ms, so mod_time won't match. Expecting different values for raw and guarded filestatus.
Raw: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/dir-c0edf4a8-d1c5-49aa-9ef2-f16a1879b458/file-1-c0edf4a8-d1c5-49aa-9ef2-f16a1879b458; isDirectory=false; length=8; replication=1; blocksize=33554432; modification_time=1560167345000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=9d54f4c0c77def4be7880a85991b04c5 versionId=.XH3.H.Oi1nYjNu3BhQa1k44rS7hqNJ8
Guarded: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/dir-c0edf4a8-d1c5-49aa-9ef2-f16a1879b458/file-1-c0edf4a8-d1c5-49aa-9ef2-f16a1879b458; isDirectory=false; length=8; replication=1; blocksize=33554432; modification_time=1560167345000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=9d54f4c0c77def4be7880a85991b04c5 versionId=.XH3.H.Oi1nYjNu3BhQa1k44rS7hqNJ8. Actual: 1560167345000
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failEquals(Assert.java:185)
	at org.junit.Assert.assertNotEquals(Assert.java:199)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.verifyFileStatusAsExpected(ITestS3GuardOutOfBandOperations.java:790)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.lambda$overwriteFileInListing$5(ITestS3GuardOutOfBandOperations.java:720)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:985)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:976)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:252)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:304)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:350)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.overwriteFileInListing(ITestS3GuardOutOfBandOperations.java:716)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testListingSameLengthOverwrite(ITestS3GuardOutOfBandOperations.java:272)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)

[ERROR] testListingDelete[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 8.198 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/dir-127ee17f-099d-453c-9d4a-48ddf8bef984/file-1-127ee17f-099d-453c-9d4a-48ddf8bef984
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.expectExceptionWhenReadingOpenFileAPI(ITestS3GuardOutOfBandOperations.java:902)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.deleteFileInListing(ITestS3GuardOutOfBandOperations.java:870)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testListingDelete(ITestS3GuardOutOfBandOperations.java:282)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/dir-127ee17f-099d-453c-9d4a-48ddf8bef984/file-1-127ee17f-099d-453c-9d4a-48ddf8bef984
	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2660)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2549)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2450)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:899)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$19(S3AFileSystem.java:3988)
	at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$20(S3AFileSystem.java:3987)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

[ERROR] testSameLengthOverwrite[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 24.555 s  <<< FAILURE!
java.lang.AssertionError: 
Authoritative is enabled, so metadata is not updated in ms, so mod_time won't match. Expecting different values for raw and guarded filestatus.
Raw: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/OverwriteFileTest-07321ede-1fd0-4c86-9899-428ac8f64350; isDirectory=false; length=13; replication=1; blocksize=33554432; modification_time=1560167382000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=1f9bd06f0b9c061cb1079a2bc30ded89 versionId=o2dWJFA7csY_tHtqlwyuZw0Tue9jr6ex
Guarded: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/OverwriteFileTest-07321ede-1fd0-4c86-9899-428ac8f64350; isDirectory=false; length=13; replication=1; blocksize=33554432; modification_time=1560167382000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=1f9bd06f0b9c061cb1079a2bc30ded89 versionId=o2dWJFA7csY_tHtqlwyuZw0Tue9jr6ex. Actual: 1560167382000
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failEquals(Assert.java:185)
	at org.junit.Assert.assertNotEquals(Assert.java:199)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.verifyFileStatusAsExpected(ITestS3GuardOutOfBandOperations.java:790)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.lambda$overwriteFile$4(ITestS3GuardOutOfBandOperations.java:631)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:985)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:976)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:252)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:304)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:350)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.overwriteFile(ITestS3GuardOutOfBandOperations.java:626)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testSameLengthOverwrite(ITestS3GuardOutOfBandOperations.java:248)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)

[ERROR] testListingLongerLengthOverwrite[0](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 25.6 s  <<< FAILURE!
java.lang.AssertionError: 
File length in authoritative table with 
Raw: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/dir-dbe6cf0c-b0e2-4b34-bccb-f4755675215b/file-1-dbe6cf0c-b0e2-4b34-bccb-f4755675215b; isDirectory=false; length=15; replication=1; blocksize=33554432; modification_time=1560167410000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=4e52748950e60c27b6c8fd79c9be5e5d versionId=MbKsfm33iPWNxRkIQIMGCTpkk1R8.bSh
Guarded: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0007/test/dir-dbe6cf0c-b0e2-4b34-bccb-f4755675215b/file-1-dbe6cf0c-b0e2-4b34-bccb-f4755675215b; isDirectory=false; length=15; replication=1; blocksize=33554432; modification_time=1560167410000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=4e52748950e60c27b6c8fd79c9be5e5d versionId=MbKsfm33iPWNxRkIQIMGCTpkk1R8.bSh. Actual: 15
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failEquals(Assert.java:185)
	at org.junit.Assert.assertNotEquals(Assert.java:199)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.verifyFileStatusAsExpected(ITestS3GuardOutOfBandOperations.java:771)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.lambda$overwriteFileInListing$5(ITestS3GuardOutOfBandOperations.java:720)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:985)
	at org.apache.hadoop.test.LambdaTestUtils$VoidCaller.call(LambdaTestUtils.java:976)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:252)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:304)
	at org.apache.hadoop.test.LambdaTestUtils.eventually(LambdaTestUtils.java:350)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.overwriteFileInListing(ITestS3GuardOutOfBandOperations.java:716)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testListingLongerLengthOverwrite(ITestS3GuardOutOfBandOperations.java:277)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)

[ERROR] testOutOfBandDeletes[1](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 7.941 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/OutOfBandDelete-977f5bc7-fb7e-498b-b4ab-0b7359ecc451
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.expectExceptionWhenReadingOpenFileAPI(ITestS3GuardOutOfBandOperations.java:902)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.outOfBandDeletes(ITestS3GuardOutOfBandOperations.java:597)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testOutOfBandDeletes(ITestS3GuardOutOfBandOperations.java:267)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/OutOfBandDelete-977f5bc7-fb7e-498b-b4ab-0b7359ecc451
	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2660)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2549)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2450)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:899)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$19(S3AFileSystem.java:3988)
	at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$20(S3AFileSystem.java:3987)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

[ERROR] testListingDelete[1](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 8.426 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/dir-63c3a32a-7b0b-489a-b629-4454e3534776/file-1-63c3a32a-7b0b-489a-b629-4454e3534776
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.expectExceptionWhenReadingOpenFileAPI(ITestS3GuardOutOfBandOperations.java:902)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.deleteFileInListing(ITestS3GuardOutOfBandOperations.java:870)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testListingDelete(ITestS3GuardOutOfBandOperations.java:282)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-0007/test/dir-63c3a32a-7b0b-489a-b629-4454e3534776/file-1-63c3a32a-7b0b-489a-b629-4454e3534776
	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2660)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2549)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2450)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:899)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$19(S3AFileSystem.java:3988)
	at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$20(S3AFileSystem.java:3987)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

steveloughran · 2019-06-10T15:04:15Z

Some of the getFileStatus-side errors go away when in S3AFileSystem.innerGetFileStatus() I revert back from

    if (hasMetadataStore()) {
      pm = S3Guard.getWithTtl(metadataStore, path, ttlTimeProvider);
    }

to

    if (hasMetadataStore()) {
      pm = metadataStore.get(path, true);
    }

But other things do fail; I think what we'll need is something which reinstates that get-with-empty-directory option with the TTL.

Anyhow, -1 as is, obviously.

I'm going to put up a branch with my other changes, which combine some of the minor style changes and adding etag and version ID checks to the assertions about files not matching. Those assertions actually fail on the listings, because they aren't always returning etags. I think all our store list operations should be returning etags, if they aren't already

steveloughran · 2019-06-10T15:20:03Z

Gabor: pushed up a branch which adds one more commit to this PR

https://github.com/steveloughran/hadoop/tree/s3/HADOOP-16279-oob-delete

Parameterized OOB test now includes auth flag in mehod name, for ease of debugging
Tests for being in sync /not being in sync now compare etag and version IDs. Highlights we aren't always getting back version IDs.
minor: formatting of spaces in some changed files.

The etag/version ID tests should actually be better tests than anything else for consistency, rather than modtime, so I've placed them first. But they are failing on etags as when the etag come out as null. It shouldn't,obviously

BTW, my tests were with your PR rebased onto trunk, if that makes a difference

bgaborg · 2019-06-11T10:47:42Z

@steveloughran, when I run the tests with my branch and without the rebase to trunk my only test failure is in org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitMRJob.

ITestS3GuardOutOfBandOperations fails with your rebased branch. I'll run some tests to check if I'm able to reproduce the issue.

bgaborg · 2019-06-11T13:18:36Z

tested and for the 2nd time I got the same errors after the rebase to trunk. working on fixing it.

steveloughran · 2019-06-12T16:04:29Z

reran the tests on the original (non-rebased) branch, got failures here and in some others (e.g)


[ERROR] testSameLengthOverwrite[auth=true](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)  Time elapsed: 23.781 s  <<< FAILURE!
java.lang.AssertionError: 
etag in authoritative table with 
Raw: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/OverwriteFileTest-6eb7e1fe-d71d-4059-b819-3feb83dcc1dc; isDirectory=false; length=13; replication=1; blocksize=33554432; modification_time=1560354336000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=1f9bd06f0b9c061cb1079a2bc30ded89 versionId=XpuRHcjnzwc1q1QqFFuI.a5jcHbPCo1C
Guarded: S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test/OverwriteFileTest-6eb7e1fe-d71d-4059-b819-3feb83dcc1dc; isDirectory=false; length=13; replication=1; blocksize=33554432; modification_time=1560354336000; access_time=0; owner=stevel; group=stevel; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=1f9bd06f0b9c061cb1079a2bc30ded89 versionId=XpuRHcjnzwc1q1QqFFuI.a5jcHbPCo1C. Actual: 1f9bd06f0b9c061cb1079a2bc30ded89
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.verifyFileStatusAsExpected(ITestS3GuardOutOfBandOperations.java:789)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.lambda$overwriteFile$4(ITestS3GuardOutOfBandOperations.java:631)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.overwriteFile(ITestS3GuardOutOfBandOperations.java:626)
	at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testSameLengthOverwrite(ITestS3GuardOutOfBandOperations.java:248)

makes me thing the DDB table is in an odd state.

Plan:

rebase to trunk
rebuild
s3guard to create an entirely new table
retest

steveloughran · 2019-06-12T17:57:11Z

Update, believe I've identified the problem, it's a subtle but fun one

with (my suggested) change to accepting configurable times (e.g. 15m) for a TTL, the time in seconds was being passed down to a ttl class expecting millis. As a result, dir listings in the tests were going nonauth ~immediately, so the guardedFS in tests was immediately resyncing with the store. Hence: none of the expected mismatches

Fixing that (one liner) with some extended OOB test looking at the internals (written during debugging) and with HADOOP-16368 S3A list operation doesn't pick up etags from results, as I've extended the match testing to include etags rather than just timestamps. much less brittle. I'm retaining the modtime test too, for regression rather than as the sole comparison

steveloughran · 2019-06-12T18:48:08Z

#952 is my patch extending this

bgaborg · 2019-06-13T11:31:15Z

Thanks for finding out what was the issue and fixing it!

vjagadish1989 nickpan47 Please take a look. Author: Prateek Maheshwari <pmaheshwari@apache.org> Reviewers: Jagadish<jagadish@apache.org> Closes apache#802 from prateekm/api-docs

ajfabbri suggested changes May 8, 2019

View reviewed changes

bgaborg force-pushed the HADOOP-16279 branch from 689c3cf to aadd27d Compare May 22, 2019 18:06

bgaborg force-pushed the HADOOP-16279 branch from aadd27d to 9e89426 Compare May 27, 2019 17:19

bgaborg force-pushed the HADOOP-16279 branch 2 times, most recently from 1d30291 to b61026c Compare May 29, 2019 08:08

bgaborg force-pushed the HADOOP-16279 branch from b61026c to 14c41f2 Compare May 29, 2019 15:59

bgaborg force-pushed the HADOOP-16279 branch 2 times, most recently from c76296a to 3e9ca9d Compare May 30, 2019 12:36

steveloughran self-requested a review May 31, 2019 18:21