Skip to content

HADOOP-17459. ADLS Gen1: Fixes for rename contract tests #2607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 12, 2021

Conversation

bilaharith
Copy link
Contributor

@bilaharith bilaharith commented Jan 8, 2021

Fix the following test case failures which are failing after the contract test update in hadoop-common

[ERROR] Failures:
[ERROR] TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected with exception, but got false
[ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting org.apache.hadoop.security.AccessControlException with text Parent path is not a folder. but got : "void"

Post updates to rename on existing file test, ADL contract test is having failure. Updates were made in the AbstractContractTest class in https://issues.apache.org/jira/browse/HADOOP-17365.

To align to test expectation of no exception but a false return from rename, ADL tests need config "fs.contract.rename-returns-false-if-dest-exists" set to true.

[Test results pasted at the end of conversation tab].

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 27s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 0m 0s test4tests The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
-1 ❌ mvninstall 30m 1s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 compile 0m 25s trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 compile 0m 22s trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 checkstyle 0m 19s trunk passed
+1 💚 mvnsite 0m 25s trunk passed
-1 ❌ shadedclient 17m 44s branch has errors when building and testing our client artifacts.
+1 💚 javadoc 0m 22s trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javadoc 0m 21s trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗 spotbugs 0m 37s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 0m 34s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 20s the patch passed
+1 💚 compile 0m 18s the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javac 0m 18s the patch passed
+1 💚 compile 0m 17s the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 javac 0m 17s the patch passed
+1 💚 checkstyle 0m 11s the patch passed
+1 💚 mvnsite 0m 19s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+1 💚 shadedclient 16m 49s patch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 19s the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javadoc 0m 18s the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 findbugs 0m 37s the patch passed
_ Other Tests _
+1 💚 unit 0m 49s hadoop-azure-datalake in the patch passed.
+1 💚 asflicense 0m 29s The patch does not generate ASF License warnings.
75m 9s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2607/1/artifact/out/Dockerfile
GITHUB PR #2607
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname Linux 810affa54ce6 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 87bd4d2
Default Java Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2607/1/testReport/
Max. process+thread count 628 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-azure-datalake U: hadoop-tools/hadoop-azure-datalake
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2607/1/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@bilaharith
Copy link
Contributor Author

Before the changes

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 expected rename(adl://bithgen1it.azuredatalakestore.net/test/source-256.txt, adl://bithgen1it.azuredatalakestore.net/test/dest-512.txt) to be rejected with exception, but got false
[ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting org.apache.hadoop.security.AccessControlException with text Parent path is not a folder. but got : "void"
[ERROR] Errors:
[ERROR] TestAdlContractGetFileStatusLive>AbstractContractGetFileStatusTest.testComplexDirActions:153->AbstractContractGetFileStatusTest.checkListStatusIteratorComplexDir:196 » NoClassDefFound
[ERROR] TestAdlContractGetFileStatusLive>AbstractContractGetFileStatusTest.testListStatusIteratorFile:363->AbstractContractGetFileStatusTest.validateListingForFile:384 » NoClassDefFound
[ERROR] TestAdlContractRootDirLive>AbstractContractRootDirectoryTest.testSimpleRootListing:251 » NoClassDefFound
[INFO]
[ERROR] Tests run: 893, Failures: 2, Errors: 3, Skipped: 3
[INFO]
[ERROR] There are test failures.

After the change

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestAdlContractGetFileStatusLive>AbstractContractGetFileStatusTest.testComplexDirActions:153->AbstractContractGetFileStatusTest.checkListStatusIteratorComplexDir:196 » NoClassDefFound
[ERROR] TestAdlContractGetFileStatusLive>AbstractContractGetFileStatusTest.testListStatusIteratorFile:363->AbstractContractGetFileStatusTest.validateListingForFile:384 » NoClassDefFound
[ERROR] TestAdlContractRootDirLive>AbstractContractRootDirectoryTest.testSimpleRootListing:251 » NoClassDefFound
[INFO]
[ERROR] Tests run: 893, Failures: 0, Errors: 3, Skipped: 3
[INFO]
[ERROR] There are test failures.

@bilaharith bilaharith changed the title DRAFT PR HADOOP-17459. ADLS Gen1: Fixes for rename contract tests HADOOP-17459. ADLS Gen1: Fixes for rename contract tests Jan 9, 2021
@bilaharith bilaharith marked this pull request as ready for review January 9, 2021 10:32
@steveloughran
Copy link
Contributor

what's causing NoClassDefFound?

@steveloughran steveloughran merged commit 6123306 into apache:trunk Jan 12, 2021
asfgit pushed a commit that referenced this pull request Jan 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants