Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDFS-16265. Refactor HDFS tool tests for better reuse #3536

Merged
merged 1 commit into from
Oct 9, 2021
Merged

HDFS-16265. Refactor HDFS tool tests for better reuse #3536

merged 1 commit into from
Oct 9, 2021

Conversation

GauthamBanasandra
Copy link
Member

Description of PR

Currently, the test cases written in hdfs-tool-test.h isn't easy to reuse. Primarily because the expectations are different for each HDFS tool. I realized this while I was creating the PR for HDFS-16260. For instance, passing more than one argument is erroneous to hdfs_allowSnapshot while it's the only valid scenario for hdfs_deleteSnapshot.

Thus, it won't be possible to reuse the test cases without decoupling the expectations from the test case definitions. The solution here is to move the expectations to the corresponding mock classes and invoke the call to set them up in the test cases after the creation of mock instances.

How was this patch tested?

Unit tests ran successfully -

image

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@GauthamBanasandra
HDFS-16265. Refactor HDFS tool tests for better reuse
38a5b4c 
* Decoupled the expectations
  from test case definitions to
  enable better reuse.
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 43m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 11 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 34m 49s trunk passed
+1 💚 compile 2m 48s trunk passed
+1 💚 mvnsite 0m 23s trunk passed
+1 💚 shadedclient 58m 54s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 14s the patch passed
+1 💚 compile 2m 40s the patch passed
+1 💚 cc 2m 40s the patch passed
+1 💚 golang 2m 40s the patch passed
+1 💚 javac 2m 40s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 20s the patch passed
+1 💚 shadedclient 20m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 115m 0s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 30s The patch does not generate ASF License warnings.
243m 57s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/artifact/out/Dockerfile
GITHUB PR #3536
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell golang
uname Linux ee58d116ab1f 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 38a5b4c
Default Java Red Hat, Inc.-1.8.0_302-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/testReport/
Max. process+thread count 556 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 25m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 11 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 24m 12s trunk passed
+1 💚 compile 2m 59s trunk passed
+1 💚 mvnsite 0m 35s trunk passed
+1 💚 shadedclient 49m 1s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 19s the patch passed
+1 💚 compile 2m 43s the patch passed
+1 💚 cc 2m 43s the patch passed
+1 💚 golang 2m 43s the patch passed
+1 💚 javac 2m 43s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 22s the patch passed
+1 💚 shadedclient 20m 54s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 103m 41s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
205m 21s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/artifact/out/Dockerfile
GITHUB PR #3536
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell golang
uname Linux 1ab012aa60b6 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 38a5b4c
Default Java Red Hat, Inc.-1.8.0_302-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/testReport/
Max. process+thread count 606 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/console
versions git=2.27.0 maven=3.6.3
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 15m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 11 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 31m 14s trunk passed
+1 💚 compile 2m 43s trunk passed
+1 💚 mvnsite 0m 20s trunk passed
+1 💚 shadedclient 63m 30s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 15s the patch passed
+1 💚 compile 2m 34s the patch passed
+1 💚 cc 2m 34s the patch passed
+1 💚 golang 2m 34s the patch passed
+1 💚 javac 2m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 15s the patch passed
+1 💚 shadedclient 33m 6s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 91m 29s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 30s The patch does not generate ASF License warnings.
209m 45s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/artifact/out/Dockerfile
GITHUB PR #3536
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell golang
uname Linux bb6d5b4a8d8a 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 38a5b4c
Default Java Debian-11.0.12+7-post-Debian-2deb10u1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/testReport/
Max. process+thread count 618 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/console
versions git=2.20.1 maven=3.6.0
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 11 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 22m 57s trunk passed
+1 💚 compile 3m 5s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 3m 9s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 mvnsite 0m 23s trunk passed
+1 💚 shadedclient 50m 47s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 14s the patch passed
+1 💚 compile 2m 54s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 cc 2m 54s the patch passed
+1 💚 golang 2m 54s the patch passed
+1 💚 javac 2m 54s the patch passed
+1 💚 compile 2m 57s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 cc 2m 57s the patch passed
+1 💚 golang 2m 57s the patch passed
+1 💚 javac 2m 57s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 17s the patch passed
+1 💚 shadedclient 21m 15s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 102m 57s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 29s The patch does not generate ASF License warnings.
185m 11s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/artifact/out/Dockerfile
GITHUB PR #3536
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell golang
uname Linux 4d6111918dd1 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 38a5b4c
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/testReport/
Max. process+thread count 593 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3536/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@goiri goiri merged commit 4f3dfb7 into apache:trunk Oct 9, 2021
@GauthamBanasandra GauthamBanasandra deleted the refactor-tests branch October 16, 2021 16:46
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@GauthamBanasandra
HDFS-16265. Refactor HDFS tool tests for better reuse (apache#3536)
4cf16c8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@goiri goiri goiri approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@GauthamBanasandra @hadoop-yetus @goiri