Skip to content

HADOOP-17123. remove guava Preconditions from Hadoop-common module #2134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

amahussein
Copy link
Contributor

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 25m 53s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 23m 34s trunk passed
+1 💚 compile 22m 51s trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
-1 ❌ compile 11m 9s root in trunk failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09.
+1 💚 checkstyle 1m 16s trunk passed
+1 💚 mvnsite 1m 46s trunk passed
+1 💚 shadedclient 19m 8s branch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 42s hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚 javadoc 1m 0s trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗 spotbugs 2m 46s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 43s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 9s the patch passed
+1 💚 compile 21m 2s the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
-1 ❌ javac 21m 2s root-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 generated 4 new + 2043 unchanged - 0 fixed = 2047 total (was 2043)
+1 💚 compile 16m 54s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
-1 ❌ javac 16m 54s root-jdkPrivateBuild-1.8.0_252-8u252-b09-118.04-b09 with JDK Private Build-1.8.0_252-8u252-b09-118.04-b09 generated 533 new + 1405 unchanged - 0 fixed = 1938 total (was 1405)
+1 💚 checkstyle 1m 18s hadoop-common-project/hadoop-common: The patch generated 0 new + 1334 unchanged - 7 fixed = 1334 total (was 1341)
+1 💚 mvnsite 1m 27s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 13m 58s patch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 45s hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚 javadoc 1m 0s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 findbugs 2m 21s the patch passed
_ Other Tests _
+1 💚 unit 9m 36s hadoop-common in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
182m 22s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/Dockerfile
GITHUB PR #2134
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 437fd9c25d03 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 0e694b2
Default Java Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/branch-compile-root-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09.txt
javadoc https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/testReport/
Max. process+thread count 3010 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-2134/1/console
versions git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be a big piece of work. not just the coding, but the merging and backporting.

  • we are going to have to backport the noguava package back many versions of releases, so that then cherrypicking is easy. That doesn't need any other changes to go...it should be a separate patch
  • I'm not sure about "noguava" as a name. "unguava"?
  • package must be scoped private/unstable
  • we need separate checks for state and arg, as different exceptions are raised.

Also, if we are going to do this, can we have one overload which takes a Supplier<String> messageSupplier as Objects.requireNonNull() does. Gives us room to play in future

Finally, big thing to consider: we need tests for the methods actually doing what we expect. That's raising exceptions and forming the correct strings.

super();
}

public static void checkIsTrue(final boolean expression, final String message,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its explicitly an argument here, hence the IllegalArgumentException.

We will also need checkState, which throw ome illegal state exception too.

We need both and so the check method names should be distinct

Copy link
Contributor Author

@amahussein amahussein Jul 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be a big piece of work. not just the coding, but the merging and backporting.

  • we are going to have to backport the noguava package back many versions of releases, so that then cherrypicking is easy. That doesn't need any other changes to go...it should be a separate patch
  • I'm not sure about "noguava" as a name. "unguava"?
  • package must be scoped private/unstable
  • we need separate checks for state and arg, as different exceptions are raised.

Also, if we are going to do this, can we have one overload which takes a Supplier<String> messageSupplier as Objects.requireNonNull() does. Gives us room to play in future

Finally, big thing to consider: we need tests for the methods actually doing what we expect. That's raising exceptions and forming the correct strings.

Thanks @steveloughran for the feedback. I created a new PR #2143 that has unguava.Validate#checkNotNull implementation.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 2s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 20m 13s trunk passed
+1 💚 compile 20m 15s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 compile 17m 15s trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 checkstyle 1m 16s trunk passed
+1 💚 mvnsite 1m 28s trunk passed
+1 💚 shadedclient 17m 46s branch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 40s hadoop-common in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.
+1 💚 javadoc 0m 59s trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗 spotbugs 2m 20s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 18s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 55s the patch passed
+1 💚 compile 20m 14s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
-1 ❌ javac 20m 14s root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 4 new + 2054 unchanged - 0 fixed = 2058 total (was 2054)
+1 💚 compile 19m 24s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 javac 19m 24s the patch passed
+1 💚 checkstyle 1m 13s hadoop-common-project/hadoop-common: The patch generated 0 new + 1339 unchanged - 7 fixed = 1339 total (was 1346)
+1 💚 mvnsite 1m 28s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 14m 21s patch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 46s hadoop-common in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.
+1 💚 javadoc 1m 0s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 findbugs 2m 20s the patch passed
_ Other Tests _
+1 💚 unit 9m 26s hadoop-common in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
156m 45s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/artifact/out/Dockerfile
GITHUB PR #2134
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux fb782608143b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 05b3337
Default Java Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
javadoc https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt
javac https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt
javadoc https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/testReport/
Max. process+thread count 1606 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/3/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 5s #2134 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
Subsystem Report/Notes
GITHUB PR #2134
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2134/1/console
versions git=2.17.1
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@steveloughran
Copy link
Contributor

revisiting this...we should get this in so it can be backported and we can keep guava out of future patches.

is the name 'noguava' the right one?

  1. should we try and be clever and use a name a bit like guava but different (Remember: guava is derived from java)
  2. or just stick in some package under util and declare private, e.g o.a.hadoop.utils.internal. We don't want anything external using these

@steveloughran
Copy link
Contributor

(oh, and for merging, I'd propose two commits: one to create the new Predicate class, one to use). Why so? Makes it straightforward to cherrypick the predicate class across all old versions so that other cherrypicked code can use it, without worrying about the big merge)

@amahussein
Copy link
Contributor Author

Thanks @steveloughran
Some time ago, I created HADOOP-17126 "implement non-guava Precondition checkNotNull" to address your feedback suggesting to break the PR into two separate phases.
Back then, I got some initial feedback but then the PR got lost and stayed pending since then.
I will close this PR.

@amahussein amahussein closed this Sep 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants