Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-1973. Implement OM RenewDelegationToken request to use Cache and DoubleBuffer. #1316

Merged
merged 2 commits into from
Aug 21, 2019

Conversation

bharatviswa504
Copy link
Contributor

No description provided.

@bharatviswa504 bharatviswa504 self-assigned this Aug 20, 2019
@bharatviswa504 bharatviswa504 changed the title HDDS-1975. Implement OM RenewDelegationToken request to use Cache and DoubleBuffer. HDDS-1973. Implement OM RenewDelegationToken request to use Cache and DoubleBuffer. Aug 20, 2019
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 45 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 36 Maven dependency ordering for branch
+1 mvninstall 752 trunk passed
+1 compile 396 trunk passed
+1 checkstyle 66 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 880 branch has no errors when building and testing our client artifacts.
+1 javadoc 167 trunk passed
0 spotbugs 418 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 619 trunk passed
_ Patch Compile Tests _
0 mvndep 18 Maven dependency ordering for patch
+1 mvninstall 541 the patch passed
+1 compile 360 the patch passed
+1 cc 360 the patch passed
+1 javac 360 the patch passed
+1 checkstyle 69 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 669 patch has no errors when building and testing our client artifacts.
+1 javadoc 154 the patch passed
+1 findbugs 634 the patch passed
_ Other Tests _
+1 unit 313 hadoop-hdds in the patch passed.
-1 unit 1836 hadoop-ozone in the patch failed.
+1 asflicense 39 The patch does not generate ASF License warnings.
7760
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/1/artifact/out/Dockerfile
GITHUB PR #1316
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname Linux ae6cbda13a4f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 4f925af
Default Java 1.8.0_222
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/1/testReport/
Max. process+thread count 4820 (vs. ulimit of 5500)
modules C: hadoop-ozone/common hadoop-ozone/ozone-manager U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 46 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 30 Maven dependency ordering for branch
+1 mvninstall 604 trunk passed
+1 compile 345 trunk passed
+1 checkstyle 60 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 790 branch has no errors when building and testing our client artifacts.
+1 javadoc 146 trunk passed
0 spotbugs 435 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 624 trunk passed
_ Patch Compile Tests _
0 mvndep 18 Maven dependency ordering for patch
+1 mvninstall 548 the patch passed
+1 compile 345 the patch passed
+1 cc 345 the patch passed
+1 javac 345 the patch passed
+1 checkstyle 62 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 628 patch has no errors when building and testing our client artifacts.
+1 javadoc 163 the patch passed
+1 findbugs 662 the patch passed
_ Other Tests _
-1 unit 222 hadoop-hdds in the patch failed.
-1 unit 2128 hadoop-ozone in the patch failed.
+1 asflicense 39 The patch does not generate ASF License warnings.
7626
Reason Tests
Failed junit tests hadoop.ozone.container.ozoneimpl.TestOzoneContainer
hadoop.ozone.TestMiniChaosOzoneCluster
hadoop.ozone.container.server.TestSecureContainerServer
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/2/artifact/out/Dockerfile
GITHUB PR #1316
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname Linux 6ba1a9cb3234 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 6244502
Default Java 1.8.0_222
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/2/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/2/testReport/
Max. process+thread count 4478 (vs. ulimit of 5500)
modules C: hadoop-ozone/common hadoop-ozone/ozone-manager U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

UpdateRenewDelegationTokenRequest.newBuilder()
.setRenewDelegationTokenRequest(renewDelegationTokenRequest)
.setRenewDelegationTokenResponse(renewResponse))
.setCmdType(getOmRequest().getCmdType())
Copy link
Contributor

@xiaoyuyao xiaoyuyao Aug 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use the cmd type of UpdateRenewDelegationTokenRequest

Copy link
Contributor Author

@bharatviswa504 bharatviswa504 Aug 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being done intentionally so that we don't need new class or some special handling in OMRatisUtils createOMClientRequest().

Same is explained in the code comments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An UpdateRenewDelegationTokenRequest body with RenewDelegationTokenRequest type in OMRequest is a bit hacky. But I'm OK with it if otherwise takes too much changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya can understand that. Otherwise, we need some changes in createClientRequest in ratis to act upon UpdateRenewDelegationTokenRequest or RenewDelegationTokenRequest to return OMGetDelegationTokenRequest object or have separate new classes for both of them. And for one only preExecute will be needed, for other validateAndUpdateCache only will be needed. So, I will leave this as it is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this is a bit hard to follow. I also had a long discussion with Bharat while reviewing the createDelegationToken patch. I am okay to live with for now to avoid creating yet another request class.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 88 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 12 Maven dependency ordering for branch
-1 mvninstall 133 hadoop-ozone in trunk failed.
-1 compile 49 hadoop-ozone in trunk failed.
+1 checkstyle 54 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 963 branch has no errors when building and testing our client artifacts.
+1 javadoc 198 trunk passed
0 spotbugs 245 Used deprecated FindBugs config; considering switching to SpotBugs.
-1 findbugs 116 hadoop-ozone in trunk failed.
_ Patch Compile Tests _
0 mvndep 20 Maven dependency ordering for patch
-1 mvninstall 176 hadoop-ozone in the patch failed.
-1 compile 67 hadoop-ozone in the patch failed.
-1 cc 67 hadoop-ozone in the patch failed.
-1 javac 67 hadoop-ozone in the patch failed.
+1 checkstyle 80 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 907 patch has no errors when building and testing our client artifacts.
+1 javadoc 176 the patch passed
-1 findbugs 123 hadoop-ozone in the patch failed.
_ Other Tests _
+1 unit 366 hadoop-hdds in the patch passed.
-1 unit 124 hadoop-ozone in the patch failed.
+1 asflicense 31 The patch does not generate ASF License warnings.
4745
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/Dockerfile
GITHUB PR #1316
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname Linux da79ce0437dc 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 269b543
Default Java 1.8.0_212
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/branch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/branch-compile-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/branch-findbugs-hadoop-ozone.txt
mvninstall https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-mvninstall-hadoop-ozone.txt
compile https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-compile-hadoop-ozone.txt
cc https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-compile-hadoop-ozone.txt
javac https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-compile-hadoop-ozone.txt
findbugs https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-findbugs-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/testReport/
Max. process+thread count 451 (vs. ulimit of 5500)
modules C: hadoop-ozone/common hadoop-ozone/ozone-manager U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1316/3/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@arp7 arp7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 lgtm.

Okay to commit assuming @xiaoyuyao is also +1.

@bharatviswa504
Copy link
Contributor Author

@xiaoyuyao Can I proceed with commit?

@xiaoyuyao
Copy link
Contributor

+1 from me as well. Seems the CI is broken pending fix from HDDS-1999/2000.

@bharatviswa504
Copy link
Contributor Author

Test failures are not related to this patch.
Thank You @arp7 and @xiaoyuyao for the review.
I will commit this to the trunk.

@bharatviswa504 bharatviswa504 merged commit 217e748 into apache:trunk Aug 21, 2019
amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants