HADOOP-17835. Use CuratorCache implementation instead of PathChildrenCache / TreeCache #3266
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…Cache / TreeCache
|
🎊 +1 overall
This message was automatically generated. |
|
@aajisaka @eolivelli Could you please take a look? |
aajisaka
reviewed
Aug 5, 2021
...rc/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java
Show resolved
Hide resolved
Randgalt
reviewed
Aug 5, 2021
| @@ -363,105 +357,122 @@ public void startThreads() throws IOException { | |||
| throw new RuntimeException("Could not create ZK paths"); | |||
| } | |||
| try { | |||
| keyCache = new PathChildrenCache(zkClient, ZK_DTSM_MASTER_KEY_ROOT, true); | |||
| keyCache = CuratorCache.bridgeBuilder(zkClient, ZK_DTSM_MASTER_KEY_ROOT) | |||
| .build(); | |||
| if (keyCache != null) { | |||
There was a problem hiding this comment.
Yeah this was not applicable even before with PathChildrenCache.
Randgalt
reviewed
Aug 5, 2021
| } | ||
| if (isTokenWatcherEnabled) { | ||
| LOG.info("TokenCache is enabled"); | ||
| try { | ||
| tokenCache = new PathChildrenCache(zkClient, ZK_DTSM_TOKENS_ROOT, true); | ||
| tokenCache = CuratorCache.bridgeBuilder(zkClient, ZK_DTSM_TOKENS_ROOT) | ||
| .build(); | ||
| if (tokenCache != null) { |
Randgalt
reviewed
Aug 5, 2021
| } | ||
| }, listenerThreadPool); | ||
| CuratorCacheListener keyCacheListener = CuratorCacheListener.builder() | ||
| .forCreates(childData -> { |
There was a problem hiding this comment.
Note: there is a forCreatesAndChanges()
Randgalt
reviewed
Aug 5, 2021
| } | ||
| }, listenerThreadPool); | ||
| CuratorCacheListener tokenCacheListener = CuratorCacheListener.builder() | ||
| .forCreates(childData -> { |
There was a problem hiding this comment.
Note: there is a forCreatesAndChanges()
Randgalt
reviewed
Aug 5, 2021
| public ListenerHandle registerPathListener(final PathListener listener) { | ||
|
|
||
| CuratorCacheListener cacheListener = CuratorCacheListener.builder() | ||
| .forCreates(childData -> { |
There was a problem hiding this comment.
Note: there is a forCreatesAndChanges()
|
@Randgalt Thanks for the review. Addressed all comments. |
|
🎊 +1 overall
This message was automatically generated. |
|
Thanks for the reviews. @aajisaka we have @eolivelli's +1 for this PR after latest revision. |
aajisaka
approved these changes
Aug 7, 2021
|
Thank you @virajjasani for your contribution and thanks @eolivelli and @Randgalt for your review. |
kiran-maturi
pushed a commit
to kiran-maturi/hadoop
that referenced
this pull request
Nov 24, 2021
…Cache / TreeCache (apache#3266) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
melissayou
pushed a commit
to melissayou/hadoop
that referenced
this pull request
Oct 23, 2022
…Cache / TreeCache (apache#3266) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
4 tasks
melissayou
pushed a commit
to melissayou/hadoop
that referenced
this pull request
Nov 5, 2022
…Cache / TreeCache (apache#3266) Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
virajith
pushed a commit
that referenced
this pull request
Nov 7, 2022
* HDFS-15383. RBF: Add support for router delegation token without watch (#2047) Improving router's performance for delegation tokens related operations. It achieves the goal by removing watchers from router on tokens since based on our experience. The huge number of watches inside Zookeeper is degrading Zookeeper's performance pretty hard. The current limit is about 1.2-1.5 million. * HADOOP-17835. Use CuratorCache implementation instead of PathChildrenCache / TreeCache (#3266) Signed-off-by: Akira Ajisaka <aajisaka@apache.org> Co-authored-by: lfengnan <lfengnan@uber.com> Co-authored-by: Viraj Jasani <vjasani@apache.org> Co-authored-by: Melissa You <myou@myou-mn1.linkedin.biz>
NyteKnight
pushed a commit
to NyteKnight/hadoop
that referenced
this pull request
Jun 25, 2024
…che#5112) (apache#26) * HDFS-15383. RBF: Add support for router delegation token without watch (apache#2047) Improving router's performance for delegation tokens related operations. It achieves the goal by removing watchers from router on tokens since based on our experience. The huge number of watches inside Zookeeper is degrading Zookeeper's performance pretty hard. The current limit is about 1.2-1.5 million. * HADOOP-17835. Use CuratorCache implementation instead of PathChildrenCache / TreeCache (apache#3266) Signed-off-by: Akira Ajisaka <aajisaka@apache.org> Co-authored-by: lfengnan <lfengnan@uber.com> Co-authored-by: Viraj Jasani <vjasani@apache.org> Co-authored-by: Melissa You <myou@linkedin.com> ACLOVERRIDE
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.