[Merged by Bors] - Optimise update_validators by decrypting key cache only when necessary
#4126
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
michaelsproul
added
val-client
|
Btw the |
michaelsproul
added
waiting-on-author
screwyprof
force-pushed
the
fix/cache-final
branch
from
2112d87 to
be283c3
Compare
screwyprof
force-pushed
the
fix/cache-final
branch
from
be283c3 to
f61ce73
Compare
|
Hey @michaelsproul, sorry for the trouble, I just noticed that after the changes you suggested I forgot to make |
|
My bad! That was my erroneous suggestion |
michaelsproul
added
ready-for-review
|
I think my validator started to miss attestation because of this. Would be great to have it in the next release 🙏 |
michaelsproul
approved these changes
Mar 28, 2023
michaelsproul
added
ready-for-merge
|
bors r+ |
bors bot
pushed a commit
that referenced
this pull request
Mar 29, 2023
…ary (#4126) ## Title Optimise `update_validators` by decrypting key cache only when necessary ## Issue Addressed Resolves [#3968: Slow performance of validator client PATCH API with hundreds of keys](#3968) ## Proposed Changes 1. Add a check to determine if there is at least one local definition before decrypting the key cache. 2. Assign an empty `KeyCache` when all definitions are of the `Web3Signer` type. 3. Perform cache-related operations (e.g., saving the modified key cache) only if there are local definitions. ## Additional Info This PR addresses the excessive CPU usage and slow performance experienced when using the `PATCH lighthouse/validators/{pubkey}` request with a large number of keys. The issue was caused by the key cache using cryptography to decipher and cipher the cache entities every time the request was made. This operation called `scrypt`, which was very slow and required a lot of memory when there were many concurrent requests. These changes have no impact on the overall functionality but can lead to significant performance improvements when working with remote signers. Importantly, the key cache is never used when there are only `Web3Signer` definitions, avoiding the expensive operation of decrypting the key cache in such cases. Co-authored-by: Maksim Shcherbo <max.shcherbo@consensys.net>
|
Pull request successfully merged into unstable. Build succeeded:
|
bors
bot
changed the title
update_validators by decrypting key cache only when necessaryupdate_validators by decrypting key cache only when necessary
ghost
pushed a commit
to oone-world/lighthouse
that referenced
this pull request
Jul 13, 2023
…ary (sigp#4126) ## Title Optimise `update_validators` by decrypting key cache only when necessary ## Issue Addressed Resolves [sigp#3968: Slow performance of validator client PATCH API with hundreds of keys](sigp#3968) ## Proposed Changes 1. Add a check to determine if there is at least one local definition before decrypting the key cache. 2. Assign an empty `KeyCache` when all definitions are of the `Web3Signer` type. 3. Perform cache-related operations (e.g., saving the modified key cache) only if there are local definitions. ## Additional Info This PR addresses the excessive CPU usage and slow performance experienced when using the `PATCH lighthouse/validators/{pubkey}` request with a large number of keys. The issue was caused by the key cache using cryptography to decipher and cipher the cache entities every time the request was made. This operation called `scrypt`, which was very slow and required a lot of memory when there were many concurrent requests. These changes have no impact on the overall functionality but can lead to significant performance improvements when working with remote signers. Importantly, the key cache is never used when there are only `Web3Signer` definitions, avoiding the expensive operation of decrypting the key cache in such cases. Co-authored-by: Maksim Shcherbo <max.shcherbo@consensys.net>
Woodpile37
pushed a commit
to Woodpile37/lighthouse
that referenced
this pull request
Jan 6, 2024
…ary (sigp#4126) ## Title Optimise `update_validators` by decrypting key cache only when necessary ## Issue Addressed Resolves [sigp#3968: Slow performance of validator client PATCH API with hundreds of keys](sigp#3968) ## Proposed Changes 1. Add a check to determine if there is at least one local definition before decrypting the key cache. 2. Assign an empty `KeyCache` when all definitions are of the `Web3Signer` type. 3. Perform cache-related operations (e.g., saving the modified key cache) only if there are local definitions. ## Additional Info This PR addresses the excessive CPU usage and slow performance experienced when using the `PATCH lighthouse/validators/{pubkey}` request with a large number of keys. The issue was caused by the key cache using cryptography to decipher and cipher the cache entities every time the request was made. This operation called `scrypt`, which was very slow and required a lot of memory when there were many concurrent requests. These changes have no impact on the overall functionality but can lead to significant performance improvements when working with remote signers. Importantly, the key cache is never used when there are only `Web3Signer` definitions, avoiding the expensive operation of decrypting the key cache in such cases. Co-authored-by: Maksim Shcherbo <max.shcherbo@consensys.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Optimise
update_validatorsby decrypting key cache only when necessaryIssue Addressed
Resolves #3968: Slow performance of validator client PATCH API with hundreds of keys
Proposed Changes
KeyCachewhen all definitions are of theWeb3Signertype.Additional Info
This PR addresses the excessive CPU usage and slow performance experienced when using the
PATCH lighthouse/validators/{pubkey}request with a large number of keys. The issue was caused by the key cache using cryptography to decipher and cipher the cache entities every time the request was made. This operation calledscrypt, which was very slow and required a lot of memory when there were many concurrent requests.These changes have no impact on the overall functionality but can lead to significant performance improvements when working with remote signers. Importantly, the key cache is never used when there are only
Web3Signerdefinitions, avoiding the expensive operation of decrypting the key cache in such cases.