Downgrade hashing to SHA-1 and other optimizations #5791
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JaroslavTulach
approved these changes
Mar 2, 2023
There was a problem hiding this comment.
I believe there should be a single "digest" per each library written to a file in library root. When it exists, all the caches associated with the library shall include that "digest" or otherwise be considered invalid. Everyone who modifies the library shall either remove or recompute the "digest" in the library file.
| @@ -2,7 +2,7 @@ | |||
|
|
|||
| import com.oracle.truffle.api.TruffleFile; | |||
| import com.oracle.truffle.api.TruffleLogger; | |||
| import org.bouncycastle.jcajce.provider.digest.SHA3; | |||
| import org.bouncycastle.jcajce.provider.digest.SHA1; | |||
There was a problem hiding this comment.
Why do we use bouncycastle and not the standard JVM implementation of the digest?
There was a problem hiding this comment.
Looks like the only other place where we use SHA3 is for Buffers. Those could also be downgraded, I believe.
4 tasks
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for. Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity. Don't initiallize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly. Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive. Finally fix a warning when generating library cache.
hubertp
force-pushed
the
wip/hubert/5763-hashing-downgrade
branch
from
492bdf0
to
435b11c
Compare
4e6
approved these changes
Mar 8, 2023
This was referenced Mar 10, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.
Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.
Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.
Avoid unnecessary conversions between Scala and Java. Those back-and-forth
asScalaandasJavaare pretty expensive.Finally fix an SBT warning when generating library cache.
Closes #5763
Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
Checklist
Please include the following checklist in your PR:
Scala,
Java,
and
Rust
style guides.
./run ide buildand./run ide watch.