feat: memoize compiler verification per worker #196
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## development #196 +/- ##
===============================================
+ Coverage 66.43% 67.00% +0.56%
===============================================
Files 163 163
Lines 3802 3825 +23
Branches 659 664 +5
===============================================
+ Hits 2526 2563 +37
+ Misses 1057 1043 -14
Partials 219 219
Continue to review full report at Codecov.
|
The solc compile task is made up of two subtasks: build, which checks the compiler, and run solc. These have both been pulled up to worker level, in prep for caching build.
Move the build subtask to the preprocessing steps (rather than the solc compile phase).
The build check step is expensive and only needs to happen once per solc version. The worker now tracks which compilers have been downloaded and checked in `compilerMetadataCache` and returns the memoized result if available based on solc version.
kanej
force-pushed
the
feat/split-solc-build-compile
branch
from
405c1fb
to
ef59b41
Compare
Merged
kanej
added a commit
that referenced
this pull request
Jun 8, 2022
The solc compile task is made up of two subtasks: build, which checks the compiler, and run solc. These have both been pulled up from hardhat to the worker level, the build subtask has been moved to the preprocessing steps (rather than the solc compile phase). The build check step is expensive and only needs to happen once per solc version. The worker now tracks which compilers have been downloaded and checked in `compilerMetadataCache` and returns the memoized result if available based on solc version.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR speeds up the solc compile step of validation.
Previously, after preprocessing to build the solc input object, to compile we ran the
TASK_COMPILE_SOLIDITY_COMPILEtask. This has three subtasks:The build subtask checks if the compiler for the given version is downloaded (downloading it if it is not), then verifies the checksum of the file. The solc or solcjs tasks (the actual solc compilation) are then run depending on which compiler was downloaded.
The build task adds hundreds of millisecond to a validation run (dwarfing the actual solc compile time).
This PR pulls up the subtasks of
TASK_COMPILE_SOLIDITY_COMPILEand memoizes the results ofTASK_COMPILE_SOLIDITY_GET_SOLC_BUILDusing a newcompilerMetadataCacheentry in theworkerStateof the worker process.The first build job in a worker for a particular solc version will take the cost of the compiler verification, but all subsequent build jobs for that worker will avoid it, speeding up validation.