Change mechanism for component evaluation #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gotwarlost
force-pushed
the
eval
branch
2 times, most recently
from
abe1633 to
0c578e6
Compare
Previously components were evaluated by creating a megacomponent that imported JSON, YAML and JSONNET files. This provided little to no context for errors. We change this to evaluate every component in its own jsonnet VM. Processing components like this sequentially leads to 30-40% overhead so we need to evaluate components concurrently. This is done with a default concurrency of 5. The user can change this number using the persistent `--eval-concurrency` flag. This change reduces the time to evaluate a large set of components _and_ produces better messages that have the failed component name when things do not load correctly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously components were evaluated by creating a megacomponent that imported
JSON, YAML and JSONNET files. This provided little to no context for errors.
We change this to evaluate every component in its own jsonnet VM.
Processing components like this sequentially leads to 30-40% overhead so we need to evaluate
components concurrently. This is done with a default concurrency of 5. The user
can change this number using the persistent
--eval-concurrencyflag.This change reduces the time to evaluate a large set of components and produces better messages
that have the failed component name when things do not load correctly.