Reuse jinja environment for a prompt #1162
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I really like the
@outlines.prompt
decorator. However, I noticed it is slow when formatting a large number of examples. The main inefficiency is creating the Jinja environment for every call ofrender
. This PR makes a minor change to re-use the same Jinja environment per prompt.Note: This makes the
render
function obsolete. However, I kept it for backwards compatibility. Please advise if this should be further refactored.Benchmark
Small benchmark to test it:
With
main
:With this PR: