[MS-292] Removal of generating dynamic benchmark prompts on the fly #71
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Removal of generating dynamic benchmark prompts on the fly
Motivation and Context
This is to remove the feature where we will generate dynamic benchmark for dataset prompts on the fly.
This feature will be replaced with augmented benchmark where it will create a new recipe with a new dataset that contains the modified prompts.
Type of Change
How to Test
Perform a run_recipe and run_cookbook to get the benchmark to run.
There are no more attack modules that will augment the prompts anymore.
Checklist
Please check all the boxes that apply to this pull request using "x":
Screenshots (if applicable)
[If the changes involve visual modifications, include screenshots or GIFs that demonstrate the changes.]
Additional Notes
There is an accompanying update in moonshot
Developer Certificate of Origin