-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RedTeam] Enable unit tests for RedTeam by updating dev requirements and min dependencies for pyrit #40311
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request updates the dependency versions for the evaluation SDK by raising the minimum version for azure-identity and loosening the version constraint for pyrit, while also adding additional exception handling to gracefully handle edge cases in ASR computation within the red team evaluation module.
- Updated azure-identity version from 1.16.0 to 1.19.0.
- Changed the pyrit dependency from an exact version (0.8.0) to a range (>=0.8.0).
- Introduced try/except blocks in the _red_team.py file to handle potential EvaluationException occurrences in various ASR calculations.
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.
File | Description |
---|---|
sdk/evaluation/azure-ai-evaluation/setup.py | Updated dependency versions for azure-identity and pyrit to align with current requirements. |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_red_team/_red_team.py | Added try/except blocks to handle EvaluationException for various ASR computations, ensuring safer calculation of metrics. |
Files not reviewed (1)
- sdk/evaluation/azure-ai-evaluation/dev_requirements.txt: Language not supported
Comments suppressed due to low confidence (2)
sdk/evaluation/azure-ai-evaluation/setup.py:71
- Ensure that updating 'azure-identity' to version 1.19.0 has been fully tested for compatibility across the SDK, as dependency changes can have broader impacts.
"azure-identity>=1.19.0",
sdk/evaluation/azure-ai-evaluation/setup.py:77
- Changing from an exact version to a version range for pyrit may introduce compatibility issues if newer versions contain breaking changes; consider verifying that this range is appropriate or pinning to a known stable version if needed.
"redteam": ["pyrit>=0.8.0"]
API change check APIView has identified API level changes in this PR and created following API reviews. |
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines