-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Add support for overriding model name in agents tests #44268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
||
| test_agents_params = { | ||
| "model_deployment_name": "gpt-4o", | ||
| "model_deployment_name": os.environ.get("AZURE_AI_PROJECTS_TESTS_AGENTS_MODEL_DEPLOYMENT_NAME") or "gpt-4o", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dargilco I am wondering whether there is some extra work with the service preparer that I will need to do in this area once we have working recordings for these tests? Since we dont have working recordings today it was hard for me to be sure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR enables the Azure AI Projects agents tests to use configurable model deployments through environment variables, allowing tests to run against different models in end-to-end scenarios. The changes maintain backward compatibility by providing default values when environment variables are not set.
Key Changes
- Added environment variable support for overriding the default agent model deployment name ("gpt-4o")
- Updated the image generation tool to explicitly use the image model deployment parameter
- Added new environment variable template entry for the agent model deployment name configuration
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
sdk/ai/azure-ai-projects/tests/test_base.py |
Added os import and modified test_agents_params to read model deployment name from environment variable with fallback to "gpt-4o" |
sdk/ai/azure-ai-projects/tests/agents/tools/test_agent_image_generation.py |
Added model parameter to ImageGenTool instantiation to explicitly pass the image model deployment |
sdk/ai/azure-ai-projects/.env.template |
Added AZURE_AI_PROJECTS_TESTS_AGENTS_MODEL_DEPLOYMENT_NAME environment variable template entry |
The implementation follows existing patterns in the codebase (e.g., AZURE_AI_PROJECTS_TESTS_IMAGE_MODEL_DEPLOYMENT_NAME) and maintains backward compatibility. All agent tests that reference test_agents_params["model_deployment_name"] will automatically benefit from this change without requiring modifications.
c64df44 to
777cadb
Compare
Description
Small changes that enable the foundry projects agents tests to use other models. I have validated this change through a devops pipeline that runs the agents tests in an e2e manner against multiple models.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines