Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for configuring AutoGen's api_type and api_version in order to support Azure models #63

Merged
merged 5 commits into from
Jul 4, 2024

Conversation

cedricvidal
Copy link
Contributor

@cedricvidal cedricvidal commented Jul 1, 2024

This PR adds the possibility to configure AutoGen's api_type and api_version through the following environment variables:

  • AUTOGEN_MODEL_API_TYPE
  • AUTOGEN_MODEL_API_VERSION

This allows to configure an Azure AI Model, for example GPT-4o with the following .env:

AUTOGEN_MODEL_NAME=gpt-4o
AUTOGEN_MODEL_API_KEY=<REPLACE_ME>
AUTOGEN_MODEL_BASE_URL=https://<REPLACE_ME>.openai.azure.com
AUTOGEN_MODEL_API_TYPE=azure
AUTOGEN_MODEL_API_VERSION=2023-03-15-preview

This PR also updates the README to explain how to use models on Azure.

Note

This PR was developed and tested successfully against a26b870. I stumbled upon #62 while trying to merge this PR on master so I haven't been able to test this PR on master.

@cedricvidal cedricvidal marked this pull request as ready for review July 1, 2024 20:42
@teaxio teaxio changed the base branch from master to dev July 2, 2024 15:02
@teaxio
Copy link
Collaborator

teaxio commented Jul 2, 2024

changed your base branch to dev, will call that out explicitly in the readme

@teaxio teaxio requested review from deepak-akkil and teaxio July 4, 2024 14:12
@teaxio
Copy link
Collaborator

teaxio commented Jul 4, 2024

@cedricvidal
We added prompt improvement as the way that gpt4 responds on azure is quite a bit different from openai’s.
I do see timeouts like this quite a bit:
INFO:openai._base_client:Retrying request to /chat/completions in 40.000000 seconds

@deepak-akkil merged some changes to your PR so we can see it work end to end. It eventually works once the timeouts are overcome, but it leaves much to be desired.
Let's collaborate on trying to figure out what's going on so that the timeouts are not happening. This is something between autogen and azure for sure.

@teaxio teaxio merged commit 6cae89c into EmergenceAI:dev Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants