-
Notifications
You must be signed in to change notification settings - Fork 7
Add OpenAIModelClass for OpenAI-compatible API clients #594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
The API has been updated to include a openai_stream_transport function which should have a signature like this: openai_stream_transport(req: star) -> Iterator[str] This is to be used when stream equals true from an API client. Can you update this PR with that additional method and update all the tests please |
Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
I've added the requested Warning Firewall rules blocked me from connecting to one or more addressesI tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Fix the lint issues |
Co-authored-by: sainivedh <39689508+sainivedh@users.noreply.github.com>
Fixed the lint issues in commit 3baf841. Fixed import ordering, removed trailing whitespace, added missing newlines at the end of files, and ensured all files pass Ruff linting checks. All tests are passing. Warning Firewall rules blocked me from connecting to one or more addressesI tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Minimum allowed line rate is |
|
||
# Process the request using the provided OpenAI client | ||
try: | ||
if stream: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it should actually never be stream if this method is hit as the API decides to use openai_transport versus openai_stream_transport based on the stream flag already.
This PR adds a new
OpenAIModelClass
similar to the existingMCPModelClass
, allowing developers to create models that interact with OpenAI-compatible API endpoints.Implementation Details
OpenAIModelClass
inclarifai/runners/models/openai_class.py
that inherits fromModelClass
openai_transport
method decorated with@ModelClass.method
to handle requests/responses with OpenAI-compatible serversload_model()
to initialize the clientget_openai_client()
to be implemented by subclassesDummyOpenAIModel
implementation for testing purposes that demonstrates how to subclassOpenAIModelClass
Usage
Developers can subclass
OpenAIModelClass
and implementget_openai_client()
to provide their own OpenAI-compatible client:The
openai_transport
method handles converting between the API formats, supporting both streaming and non-streaming responses.Fixes #593.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.