Skip to content

Add OpenAIModelClass for OpenAI-compatible API clients #594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 26, 2025

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented May 23, 2025

This PR adds a new OpenAIModelClass similar to the existing MCPModelClass, allowing developers to create models that interact with OpenAI-compatible API endpoints.

Implementation Details

  • Created OpenAIModelClass in clarifai/runners/models/openai_class.py that inherits from ModelClass
  • Added the openai_transport method decorated with @ModelClass.method to handle requests/responses with OpenAI-compatible servers
  • Implemented similar patterns as MCPModelClass with methods:
    • load_model() to initialize the client
    • get_openai_client() to be implemented by subclasses
    • Helper methods to process streaming and non-streaming responses
  • Created a DummyOpenAIModel implementation for testing purposes that demonstrates how to subclass OpenAIModelClass
  • Updated init.py files to expose the new classes
  • Added comprehensive tests to verify the implementation works correctly

Usage

Developers can subclass OpenAIModelClass and implement get_openai_client() to provide their own OpenAI-compatible client:

from clarifai.runners.models import OpenAIModelClass

class MyOpenAIModel(OpenAIModelClass):
    def get_openai_client(self):
        # Return an OpenAI-compatible client that connects to your server
        return YourOpenAIClient(...)

The openai_transport method handles converting between the API formats, supporting both streaming and non-streaming responses.

Fixes #593.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits May 23, 2025 12:17
Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] Add an OpenAIModelClass Add OpenAIModelClass for OpenAI-compatible API clients May 23, 2025
@Copilot Copilot AI requested a review from zeiler May 23, 2025 12:20
Copilot finished work on behalf of zeiler May 23, 2025 12:20
@zeiler
Copy link
Member

zeiler commented May 26, 2025

The API has been updated to include a openai_stream_transport function which should have a signature like this:

openai_stream_transport(req: star) -> Iterator[str]

This is to be used when stream equals true from an API client. Can you update this PR with that additional method and update all the tests please

Co-authored-by: zeiler <2138258+zeiler@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 26, 2025

The API has been updated to include a openai_stream_transport function which should have a signature like this:

openai_stream_transport(req: star) -> Iterator[str]
...

I've added the requested openai_stream_transport method to the OpenAIModelClass with the signature openai_stream_transport(req: str) -> Iterator[str]. This method handles direct streaming responses when stream=True without JSON serialization. I've also updated all tests to verify this functionality works correctly with both normal and error cases. The implementation is available in commit fde9936.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.fwupd.org
    • Triggering command: /usr/bin/fwupdmgr refresh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of zeiler May 26, 2025 04:39
@sainivedh sainivedh marked this pull request as ready for review May 26, 2025 06:05
@sainivedh
Copy link
Contributor

Fix the lint issues

Co-authored-by: sainivedh <39689508+sainivedh@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 26, 2025

Fix the lint issues

Fixed the lint issues in commit 3baf841. Fixed import ordering, removed trailing whitespace, added missing newlines at the end of files, and ensured all files pass Ruff linting checks. All tests are passing.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.fwupd.org
    • Triggering command: /usr/bin/fwupdmgr refresh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copy link

Code Coverage

Package Line Rate Health
clarifai 43%
clarifai.cli 43%
clarifai.client 71%
clarifai.client.auth 74%
clarifai.constants 100%
clarifai.datasets 100%
clarifai.datasets.export 80%
clarifai.datasets.upload 75%
clarifai.datasets.upload.loaders 37%
clarifai.models 100%
clarifai.modules 0%
clarifai.rag 72%
clarifai.runners 14%
clarifai.runners.models 59%
clarifai.runners.utils 63%
clarifai.runners.utils.data_types 72%
clarifai.schema 100%
clarifai.urls 75%
clarifai.utils 73%
clarifai.utils.evaluation 67%
clarifai.workflows 94%
Summary 66% (6277 / 9572)

Minimum allowed line rate is 50%

@sainivedh sainivedh enabled auto-merge (squash) May 26, 2025 11:59
@sainivedh sainivedh merged commit b8a6978 into master May 26, 2025
9 checks passed
@sainivedh sainivedh deleted the copilot/fix-593 branch May 26, 2025 11:59

# Process the request using the provided OpenAI client
try:
if stream:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should actually never be stream if this method is hit as the API decides to use openai_transport versus openai_stream_transport based on the stream flag already.

@srikanthbachala20 srikanthbachala20 mentioned this pull request May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Add an OpenAIModelClass
3 participants