Skip to content

Add support for streaming #100

@felixarntz

Description

@felixarntz

See #8: A crucial piece of functionality still missing in our SDK is response streaming - a quite complex piece of functionality to get right, especially with the incomplete documentation of how different providers implement it. But we need it, that's for sure.

What we'll need to do is:

  • Implement the primitives needed for streaming (likely some kind of stream response class, potentially a new stream method on HttpTransporter).
  • Implement actual streaming for the three providers (which can fortunately be centralized in the OpenAI API compatible implementation for now).

Putting here some references for inspiration in how I implemented streaming (still with some limitations) in the AI Services plugin for WordPress:

There's definitely things to improve there which we should do here, but it's a decent starting point to review this and come up with effectively a v2 of that implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    [Feature]New feature to highlight in changelogs.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions