Skip to content

Feature: Calculate usage info for streamed responses #46

Open
@jschuur

Description

@jschuur

Streamed responses don't include usage info in the response. Would have to calculate this via [tiktoken])(https://github.com/dqbd/tiktoken) e.g., but that would be a big dependency for the project.

Alternatively, count chunks manually for rough estimate (more info).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or requesthelp wantedExtra attention is neededusage

    Projects

    Status

    Next

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions