Skip to content

Conversation

@pgrayy
Copy link
Member

@pgrayy pgrayy commented May 21, 2025

Description

Certain models executed under LiteLLM will return additional payloads inbetween the finish_reason message and the usage message. Here is an example of an inbetween payload:

ModelResponseStream(id='chatcmpl-3cdd4223-9c65-4ce0-9456-0a41d4a70b86', created=1747836095, model='gpt-4.1-mini-2025-04-14', object='chat.completion.chunk', system_fingerprint='fp_c8225066ea', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content=None, role=None, function_call=None, tool_calls=None, audio=None), logprobs=None)], provider_specific_fields=None, stream_options={'include_usage': True})

We don't have any use for these inbetween messages and so for now, we will skip over them so we can get to the final usage message for reporting.

Related Issues

#47

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

  • hatch fmt --linter
  • hatch fmt --formatter
  • hatch test --all: Relying on existing unit tests and integration tests. They still cover these changes.
  • Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

Checklist

  • I have read the CONTRIBUTING document
  • I have added tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@pgrayy pgrayy requested a review from a team as a code owner May 21, 2025 14:29
zastrowm
zastrowm previously approved these changes May 21, 2025
@pgrayy pgrayy merged commit d0a57a9 into strands-agents:main May 21, 2025
10 checks passed
@pgrayy pgrayy deleted the litellm-usage branch May 27, 2025 13:35
mkmeral pushed a commit to mkmeral/sdk-python that referenced this pull request Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants