Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Total tokens not tallied with DSPy and AzureOpenAI client #4730

Closed
arthurgreef opened this issue Sep 24, 2024 · 1 comment
Closed

[BUG] Total tokens not tallied with DSPy and AzureOpenAI client #4730

arthurgreef opened this issue Sep 24, 2024 · 1 comment
Labels
bug Something isn't working triage issues that need triage

Comments

@arthurgreef
Copy link

Describe the bug
The total tokens count is not tallied when using Phoenix with DSPy and AzureOpenAI.

To Reproduce
Steps to reproduce the behavior:

  1. Run phoenix in python session.
  2. Trace any DSPy program that uses the AzureOpenAI client, with Phoenix.
  3. Open the Phoenix browser app.
  4. Total Tokens is '0'

Expected behavior
Expect the total tokens to be tallied from the following completion response.

"created": 1727198954,
  "model": "gpt-4o-2024-08-06",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": "fp_67802d9a6d",
  "usage": {
    "completion_tokens": 13,
    "prompt_tokens": 609,
    "total_tokens": 622
  },

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: Windows
  • Notebook Runtime N/A
  • Browser Edge
  • Version Latest

Additional context

@arthurgreef arthurgreef added bug Something isn't working triage issues that need triage labels Sep 24, 2024
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Sep 24, 2024
@axiomofjoy
Copy link
Contributor

Thanks @arthurgreef. This issue is a duplicate of #3119. The recommended workaround is to use our OpenAIInstrumentor in conjunction with DSPyInstrumentor.

@github-project-automation github-project-automation bot moved this from 📘 Todo to ✅ Done in phoenix Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage issues that need triage
Projects
Archived in project
Development

No branches or pull requests

2 participants