Closed
Description
Package Name
ddtrace
Package Version(s)
3.10.0rc1
Describe the goal of the feature
Enhance ddtrace’s OpenAI integration to:()
• Extract the cached_tokens value from API responses.
• Include cached_tokens in relevant metrics and traces.
• Adjust cost calculations to account for the discounted rate of cached tokens.()
Additional context
OpenAI’s prompt caching is available for models like gpt-4o, gpt-4o-mini, o1-preview, and o1-mini with API versions such as 2024-10-01-preview. Incorporating cached_tokens into ddtrace would provide more accurate observability and cost tracking for applications utilizing these models.