Skip to content

OpenAI Responses API Instrumentation - #1813

Open
TimPansino wants to merge 15 commits into
mainfrom
feat-openai-responses-apis
Open

OpenAI Responses API Instrumentation#1813
TimPansino wants to merge 15 commits into
mainfrom
feat-openai-responses-apis

Conversation

@TimPansino

@TimPansino TimPansino commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Overview

  • Instrument the newer openai.responses APIs.
  • Add a new set of tests that mirrors the original chat_completions tests.

@TimPansino
TimPansino requested a review from a team as a code owner August 6, 2026 17:27
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 9 0 0 1.13s
✅ MARKDOWN markdownlint 7 0 0 0 1.41s
✅ PYTHON ruff 1096 0 0 0 1.26s
✅ PYTHON ruff-format 1096 0 0 0 0.35s
✅ YAML prettier 23 0 0 0 1.82s
✅ YAML v8r 23 0 0 5.93s
✅ YAML yamllint 23 0 0 0.81s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 6, 2026
@codecov-commenter

codecov-commenter commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.93103% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.14%. Comparing base (b67264e) to head (d7ef060).

Files with missing lines Patch % Lines
newrelic/hooks/mlmodel_openai.py 87.82% 2 Missing and 12 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1813      +/-   ##
==========================================
+ Coverage   82.12%   82.14%   +0.01%     
==========================================
  Files         218      218              
  Lines       27085    27188     +103     
  Branches     4286     4309      +23     
==========================================
+ Hits        22244    22333      +89     
- Misses       3353     3355       +2     
- Partials     1488     1500      +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TimPansino
TimPansino force-pushed the feat-openai-responses-apis branch from 5f17359 to 12dfc71 Compare August 7, 2026 21:22
@mergify mergify Bot removed the tests-failing Tests failing in CI. label Aug 7, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 10, 2026
@mergify mergify Bot removed the tests-failing Tests failing in CI. label Aug 10, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 12, 2026
@mergify mergify Bot removed the tests-failing Tests failing in CI. label Aug 13, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 13, 2026
if isinstance(item, dict) and item.get("role") is not None:
content = item.get("content")
# Content is a string, or a list of typed parts each carrying `text`.
if isinstance(content, (list, tuple)):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For chat completions, we deliberately suppress content-less tool call messages (on line 595 by checking "tool_call" in finish_reason). Since we don't want to report every tool call as an LlmChatCompletionMessage with content: "", do we want to mirror that suppression behavior here if possible?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is that why "role" is being checked for explicitly?

)
stream_proxy._nr_openai_attrs["content"] = stream_proxy._nr_openai_attrs.get("content", "") + delta
stream_proxy._nr_openai_attrs["role"] = stream_proxy._nr_openai_attrs.get("role") or "assistant"
elif event_type in ("response.completed", "response.incomplete", "response.failed"):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we capturing errors surfaced via response.failed somewhere? From my scan of the codebase, it seems like APIErrors are raised when the raw payload has a top-level error key. response.failed events nest errors inside response.error and the stream just ends normally afterwards. I just want to make sure we are capturing errors in this case also.

@mergify mergify Bot removed the tests-failing Tests failing in CI. label Aug 17, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 19, 2026
@mergify mergify Bot removed the tests-failing Tests failing in CI. label Aug 19, 2026
@mergify mergify Bot added the tests-failing Tests failing in CI. label Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests-failing Tests failing in CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants