Add Responses OpenTelemetry instrumentation - #1347
Conversation
Instrument non-streaming response creation with GenAI spans and metrics. Support requests without an explicit model and cover independent tracing and metrics paths.
|
@lmolkova: Is there any chance that you may be able to clarify the expected OpenTelemetry behavior for an OpenAI Responses request where the SDK does not receive a model value? The base span_refinements:
- id: openai.inference.client
ref: gen_ai.inference.client
attributes:
- ref: gen_ai.request.model
requirement_level: requiredThe generated OpenAI documentation also lists I'm not clear what the right behavior is when a valid Responses request omits the client-side In this case, should client instrumentation:
Is there existing guidance, or should the OpenAI refinement explicitly describe model selection delegated to a saved prompt or other server-side request state? Thank you for any insights! Sources: |
There was a problem hiding this comment.
Pull request overview
Adds experimental OpenTelemetry instrumentation for non-streaming OpenAI.Responses.ResponsesClient.CreateResponse / CreateResponseAsync, extending the SDK’s existing telemetry surface area while honoring GenAI semantic convention stability opt-in and privacy constraints.
Changes:
- Introduces a dedicated
OpenAI.ResponsesClientActivitySource/Meterand records spans + GenAI metrics for non-streaming Responses creation. - Captures additional latest-experimental semantic convention attributes (conversation IDs, service tiers, reasoning effort, output type, cache/reasoning token details, compaction state).
- Adds synthetic telemetry tests plus documentation/changelog updates describing new coverage and limitations.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Telemetry/ResponsesTelemetryTests.cs | Adds synthetic tests validating Responses telemetry behavior (traces, metrics, semantic convention opt-in, privacy exclusions). |
| OpenAI/src/Utility/Telemetry/OpenTelemetrySource.cs | Adds Responses scope entrypoint and centralizes semantic convention selection state. |
| OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs | Implements Responses-specific span/metric recording, refactors histogram creation, and updates error handling behavior. |
| OpenAI/src/Utility/Telemetry/OpenTelemetryConstants.cs | Adds new attribute/metric keys needed for Responses + latest semantic conventions. |
| OpenAI.Responses/src/Custom/ResponsesClient.cs | Wraps strongly-typed non-streaming CreateResponse* methods with telemetry scopes and exception recording. |
| docs/Observability.md | Updates semantic conventions link and documents Responses telemetry coverage + privacy stance. |
| CHANGELOG.md | Notes new experimental Responses OpenTelemetry coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 317ca9a591
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
tests/Telemetry/ResponsesTelemetryTests.cs:521
- The synthetic Responses payload uses an uppercased item type ("COMPACTION"), but the Responses TypeSpec and existing session recordings use the lowercase value "compaction". Keeping the casing consistent with real API payloads makes the test more representative and avoids masking potential casing-related issues.
"type": "COMPACTION",
Summary
This change adds experimental OpenTelemetry instrumentation for non-streaming
ResponsesClient.CreateResponseandCreateResponseAsyncoperations.The implementation builds on the existing Chat telemetry infrastructure while using a separate
OpenAI.ResponsesClientactivity source and meter. It follows the current OpenTelemetry Generative AI semantic conventions and supports both the default convention behavior andOTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental.Partially addresses #653. Streaming Responses telemetry remains outside the scope of this change and will be added in a follow-up pull request after this merges..
Details
The new instrumentation:
gen_ai.operation.namewith thechatoperation.chat {model}as the span name when the requested model is available andchatwhen it is not.gen_ai.client.operation.durationandgen_ai.client.token.usagemetrics through theOpenAI.ResponsesClientmeter.Telemetry is attached only to the strongly typed non-streaming methods. Generated protocol methods, streaming response creation, stored-response retrieval, cancellation, deletion, and input-item listing are not instrumented by this change.
Privacy
The implementation does not record prompts, generated output, instructions, tool definitions, tool arguments or results, metadata, safety identifiers, end-user identifiers, prompt cache keys, multimodal payloads, or provider-controlled error messages.
Enabling experimental OpenTelemetry support does not opt callers into content capture.
Testing
The synthetic telemetry coverage includes:
The observability documentation and changelog have also been updated to describe the new coverage and its current limitations.