Skip to content

Add Responses OpenTelemetry instrumentation - #1347

Open
jsquire wants to merge 3 commits into
openai:mainfrom
jsquire:responses/otel
Open

Add Responses OpenTelemetry instrumentation#1347
jsquire wants to merge 3 commits into
openai:mainfrom
jsquire:responses/otel

Conversation

@jsquire

@jsquire jsquire commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

This change adds experimental OpenTelemetry instrumentation for non-streaming ResponsesClient.CreateResponse and CreateResponseAsync operations.

The implementation builds on the existing Chat telemetry infrastructure while using a separate OpenAI.ResponsesClient activity source and meter. It follows the current OpenTelemetry Generative AI semantic conventions and supports both the default convention behavior and OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental.

Partially addresses #653. Streaming Responses telemetry remains outside the scope of this change and will be added in a follow-up pull request after this merges..

Details

The new instrumentation:

  • Creates client spans for strongly typed, non-streaming Responses operations.
  • Records gen_ai.operation.name with the chat operation.
  • Uses chat {model} as the span name when the requested model is available and chat when it is not.
  • Records request model, response model, response ID, supported request parameters, finish reasons, token usage, service tier, reasoning effort, conversation details, and compaction state where defined by the active semantic conventions.
  • Emits gen_ai.client.operation.duration and gen_ai.client.token.usage metrics through the OpenAI.ResponsesClient meter.
  • Marks failed and cancelled responses as errors and records low-cardinality error types without capturing provider-controlled error messages.
  • Makes sampling-relevant attributes available when the span is created.
  • Supports requests where the model is supplied implicitly, such as future saved-prompt scenarios, while preserving an explicitly supplied model as the requested model.

Telemetry is attached only to the strongly typed non-streaming methods. Generated protocol methods, streaming response creation, stored-response retrieval, cancellation, deletion, and input-item listing are not instrumented by this change.

Privacy

The implementation does not record prompts, generated output, instructions, tool definitions, tool arguments or results, metadata, safety identifiers, end-user identifiers, prompt cache keys, multimodal payloads, or provider-controlled error messages.

Enabling experimental OpenTelemetry support does not opt callers into content capture.

Testing

The synthetic telemetry coverage includes:

  • Synchronous and asynchronous response creation.
  • Default and latest experimental semantic conventions.
  • Metrics enabled without tracing.
  • Tracing enabled without metrics.
  • Request and response attributes.
  • Sampling-time attributes.
  • Duration and token usage metrics.
  • Requests without an explicit model.
  • HTTP failures and response status mapping.
  • Completed, incomplete, failed, and cancelled responses.
  • Unknown future incomplete reasons.
  • Response compaction.
  • Sensitive-data exclusion.
  • Protocol method exclusion.
  • Convenience overload deduplication.

The observability documentation and changelog have also been updated to describe the new coverage and its current limitations.

Instrument non-streaming response creation with GenAI
spans and metrics. Support requests without an explicit
model and cover independent tracing and metrics paths.
Copilot AI lite review requested due to automatic review settings August 28, 2026 23:13
@jsquire jsquire self-assigned this Aug 28, 2026
@jsquire jsquire added the area: responses This item is related to Responses. label Aug 28, 2026
@jsquire

jsquire commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

@lmolkova: Is there any chance that you may be able to clarify the expected OpenTelemetry behavior for an OpenAI Responses request where the SDK does not receive a model value?

The base gen_ai.inference.client span defines gen_ai.request.model as conditionally required when available. The OpenAI-specific openai.inference.client refinement overrides that inherited requirement and marks the attribute as required:

span_refinements:
  - id: openai.inference.client
    ref: gen_ai.inference.client
    attributes:
      - ref: gen_ai.request.model
        requirement_level: required

The generated OpenAI documentation also lists gen_ai.request.model as required and specifies the span name as {gen_ai.operation.name} {gen_ai.request.model}.

I'm not clear what the right behavior is when a valid Responses request omits the client-side model parameter from the client while still using a model selected by the service. For example, a saved prompt can configure the effective model server-side. The SDK knows the prompt identifier, but it does not know the model selected by that prompt when it starts the activity. Inferring a model from the prompt identifier would be inaccurate, and gen_ai.response.model is not available until after the request completes, which is too late for head-based sampling.

In this case, should client instrumentation:

  1. Create the openai.inference.client span without gen_ai.request.model
  2. Omit inference instrumentation for the valid operation
  3. Use a different convention or attribute pattern

Is there existing guidance, or should the OpenAI refinement explicitly describe model selection delegated to a saved prompt or other server-side request state? Thank you for any insights!

Sources:

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds experimental OpenTelemetry instrumentation for non-streaming OpenAI.Responses.ResponsesClient.CreateResponse / CreateResponseAsync, extending the SDK’s existing telemetry surface area while honoring GenAI semantic convention stability opt-in and privacy constraints.

Changes:

  • Introduces a dedicated OpenAI.ResponsesClient ActivitySource/Meter and records spans + GenAI metrics for non-streaming Responses creation.
  • Captures additional latest-experimental semantic convention attributes (conversation IDs, service tiers, reasoning effort, output type, cache/reasoning token details, compaction state).
  • Adds synthetic telemetry tests plus documentation/changelog updates describing new coverage and limitations.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/Telemetry/ResponsesTelemetryTests.cs Adds synthetic tests validating Responses telemetry behavior (traces, metrics, semantic convention opt-in, privacy exclusions).
OpenAI/src/Utility/Telemetry/OpenTelemetrySource.cs Adds Responses scope entrypoint and centralizes semantic convention selection state.
OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs Implements Responses-specific span/metric recording, refactors histogram creation, and updates error handling behavior.
OpenAI/src/Utility/Telemetry/OpenTelemetryConstants.cs Adds new attribute/metric keys needed for Responses + latest semantic conventions.
OpenAI.Responses/src/Custom/ResponsesClient.cs Wraps strongly-typed non-streaming CreateResponse* methods with telemetry scopes and exception recording.
docs/Observability.md Updates semantic conventions link and documents Responses telemetry coverage + privacy stance.
CHANGELOG.md Notes new experimental Responses OpenTelemetry coverage.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs Outdated
Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 317ca9a591

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs Outdated
Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs Outdated
Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs Outdated
Copilot AI review requested due to automatic review settings August 28, 2026 23:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment thread OpenAI/src/Utility/Telemetry/OpenTelemetryScope.cs
Copilot AI review requested due to automatic review settings August 29, 2026 00:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/Telemetry/ResponsesTelemetryTests.cs:521

  • The synthetic Responses payload uses an uppercased item type ("COMPACTION"), but the Responses TypeSpec and existing session recordings use the lowercase value "compaction". Keeping the casing consistent with real API payloads makes the test more representative and avoids masking potential casing-related issues.
              "type": "COMPACTION",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: responses This item is related to Responses.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants