[OpenTelemetry] Fixed dropped self-diagnostics by martincostello · Pull Request #7543 · open-telemetry/opentelemetry-dotnet

martincostello · 2026-07-21T14:54:02Z

Changes

Fix self-diagnostic logs being dropped if they contained too many 3-byte characters. Now the content is only truncated.

Merge requirement checklist

CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
Unit tests added/updated
Appropriate CHANGELOG.md files updated for non-trivial changes
~~Changes in public API reviewed (if applicable)~~

Fix self-diagnostic logs being dropped if they contained too many 3-byte characters.

Add PR number.

codecov · 2026-07-21T15:11:43Z

Codecov Report

❌ Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.27%. Comparing base (f101632) to head (f1abd10).
⚠️ Report is 14 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...Telemetry/Internal/SelfDiagnosticsEventListener.cs	95.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7543      +/-   ##
==========================================
+ Coverage   90.23%   90.27%   +0.04%     
==========================================
  Files         290      290              
  Lines       15969    15989      +20     
==========================================
+ Hits        14409    14434      +25     
+ Misses       1560     1555       -5

Flag	Coverage Δ
unittests-Project-Experimental	`90.21% <95.00%> (-0.03%)`	⬇️
unittests-Project-Stable	`90.25% <95.00%> (-0.06%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...Telemetry/Internal/SelfDiagnosticsEventListener.cs	`94.77% <95.00%> (+0.03%)`	⬆️

... and 4 files with indirect coverage changes

opentelemetry-pr-dashboard · 2026-07-21T15:23:25Z

Pull request dashboard status

Status last refreshed: 2026-07-27 21:51:58 UTC.

Status: Merged.

This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected.

Copilot

Pull request overview

This PR fixes an edge case in OpenTelemetry’s self-diagnostics logging where UTF-8 byte-size estimation could overflow the internal buffer when messages/parameters contain enough 3-byte UTF-8 characters, causing Encoding.UTF8.GetBytes to throw and the entire log line to be dropped. The change adds a UTF-8 byte-counting walk to safely truncate content and adds regression tests plus a CHANGELOG entry.

Changes:

Adjusted SelfDiagnosticsEventListener.EncodeInBuffer to compute how many UTF-16 code units actually fit in the remaining buffer based on UTF-8 byte length, preventing exceptions and log-line drops.
Added unit tests covering truncation behavior for 3-byte UTF-8 characters (both message and parameter cases).
Added a CHANGELOG entry describing the behavior fix.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
`src/OpenTelemetry/Internal/SelfDiagnosticsEventListener.cs`	Adds UTF-8-aware truncation logic to avoid buffer-overflow exceptions that previously dropped self-diagnostic log lines.
`test/OpenTelemetry.Tests/Internal/SelfDiagnosticsEventListenerTests.cs`	Adds regression tests validating truncation for 3-byte UTF-8 characters and keeps source ASCII-only via escape sequence.
`src/OpenTelemetry/CHANGELOG.md`	Documents the self-diagnostics truncation fix in the Unreleased section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Fix handling of low/high surrogates.

stevejgordon

Looks good as proposed

xiang17

Looks good. This rare corner case has been causing silent drops that's hard to find.

xiang17 · 2026-07-27T19:30:01Z

+        var endPos = SelfDiagnosticsEventListener.EncodeInBuffer(str, false, buffer, 0);
+
+        var written = Encoding.UTF8.GetString(buffer, 0, endPos);
+        Assert.StartsWith(ThreeByteChar.ToString(), written, StringComparison.Ordinal);


nit: would it be better to verify the exact number of ThreeByteChars?

….0 (#1316) Updated [OpenTelemetry.Exporter.OpenTelemetryProtocol](https://github.com/open-telemetry/opentelemetry-dotnet) from 1.17.0 to 1.18.0. <details> <summary>Release notes</summary> _Sourced from [OpenTelemetry.Exporter.OpenTelemetryProtocol's releases](https://github.com/open-telemetry/opentelemetry-dotnet/releases)._ ## 1.18.0 For highlights and announcements pertaining to this release see: [Release Notes > 1.18.0](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/RELEASENOTES.md#1180). The following changes are from the previous release [1.17.0](https://github.com/open-telemetry/opentelemetry-dotnet/releases/tag/core-1.17.0). * NuGet: [OpenTelemetry v1.18.0](https://www.nuget.org/packages/OpenTelemetry/1.18.0) * Fixed self-diagnostics log lines being silently dropped when an event message or parameter contained enough 3-byte UTF-8 characters to overflow the internal buffer estimate. Such content is now truncated. ([#7543](open-telemetry/opentelemetry-dotnet#7543)) * Fixed activity creation throwing when multiple tracer providers return a sampler attribute with the same key. ([#7558](open-telemetry/opentelemetry-dotnet#7558)) * Added the `otel.sdk.processor.log.processed` SDK self-observability metric. ([#7486](open-telemetry/opentelemetry-dotnet#7486)) * Added the `otel.sdk.processor.span.processed` SDK self-observability metric. ([#7598](open-telemetry/opentelemetry-dotnet#7598)) * `BatchActivityExportProcessor` and `SimpleActivityExportProcessor` no longer forward spans to the exporter once `Shutdown` has been called, and `BatchActivityExportProcessor.Shutdown` now waits for in-flight `OnEnd` calls to finish enqueueing before flushing. ([#7598](open-telemetry/opentelemetry-dotnet#7598)) * Fix logger, meter and tracer providers leaking background threads if an exception is thrown by their constructor after resource creation. ([#7615](open-telemetry/opentelemetry-dotnet#7615)) * `CircularBufferBuckets.Copy` optimized to use bulk array copies. ([#7670](open-telemetry/opentelemetry-dotnet#7670)) * Restored configured `MaxScale` after delta exponential histogram collection. ([#7671](open-telemetry/opentelemetry-dotnet#7671)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0/src/OpenTelemetry/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Api v1.18.0](https://www.nuget.org/packages/OpenTelemetry.Api/1.18.0) * Avoid formatting exceptions and creating exception attributes when `RecordException` is called on a span that is not recorded. ([#7669](open-telemetry/opentelemetry-dotnet#7669)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0/src/OpenTelemetry.Api/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Api.ProviderBuilderExtensions v1.18.0](https://www.nuget.org/packages/OpenTelemetry.Api.ProviderBuilderExtensions/1.18.0) No notable changes. See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0/src/OpenTelemetry.Api.ProviderBuilderExtensions/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Exporter.Console v1.18.0](https://www.nuget.org/packages/OpenTelemetry.Exporter.Console/1.18.0) * Added support for serializing attribute values that are key/value lists (`IEnumerable<KeyValuePair<string, object?>>`). These attributes will be serialized as JSON objects. ([#7015](open-telemetry/opentelemetry-dotnet#7015)) ... (truncated) ## 1.18.0-rc.1 The following changes are from the previous release [1.17.0](https://github.com/open-telemetry/opentelemetry-dotnet/releases/tag/core-1.17.0). * NuGet: [OpenTelemetry v1.18.0-rc.1](https://www.nuget.org/packages/OpenTelemetry/1.18.0-rc.1) * Fixed self-diagnostics log lines being silently dropped when an event message or parameter contained enough 3-byte UTF-8 characters to overflow the internal buffer estimate. Such content is now truncated. ([#7543](open-telemetry/opentelemetry-dotnet#7543)) * Fixed activity creation throwing when multiple tracer providers return a sampler attribute with the same key. ([#7558](open-telemetry/opentelemetry-dotnet#7558)) * Added the `otel.sdk.processor.log.processed` SDK self-observability metric. ([#7486](open-telemetry/opentelemetry-dotnet#7486)) * Added the `otel.sdk.processor.span.processed` SDK self-observability metric. ([#7598](open-telemetry/opentelemetry-dotnet#7598)) * `BatchActivityExportProcessor` and `SimpleActivityExportProcessor` no longer forward spans to the exporter once `Shutdown` has been called, and `BatchActivityExportProcessor.Shutdown` now waits for in-flight `OnEnd` calls to finish enqueueing before flushing. ([#7598](open-telemetry/opentelemetry-dotnet#7598)) * Fix logger, meter and tracer providers leaking background threads if an exception is thrown by their constructor after resource creation. ([#7615](open-telemetry/opentelemetry-dotnet#7615)) * `CircularBufferBuckets.Copy` optimized to use bulk array copies. ([#7670](open-telemetry/opentelemetry-dotnet#7670)) * Restored configured `MaxScale` after delta exponential histogram collection. ([#7671](open-telemetry/opentelemetry-dotnet#7671)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0-rc.1/src/OpenTelemetry/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Api v1.18.0-rc.1](https://www.nuget.org/packages/OpenTelemetry.Api/1.18.0-rc.1) * Avoid formatting exceptions and creating exception attributes when `RecordException` is called on a span that is not recorded. ([#7669](open-telemetry/opentelemetry-dotnet#7669)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0-rc.1/src/OpenTelemetry.Api/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Api.ProviderBuilderExtensions v1.18.0-rc.1](https://www.nuget.org/packages/OpenTelemetry.Api.ProviderBuilderExtensions/1.18.0-rc.1) No notable changes. See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0-rc.1/src/OpenTelemetry.Api.ProviderBuilderExtensions/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Exporter.Console v1.18.0-rc.1](https://www.nuget.org/packages/OpenTelemetry.Exporter.Console/1.18.0-rc.1) * Added support for serializing attribute values that are key/value lists (`IEnumerable<KeyValuePair<string, object?>>`). These attributes will be serialized as JSON objects. ([#7015](open-telemetry/opentelemetry-dotnet#7015)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/core-1.18.0-rc.1/src/OpenTelemetry.Exporter.Console/CHANGELOG.md) for details. ... (truncated) ## 1.18.0-beta.1 The following changes are from the previous release [1.17.0-beta.1](https://github.com/open-telemetry/opentelemetry-dotnet/releases/tag/coreunstable-1.17.0-beta.1). * NuGet: [OpenTelemetry.Exporter.Prometheus.AspNetCore v1.18.0-beta.1](https://www.nuget.org/packages/OpenTelemetry.Exporter.Prometheus.AspNetCore/1.18.0-beta.1) * Fix concurrent scrapes returning an empty response under contention. Now the exporter will return an HTTP 500 error instead. ([#7571](open-telemetry/opentelemetry-dotnet#7571)) * Waiting for concurrent scrapes to finish before collecting no longer blocks, which could stall concurrent scrapes being waited on. ([#7571](open-telemetry/opentelemetry-dotnet#7571)) * Fixed the interaction between `PrometheusAspNetCoreOptions.TranslationStrategy` and content negotiation. The configured strategy is now applied before content negotiation, instead of the negotiated escaping scheme replacing the strategy's, and the `Content-Type` header now reports the escaping scheme that was applied rather than the one that was negotiated. ([#7610](open-telemetry/opentelemetry-dotnet#7610)) * Fixed metric values and histogram bucket bounds being written with 17 significant digits instead of their shortest round-trippable representation. ([#7589](open-telemetry/opentelemetry-dotnet#7589)) * Fixed the canonical representation used for histogram `le` and summary `quantile` label values falling back to 17 significant digits incorrectly. ([#7589](open-telemetry/opentelemetry-dotnet#7589)) * Fixed a race where a slow scrape could return an HTTP 200 instead of 408. ([#7615](open-telemetry/opentelemetry-dotnet#7615)) * Updated OpenTelemetry core component version(s) to `1.18.0`. ([#7674](open-telemetry/opentelemetry-dotnet#7674)) See [CHANGELOG](https://github.com/open-telemetry/opentelemetry-dotnet/blob/coreunstable-1.18.0-beta.1/src/OpenTelemetry.Exporter.Prometheus.AspNetCore/CHANGELOG.md) for details. * NuGet: [OpenTelemetry.Exporter.Prometheus.HttpListener v1.18.0-beta.1](https://www.nuget.org/packages/OpenTelemetry.Exporter.Prometheus.HttpListener/1.18.0-beta.1) * Fix concurrent scrapes returning an empty response under contention. Now the exporter will return an HTTP 500 error instead. ([#7571](open-telemetry/opentelemetry-dotnet#7571)) * Waiting for concurrent scrapes to finish before collecting no longer blocks, which could stall concurrent scrapes being waited on. ([#7571](open-telemetry/opentelemetry-dotnet#7571)) * A scrape which is still collecting when the listener is disposed now returns an HTTP 503 response. ([#7587](open-telemetry/opentelemetry-dotnet#7587)) * Shutting down the listener no longer waits indefinitely for its request processing loop to stop, and no longer throws if the loop faulted. ([#7587](open-telemetry/opentelemetry-dotnet#7587)) * Fixed the interaction between `PrometheusHttpListenerOptions.TranslationStrategy` and content negotiation. The configured strategy is now applied before content negotiation, instead of the negotiated escaping scheme replacing the strategy's, and the `Content-Type` header now reports the escaping scheme that was applied rather than the one that was negotiated. ([#7610](open-telemetry/opentelemetry-dotnet#7610)) * Fixed metric values and histogram bucket bounds being written with 17 significant digits instead of their shortest round-trippable representation. ([#7589](open-telemetry/opentelemetry-dotnet#7589)) * Fixed the canonical representation used for histogram `le` and summary `quantile` label values falling back to 17 significant digits incorrectly. ([#7589](open-telemetry/opentelemetry-dotnet#7589)) ... (truncated) Commits viewable in [compare view](open-telemetry/opentelemetry-dotnet@core-1.17.0...core-1.18.0). </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=OpenTelemetry.Exporter.OpenTelemetryProtocol&package-manager=nuget&previous-version=1.17.0&new-version=1.18.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[OpenTelemetry] Fixed dropped self-diagnostics

733dcae

Fix self-diagnostic logs being dropped if they contained too many 3-byte characters.

martincostello commented Jul 21, 2026

View reviewed changes

Comment thread src/OpenTelemetry/CHANGELOG.md Outdated

[OpenTelemetry] Update CHANGELOG

7a3276d

Add PR number.

github-actions Bot added the pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package label Jul 21, 2026

opentelemetry-pr-dashboard Bot mentioned this pull request Jul 21, 2026

Pull Request Dashboard #7539

Open

martincostello marked this pull request as ready for review July 21, 2026 15:50

martincostello requested a review from a team as a code owner July 21, 2026 15:50

Copilot AI review requested due to automatic review settings July 21, 2026 15:50

Copilot started reviewing on behalf of martincostello July 21, 2026 15:51 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread src/OpenTelemetry/Internal/SelfDiagnosticsEventListener.cs

[OpenTelemetry] Address feedback

f1abd10

Fix handling of low/high surrogates.

martincostello enabled auto-merge July 21, 2026 16:32

martincostello added the keep-open Prevents issues and pull requests being closed as stale label Jul 22, 2026

stevejgordon approved these changes Jul 24, 2026

View reviewed changes

xiang17 approved these changes Jul 27, 2026

View reviewed changes

rajkumar-rangaraj approved these changes Jul 27, 2026

View reviewed changes

martincostello added this pull request to the merge queue Jul 27, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 27, 2026

martincostello added this pull request to the merge queue Jul 27, 2026

Merged via the queue into open-telemetry:main with commit 008e176 Jul 27, 2026
75 checks passed

martincostello deleted the fixed-dropped-logs branch July 27, 2026 21:51

martincostello removed the keep-open Prevents issues and pull requests being closed as stale label Aug 13, 2026

martincostello added this to the v1.18.0 milestone Aug 13, 2026

Navigation Menu

Uh oh!

[OpenTelemetry] Fixed dropped self-diagnostics - #7543

[OpenTelemetry] Fixed dropped self-diagnostics#7543
martincostello merged 3 commits into
open-telemetry:mainfrom
martincostello:fixed-dropped-logs

martincostello commented Jul 21, 2026

Uh oh!

Uh oh!

codecov Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

opentelemetry-pr-dashboard Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

stevejgordon left a comment

Uh oh!

xiang17 left a comment

Uh oh!

xiang17 Jul 27, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

martincostello commented Jul 21, 2026

Changes

Merge requirement checklist

Uh oh!

Uh oh!

codecov Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

opentelemetry-pr-dashboard Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

stevejgordon left a comment

Choose a reason for hiding this comment

Uh oh!

xiang17 left a comment

Choose a reason for hiding this comment

Uh oh!

xiang17 Jul 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Jul 21, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Jul 21, 2026 •

edited

Loading