Skip to content

Update OTLP headers to accept string format (spec v1.13.0)#3338

Merged
lpcox merged 3 commits intomainfrom
copilot/update-gateway-accept-otlp-headers
Apr 7, 2026
Merged

Update OTLP headers to accept string format (spec v1.13.0)#3338
lpcox merged 3 commits intomainfrom
copilot/update-gateway-accept-otlp-headers

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 7, 2026

Spec v1.13.0 changes opentelemetry.headers from an object to a comma-separated key=value string. The compiler now emits "headers": "Authorization=******" instead of a JSON object.

Changes

  • TracingConfig.Headers / StdinOpenTelemetryConfig.Headers: map[string]stringstring
  • expandTracingVariables: expands the headers string directly instead of iterating map values
  • parseOTLPHeaders (new): parses "key=value,key=value" into map[string]string at export time; logs a warning for malformed pairs
  • resolveHeaders: delegates to parseOTLPHeaders
  • JSON schema: opentelemetry.headers changed from type: object + additionalProperties: string to type: string

Example

# Before (TOML object syntax)
[gateway.opentelemetry.headers]
Authorization = "******"

# After (string)
[gateway.opentelemetry]
headers = "Authorization=******"
// Stdin JSON — before
"headers": { "Authorization": "******" }

// After
"headers": "Authorization=******"

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /tmp/go-build113898129/b514/launcher.test /tmp/go-build113898129/b514/launcher.test -test.testlogfile=/tmp/go-build113898129/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build113898129/b216/vet.cfg 7845551/b412/_pkg_.a 64/src/go/scanner/errors.go ache/go/1.25.8/x64/pkg/tool/linu-nilfunc -p l -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linu-tests -w 738659/b232/_pkg_.a ylov/l0IgnnYtDUbmqPCsylov x_amd64/vet -c -I /tmp/go-build171-bool x_amd64/vet (dns block)
  • invalid-host-that-does-not-exist-12345.com
    • Triggering command: /tmp/go-build861738659/b222/config.test /tmp/go-build861738659/b222/config.test -test.testlogfile=/tmp/go-build861738659/b222/testlog.txt -test.paniconexit0 -test.timeout=10m0s -c go t.go x_amd64/compile (dns block)
  • nonexistent.local
    • Triggering command: /tmp/go-build113898129/b514/launcher.test /tmp/go-build113898129/b514/launcher.test -test.testlogfile=/tmp/go-build113898129/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build113898129/b216/vet.cfg 7845551/b412/_pkg_.a 64/src/go/scanner/errors.go ache/go/1.25.8/x64/pkg/tool/linu-nilfunc -p l -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linu-tests -w 738659/b232/_pkg_.a ylov/l0IgnnYtDUbmqPCsylov x_amd64/vet -c -I /tmp/go-build171-bool x_amd64/vet (dns block)
  • slow.example.com
    • Triggering command: /tmp/go-build113898129/b514/launcher.test /tmp/go-build113898129/b514/launcher.test -test.testlogfile=/tmp/go-build113898129/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build113898129/b216/vet.cfg 7845551/b412/_pkg_.a 64/src/go/scanner/errors.go ache/go/1.25.8/x64/pkg/tool/linu-nilfunc -p l -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linu-tests -w 738659/b232/_pkg_.a ylov/l0IgnnYtDUbmqPCsylov x_amd64/vet -c -I /tmp/go-build171-bool x_amd64/vet (dns block)
  • this-host-does-not-exist-12345.com
    • Triggering command: /tmp/go-build113898129/b523/mcp.test /tmp/go-build113898129/b523/mcp.test -test.testlogfile=/tmp/go-build113898129/b523/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build113898129/b257/vet.cfg 7845551/b362/_pkg_.a -trimpath (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Update gateway to accept OTLP headers as string Update OTLP headers to accept string format (spec v1.13.0) Apr 7, 2026
Copilot finished work on behalf of lpcox April 7, 2026 18:07
Copilot AI requested a review from lpcox April 7, 2026 18:07
@lpcox lpcox marked this pull request as ready for review April 7, 2026 18:13
Copilot AI review requested due to automatic review settings April 7, 2026 18:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the gateway’s OpenTelemetry OTLP header configuration to match spec v1.13.0, changing opentelemetry.headers from a key/value object to a comma-separated key=value string and parsing it into a map at export time.

Changes:

  • Change TracingConfig.Headers / StdinOpenTelemetryConfig.Headers from map[string]string to string.
  • Update config variable expansion to expand the headers string as a whole.
  • Add parseOTLPHeaders and route resolveHeaders through it; update schema to reflect headers: string.
Show a summary per file
File Description
internal/tracing/provider.go Adds parsing for comma-separated OTLP headers and uses it when constructing exporter options.
internal/tracing/provider_test.go Updates OTLP header forwarding test to use the new string format.
internal/config/validation.go Updates ${VAR} expansion for headers from per-map-value expansion to whole-string expansion.
internal/config/schema/mcp-gateway-config.schema.json Switches gateway.opentelemetry.headers schema type from object to string and updates description.
internal/config/config_tracing.go Updates TracingConfig.Headers type and example docs to the new string format.
internal/config/config_tracing_test.go Updates tracing config tests to match the new headers type/expansion behavior.
internal/config/config_stdin.go Updates stdin JSON OpenTelemetry config headers type to string.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 7/7 changed files
  • Comments generated: 4

Comment on lines +100 to +113
func parseOTLPHeaders(raw string) map[string]string {
headers := make(map[string]string)
for _, pair := range strings.Split(raw, ",") {
trimmed := strings.TrimSpace(pair)
if trimmed == "" {
continue
}
k, v, ok := strings.Cut(trimmed, "=")
if !ok {
logTracing.Printf("Warning: skipping malformed OTLP header pair (missing '='): %q", trimmed)
continue
}
headers[strings.TrimSpace(k)] = strings.TrimSpace(v)
}
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseOTLPHeaders can produce an entry with an empty header name when the input contains pairs like "=value" or " = value". Passing a map with an empty key to the OTLP HTTP exporter can cause request creation to fail due to an invalid HTTP header field name. After trimming, validate that the header key is non-empty (and ideally a valid HTTP token) before adding it; otherwise log a warning and skip the pair.

Copilot uses AI. Check for mistakes.
}
k, v, ok := strings.Cut(trimmed, "=")
if !ok {
logTracing.Printf("Warning: skipping malformed OTLP header pair (missing '='): %q", trimmed)
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning log for malformed OTLP header pairs includes the raw (trimmed) pair content. If a user accidentally misformats an Authorization header (e.g., missing '='), this could leak secrets into logs. Consider logging without echoing the full pair (or redact/truncate values) to avoid accidental credential exposure.

Suggested change
logTracing.Printf("Warning: skipping malformed OTLP header pair (missing '='): %q", trimmed)
logTracing.Printf("Warning: skipping malformed OTLP header pair (missing '=')")

Copilot uses AI. Check for mistakes.
Comment on lines 333 to 336
cfg := &config.TracingConfig{
Endpoint: ts.URL,
Headers: map[string]string{"Authorization": "Bearer test-token"},
Headers: "Authorization=Bearer test-token",
}
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There’s new parsing behavior for comma-separated OTLP headers (trimming, skipping malformed pairs). provider_test only verifies a single well-formed header is forwarded; adding focused test coverage for multiple headers + whitespace trimming + malformed/empty-key cases would help prevent regressions in parseOTLPHeaders/resolveHeaders.

See below for a potential fix:

	tests := []struct {
		name           string
		headers        string
		expected       map[string]string
		notExpectedSet []string
	}{
		{
			name:    "single well formed header",
			headers: "Authorization=Bearer test-token",
			expected: map[string]string{
				"Authorization": "Bearer test-token",
			},
		},
		{
			name:    "multiple headers with whitespace are trimmed",
			headers: " Authorization = Bearer test-token , X-Request-ID = req-123 ",
			expected: map[string]string{
				"Authorization": "Bearer test-token",
				"X-Request-Id":  "req-123",
			},
		},
		{
			name:    "malformed and empty key headers are skipped",
			headers: "Authorization=Bearer test-token, malformed, =empty-key, X-Trace-ID=trace-123",
			expected: map[string]string{
				"Authorization": "Bearer test-token",
				"X-Trace-Id":    "trace-123",
			},
			notExpectedSet: []string{"", "malformed"},
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			// Channel signals when the test server receives an export request.
			received := make(chan http.Header, 1)
			ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
				headers := r.Header.Clone()
				select {
				case received <- headers:
				default:
				}
				w.WriteHeader(http.StatusOK)
			}))
			defer ts.Close()

			cfg := &config.TracingConfig{
				Endpoint: ts.URL,
				Headers:  tt.headers,
			}

			provider, err := tracing.InitProvider(ctx, cfg)
			require.NoError(t, err)
			require.NotNil(t, provider)

			// Create and end a span to trigger an export attempt.
			tr := provider.Tracer()
			_, span := tr.Start(ctx, "header-test-span")
			span.End()

			// Shutdown flushes the batch processor, ensuring the export is sent.
			shutdownCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
			defer cancel()
			_ = provider.Shutdown(shutdownCtx)

			// Wait for the export request with a timeout.
			select {
			case headers := <-received:
				for key, expectedValue := range tt.expected {
					assert.Equal(t, expectedValue, headers.Get(key),
						fmt.Sprintf("%s header must be forwarded to the OTLP collector", key))
				}
				for _, key := range tt.notExpectedSet {
					assert.Empty(t, headers.Get(key),
						fmt.Sprintf("%s header must be skipped when OTLP headers are malformed", key))
				}
			case <-time.After(3 * time.Second):
				t.Fatal("timed out waiting for OTLP export request — headers test is non-deterministic")
			}
		})

Copilot uses AI. Check for mistakes.
},
"headers": {
"type": "string",
"description": "Comma-separated key=value HTTP headers sent with every OTLP export request (e.g. \"Authorization=Bearer ${OTEL_TOKEN}\"). Supports ${VAR} expansion."
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The schema now documents opentelemetry.headers as a comma-separated key=value string, but it accepts any string. If this schema is used for validation/autocomplete, consider adding a (reasonably permissive) pattern to catch obviously malformed values (e.g., missing '=' between key/value) early, before runtime warnings.

Suggested change
"description": "Comma-separated key=value HTTP headers sent with every OTLP export request (e.g. \"Authorization=Bearer ${OTEL_TOKEN}\"). Supports ${VAR} expansion."
"description": "Comma-separated key=value HTTP headers sent with every OTLP export request (e.g. \"Authorization=Bearer ${OTEL_TOKEN}\"). Supports ${VAR} expansion.",
"pattern": "^(\\$\\{[A-Za-z_][A-Za-z0-9_]*\\}|\\s*[^=\\s,][^=,]*=.*?(\\s*,\\s*[^=\\s,][^=,]*=.*?)*)\\s*$"

Copilot uses AI. Check for mistakes.
@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented Apr 7, 2026

@copilot apply changes based on the comments in this thread

…ecret log, better tests, schema pattern

Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/67af45f2-b80e-4dce-b095-042952b04590

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 7, 2026

@copilot apply changes based on the comments in this thread

Applied all four review suggestions in commit 193f2a1:

  1. Empty-key guard (provider.go): parseOTLPHeaders now checks key == "" after trimming and skips with a warning — covers =value and = value inputs.
  2. No credential leakage in logs: removed %q, trimmed from the "missing '='" warning so raw header content isn't echoed.
  3. Expanded test coverage (parse_headers_test.go + provider_test.go): added a white-box unit test (TestParseOTLPHeaders) covering empty string, multiple pairs, whitespace trimming, = in values, malformed pairs, empty/whitespace-only keys, and trailing commas. TestInitProvider_WithHeaders is now table-driven with single-header, multi-header, and malformed/empty-key sub-tests.
  4. Schema pattern (mcp-gateway-config.schema.json): added the permissive pattern from the suggestion to catch obviously malformed values (missing =) at schema validation time.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /tmp/go-build721400787/b514/launcher.test /tmp/go-build721400787/b514/launcher.test -test.testlogfile=/tmp/go-build721400787/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s -W olang.org/protobuf@v1.36.11/enco/tmp/go-build3069612943/b451/symabis .cfg x_amd64/vet . --gdwarf2 --64 x_amd64/vet -I 8430233/b345/_pk-errorsas -I x_amd64/vet --gdwarf-5 g/grpc/codes -o x_amd64/vet (dns block)
  • invalid-host-that-does-not-exist-12345.com
    • Triggering command: /tmp/go-build721400787/b496/config.test /tmp/go-build721400787/b496/config.test -test.testlogfile=/tmp/go-build721400787/b496/testlog.txt -test.paniconexit0 -test.timeout=10m0s -I ternal/descriptor/table.go -I x_amd64/asm --gdwarf-5 --64 -o x_amd64/asm -I QYlEh5z2z .cfg x_amd64/vet --gdwarf-5 --64 -o x_amd64/vet (dns block)
  • nonexistent.local
    • Triggering command: /tmp/go-build721400787/b514/launcher.test /tmp/go-build721400787/b514/launcher.test -test.testlogfile=/tmp/go-build721400787/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s -W olang.org/protobuf@v1.36.11/enco/tmp/go-build3069612943/b451/symabis .cfg x_amd64/vet . --gdwarf2 --64 x_amd64/vet -I 8430233/b345/_pk-errorsas -I x_amd64/vet --gdwarf-5 g/grpc/codes -o x_amd64/vet (dns block)
  • slow.example.com
    • Triggering command: /tmp/go-build721400787/b514/launcher.test /tmp/go-build721400787/b514/launcher.test -test.testlogfile=/tmp/go-build721400787/b514/testlog.txt -test.paniconexit0 -test.timeout=10m0s -W olang.org/protobuf@v1.36.11/enco/tmp/go-build3069612943/b451/symabis .cfg x_amd64/vet . --gdwarf2 --64 x_amd64/vet -I 8430233/b345/_pk-errorsas -I x_amd64/vet --gdwarf-5 g/grpc/codes -o x_amd64/vet (dns block)
  • this-host-does-not-exist-12345.com
    • Triggering command: /tmp/go-build721400787/b523/mcp.test /tmp/go-build721400787/b523/mcp.test -test.testlogfile=/tmp/go-build721400787/b523/testlog.txt -test.paniconexit0 -test.timeout=10m0s -I @v1.1.3/cpu/cpu.go -fPIC x_amd64/vet -pthread g/grpc/internal//usr/bin/runc -fmessage-length--version x_amd64/vet (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of lpcox April 7, 2026 18:25
@lpcox lpcox merged commit 81c70b4 into main Apr 7, 2026
13 checks passed
@lpcox lpcox deleted the copilot/update-gateway-accept-otlp-headers branch April 7, 2026 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update gateway to accept OTLP headers as string (spec v1.13.0)

3 participants