[EPIC] v0.19.0 — Observability Exporters & Monitoring

[dlrivada]

Objective

Implement OpenTelemetry exporters for all major observability backends (Azure Monitor, AWS X-Ray, Prometheus, Jaeger, Datadog, Sentry, Zipkin, New Relic, Raygun, Rollbar), deliver pre-built Grafana dashboard templates, add ODBC provider support for legacy database connectivity, and establish service discovery abstractions with schema registry integration.

Motivation

Encina's OpenTelemetry core is established, but teams cannot yet ship telemetry to their chosen backends without manual wiring. Cloud teams need first-class Azure Monitor and AWS X-Ray integration; platform teams need Prometheus scraping and Grafana dashboards out of the box; product teams need Datadog and Sentry for APM and error tracking. The ODBC provider fills the gap for organizations with legacy databases (IBM DB2, Firebird, Informix) that cannot migrate to supported providers. Service discovery and schema registry abstractions complete the cloud-native infrastructure story.

Scope

In Scope

• Foundational OTel core wiring: WithLogging block in WithEncina ([FEATURE] Wire WithLogging in Encina.OpenTelemetry.WithEncina (prerequisite for OTLP logs) #1048) — prerequisite for OTLP logs
• OTel exporters: Azure Monitor ([FEATURE] Observability: Create Encina.OpenTelemetry.AzureMonitor Package #178), AWS X-Ray ([FEATURE] Observability: Create Encina.OpenTelemetry.AwsXRay Package #179), Prometheus ([FEATURE] Observability: Create Encina.OpenTelemetry.Prometheus Package #180), Serilog bridge ([FEATURE] Observability: Create Encina.Serilog.OpenTelemetry Package #182), sampling behaviors ([FEATURE] Observability: Implement Sampling Behaviors for Cost Optimization #183), Jaeger ([FEATURE] Encina.OpenTelemetry.Jaeger - Jaeger Distributed Tracing Integration #613), Datadog ([FEATURE] Encina.OpenTelemetry.Datadog - Datadog APM Integration #615), Zipkin ([FEATURE] Encina.OpenTelemetry.Zipkin - Zipkin Distributed Tracing Integration #614), New Relic ([FEATURE] Encina.OpenTelemetry.NewRelic - New Relic Observability Integration #616), Sentry ([FEATURE] Encina.Sentry - Sentry Error Monitoring Integration #617), Raygun ([FEATURE] Encina.Raygun - Raygun Error Monitoring Integration #618), Rollbar ([FEATURE] Encina.Rollbar - Rollbar Error Monitoring Integration #619)
• OTLP exporter (canonical wire protocol): [FEATURE] Wire OTLP exporter as opt-in option in Encina.OpenTelemetry #1043 — opt-in EnableOtlpExporter flag for traces, metrics, and logs
• Observability hardening: OTLP collector health check ([FEATURE] OTLP collector reachability health check (Encina.OpenTelemetry) #1049), EventId range registration ([DEBT] Register Encina.OpenTelemetry EventId range 7000-7099 in EventIdRanges.cs #1050), CPM transitive pinning spike ([SPIKE] Re-evaluate CentralPackageTransitivePinningEnabled to retire WireMock PrivateAssets="all" workaround #1051)
• Pre-built Grafana dashboard templates ([FEATURE] Observability: Create Pre-built Grafana Dashboard Templates #187)
• ODBC provider for legacy databases ([FEATURE] ODBC Provider - legacy database support #56)
• Service discovery abstraction ([FEATURE] IServiceDiscoveryProvider: Abstraction and Implementations (Consul, K8s, Eureka, Aspire) #607) — Consul, Kubernetes, Eureka, Aspire
• Schema registry abstraction ([FEATURE] ISchemaRegistryProvider: Abstraction and Implementations (Confluent, Azure, Apicurio) #602) — Confluent, Azure, Apicurio
• Supporting: request batching ([FEATURE] Request Batching / Bulk Operations #349), batch command processing ([FEATURE] Batch Command Processing #339), partitioned sequential messaging ([FEATURE] Partitioned Sequential Messaging #469)
• AzureFunctions Durable Functions integration tests ([DEBT] AzureFunctions integration tests for Durable Functions extensions #523)
• Aspire POC test disposition ([TEST] Decide disposition of Encina.Aspire.POC.Tests project #517)
• Audit log store for Cosmos DB ([FEATURE] IAuditLogStore for Azure Cosmos DB #582)

Out of Scope

• Grafana Loki (log aggregation) — separate initiative
• Custom OTel collectors/agents
• APM SDK integrations beyond OTel exporters (e.g., Datadog APM tracer)
• Event collaboration/process manager pattern ([FEATURE] Event Collaboration / Process Manager Pattern #392) — belongs in v0.18.1 messaging scope

Child Issues (Ordered by Priority → Dependencies → Complexity)

🔴 Critical

Order	Issue	Title	Priority	Complexity	Dependencies
1	#1048	[FEATURE] Wire WithLogging in Encina.OpenTelemetry.WithEncina (prerequisite for OTLP logs)	Medium	Low	None — foundational
2	#178	Observability: Create Encina.OpenTelemetry.AzureMonitor Package	Medium	Low	v0.18.2 OTel LLM
3	#179	Observability: Create Encina.OpenTelemetry.AwsXRay Package	Medium	Low	v0.18.2 OTel LLM
4	#180	Observability: Create Encina.OpenTelemetry.Prometheus Package	Medium	Medium	None
5	#187	Observability: Create Pre-built Grafana Dashboard Templates	Medium	Medium	#180, #178
6	#613	Encina.OpenTelemetry.Jaeger - Jaeger Distributed Tracing Integration	Medium	Low	None

🟠 High Priority

Order	Issue	Title	Priority	Complexity	Dependencies
7	#1043	[FEATURE] Wire OTLP exporter as opt-in option in Encina.OpenTelemetry	Low	Medium	#1048 (logs wiring), follow-up to #1041 (security bump)
8	#615	Encina.OpenTelemetry.Datadog - Datadog APM Integration	Medium	Medium	None
9	#617	Encina.Sentry - Sentry Error Monitoring Integration	Medium	Medium	None
10	#182	Observability: Create Encina.Serilog.OpenTelemetry Package	Medium	Medium	None
11	#183	Observability: Implement Sampling Behaviors for Cost Optimization	Medium	Medium	#178, #179
12	#607	IServiceDiscoveryProvider: Abstraction and Implementations (Consul, K8s, Eureka, Aspire)	Medium	High	None
13	#56	ODBC Provider - legacy database support	Low	Medium	None

🟡 Medium Priority

Order	Issue	Title	Priority	Complexity	Dependencies
14	#1049	[FEATURE] OTLP collector reachability health check (Encina.OpenTelemetry)	Low	Low	#1043 (OTLP wiring)
15	#614	Encina.OpenTelemetry.Zipkin - Zipkin Distributed Tracing Integration	Low	Low	None
16	#602	ISchemaRegistryProvider: Abstraction and Implementations (Confluent, Azure, Apicurio)	Medium	Medium	None
17	#582	IAuditLogStore for Azure Cosmos DB	Medium	Medium	None
18	#349	Request Batching / Bulk Operations	Medium	Medium	None
19	#339	Batch Command Processing	Medium	Medium	#349
20	#469	Partitioned Sequential Messaging	Medium	High	None

🟢 Low Priority

Order	Issue	Title	Priority	Complexity	Dependencies
21	#616	Encina.OpenTelemetry.NewRelic - New Relic Observability Integration	Low	Medium	None
22	#618	Encina.Raygun - Raygun Error Monitoring Integration	Low	Medium	None
23	#619	Encina.Rollbar - Rollbar Error Monitoring Integration	Low	Medium	None
24	#523	AzureFunctions integration tests for Durable Functions extensions	Low	Medium	v0.18.2
25	#517	Decide disposition of Encina.Aspire.POC.Tests project	Low	Low	None
26	#1050	[DEBT] Register Encina.OpenTelemetry EventId range 7000-7099 in EventIdRanges.cs	Low	Low	None — small cleanup
27	#1051	[SPIKE] Re-evaluate CentralPackageTransitivePinningEnabled to retire WireMock workaround	Low	Medium	None — investigation; may spawn [INFRA] follow-up
28	#392	Event Collaboration / Process Manager Pattern	Medium	Medium	v0.18.1

Milestone Dependencies

• Depends on: v0.18.2 (LLM observability metrics from [FEATURE] LLM Observability Integration #486 define the OTel semantic conventions extended by cloud exporters here)
• Blocks: v0.20.0 (full observability stack must be complete before enterprise compliance and audit trail features)

Acceptance Criteria

• WithLogging wired into WithEncina ([FEATURE] Wire WithLogging in Encina.OpenTelemetry.WithEncina (prerequisite for OTLP logs) #1048), unblocking OTLP logs export
• OTLP exporter ([FEATURE] Wire OTLP exporter as opt-in option in Encina.OpenTelemetry #1043) opt-in for all three signals (traces, metrics, logs) once [FEATURE] Wire WithLogging in Encina.OpenTelemetry.WithEncina (prerequisite for OTLP logs) #1048 lands
• OTLP collector health check ([FEATURE] OTLP collector reachability health check (Encina.OpenTelemetry) #1049) opt-in via EncinaOpenTelemetryOptions.AddOtlpHealthCheck
• EventId range 7000-7099 registered in EventIdRanges.cs ([DEBT] Register Encina.OpenTelemetry EventId range 7000-7099 in EventIdRanges.cs #1050) so architecture tests enforce uniqueness
• CPM transitive pinning decision documented ([SPIKE] Re-evaluate CentralPackageTransitivePinningEnabled to retire WireMock PrivateAssets="all" workaround #1051) — either enabled globally or workaround formalized
• All cloud OTel exporters (Azure Monitor, AWS X-Ray, Prometheus, Jaeger, Datadog, Sentry) pass integration tests against real endpoints or emulators
• Grafana dashboard templates display Encina metrics (outbox lag, lock contention, request latency) out of the box
• ODBC provider implements all 13-provider-equivalent interfaces (IOutboxStore, IInboxStore, etc.)
• Service discovery abstraction resolves service endpoints via at minimum Consul and Kubernetes
• Schema registry validates event schemas on publish and consume
• Sampling behaviors reduce OTel overhead by ≥50% under configured rules
• All packages have zero build warnings and complete PublicAPI.Unshipped.txt entries
• Partitioned sequential messaging guarantees ordering within partition key