Randomness requirements following W3C Trace Context level 2

CHANGELOG.md

@@ -11,8 +11,14 @@ release.
 
 ### Traces
 
+- Remove restriction that sampler description is immutable.
+  ([#4137](https://github.com/open-telemetry/opentelemetry-specification/pull/4137))
+- Add in-development `OnEnding` callback to SDK `SpanProcessor` interface.
+  ([#4024](https://github.com/open-telemetry/opentelemetry-specification/pull/4024))
 - Minor clarification on BatchExportingProcessor behavior.
   ([#4164](https://github.com/open-telemetry/opentelemetry-specification/pull/4164))
+- Define randomness value requirements for W3C Trace Context Level 2.
+  ([#4162](https://github.com/open-telemetry/opentelemetry-specification/pull/4162))
 
 ### Metrics
 

spec-compliance-matrix.md

@@ -87,6 +87,8 @@ formats is required. Implementing more than one format is optional.
 | [Built-in `SpanProcessor`s implement `ForceFlush` spec](specification/trace/sdk.md#forceflush-1) |          |     | +    |     | +      | +    | +      | +   | +    | +   | +    |       |
 | [Attribute Limits](specification/common/README.md#attribute-limits)                              | X        |     | +    |     | +      | +    | +      | +   |      |     |      |       |
 | Fetch InstrumentationScope from ReadableSpan                                                     |          |     | +    |     | +      |      |        | +   |      |     |      |       |
+| TraceID generator implements W3C Trace Context Level 2 randomness                                |          |     |      |     |        |      |        |     |      |     |      |       |
+| OpenTelemetry explicit randomness supported                                                      | X        |     |      |     |        |      |        |     |      |     |      |       |
 
 ## Baggage
 

specification/context/api-propagators.md

@@ -31,6 +31,7 @@
   * [Get Global Propagator](#get-global-propagator)
   * [Set Global Propagator](#set-global-propagator)
 - [Propagators Distribution](#propagators-distribution)
+  * [W3C Trace Context Requirements](#w3c-trace-context-requirements)
   * [B3 Requirements](#b3-requirements)
     + [B3 Extract](#b3-extract)
     + [B3 Inject](#b3-inject)
@@ -355,6 +356,17 @@ Additional `Propagator`s implementing vendor-specific protocols such as AWS
 X-Ray trace header protocol MUST NOT be maintained or distributed as part of
 the Core OpenTelemetry repositories.
 
+### W3C Trace Context Requirements
+
+A W3C Trace Context propagator MUST parse and set the `traceparent` and `tracestate` HTTP headers as specified in [W3C Trace Context Level 2](https://www.w3.org/TR/trace-context-2/).
+
+When injecting and extracting trace context to or from a carrier, the following fields from the `SpanContext` are propagated.
+
+- TraceID (16 bytes)
+- SpanID (8 bytes)
+- TraceFlags (8 bits)
+- TraceState (string)
+
 ### B3 Requirements
 
 B3 has both single and multi-header encodings. It also has semantics that do not

specification/trace/api.md

@@ -244,16 +244,17 @@ non-zero byte.
 `SpanId` A valid span identifier is an 8-byte array with at least one non-zero
 byte.
 
-`TraceFlags` contain details about the trace. Unlike TraceState values,
-TraceFlags are present in all traces. The current version of the specification
-only supports a single flag called [sampled](https://www.w3.org/TR/trace-context/#sampled-flag).
-
-`TraceState` carries vendor-specific trace identification data, represented as a list
-of key-value pairs. TraceState allows multiple tracing
-systems to participate in the same trace. It is fully described in the [W3C Trace Context
-specification](https://www.w3.org/TR/trace-context/#tracestate-header). For
-specific OTel values in `TraceState`, see the [TraceState Handling](tracestate-handling.md)
-document.
+`TraceFlags` contain details about the trace.
+Unlike TraceState values, TraceFlags are present in all traces.
+The current version of the specification supports two flags:
+
+- [Sampled](https://www.w3.org/TR/trace-context-2/#sampled-flag)
+- [Random](https://www.w3.org/TR/trace-context-2/#random-trace-id-flag)
+
+`TraceState` carries vendor-specific trace identification data, represented as a list of key-value pairs.
+TraceState allows multiple tracing systems to participate in the same trace.
+It is fully described in the [W3C Trace Context specification](https://www.w3.org/TR/trace-context-2/#tracestate-header).
+For specific OpenTelemetry values in `TraceState`, see the [TraceState Handling](tracestate-handling.md) document.
 
 `IsRemote`, a boolean indicating whether the SpanContext was received from somewhere
 else or locally generated, see [IsRemote](#isremote).

specification/trace/sdk.md

@@ -23,6 +23,7 @@ linkTitle: SDK
 - [Sampling](#sampling)
   * [Recording Sampled reaction table](#recording-sampled-reaction-table)
   * [SDK Span creation](#sdk-span-creation)
+    + [Span flags](#span-flags)
   * [Sampler](#sampler)
     + [ShouldSample](#shouldsample)
     + [GetDescription](#getdescription)
@@ -33,6 +34,13 @@ linkTitle: SDK
       - [Requirements for `TraceIdRatioBased` sampler algorithm](#requirements-for-traceidratiobased-sampler-algorithm)
     + [ParentBased](#parentbased)
     + [JaegerRemoteSampler](#jaegerremotesampler)
+  * [Sampling Requirements](#sampling-requirements)
+    + [TraceID randomness](#traceid-randomness)
+    + [Random trace flag](#random-trace-flag)
+    + [Explicit trace randomness](#explicit-trace-randomness)
+    + [Presumption of TraceID randomness](#presumption-of-traceid-randomness)
+    + [User-defined explicit trace randomness](#user-defined-explicit-trace-randomness)
+    + [IdGenerator randomness](#idgenerator-randomness)
 - [Span Limits](#span-limits)
 - [Id Generators](#id-generators)
 - [Span processor](#span-processor)
@@ -265,7 +273,7 @@ The OpenTelemetry API has two properties responsible for the data collection:
   receive them unless the `Sampled` flag was also set.
 * `Sampled` flag in `TraceFlags` on `SpanContext`. This flag is propagated via
   the `SpanContext` to child Spans. For more details see the [W3C Trace Context
-  specification](https://www.w3.org/TR/trace-context/#sampled-flag). This flag indicates that the `Span` has been
+  specification][W3CCONTEXTSAMPLEDFLAG]. This flag indicates that the `Span` has been
   `sampled` and will be exported. [Span Exporters](#span-exporter) MUST
   receive those spans which have `Sampled` flag set to true and they SHOULD NOT receive the ones
   that do not.
@@ -316,6 +324,14 @@ When asked to create a Span, the SDK MUST act as if doing the following in order
    `Span` is created without an SDK installed or as described in
    [wrapping a SpanContext in a Span](api.md#wrapping-a-spancontext-in-a-span).
 
+#### Span flags
+
+The OTLP representation for Span and Span Link include a 32-bit field declared as Span Flags.
+
+Bits 0-7 of the Span Flags field are reserved for the 8 bits of Trace Context flags,
+specified in the [W3C Trace Context Level 2][W3CCONTEXTMAIN] Candidate Recommendation.
+[See the list of recognized flags](./api.md#spancontext).
+
 ### Sampler
 
 `Sampler` interface allows users to create custom samplers which will return a
@@ -466,6 +482,51 @@ The following configuration properties should be available when creating the sam
 [jaeger-remote-sampling-api]: https://www.jaegertracing.io/docs/1.41/apis/#remote-sampling-configuration-stable
 [jaeger-adaptive-sampling]: https://www.jaegertracing.io/docs/1.41/sampling/#adaptive-sampling
 
+### Sampling Requirements
+
+The [W3C Trace Context Level 2][W3CCONTEXTMAIN] Candidate Recommendation includes [a Random trace flag][W3CCONTEXTRANDOMFLAG] for indicating that the TraceID contains 56 random bits, specified for statistical purposes.
+This flag indicates that [the least-significant ("rightmost") 7 bytes or 56 bits of the TraceID are random][W3CCONTEXTTRACEID].
+
+Note the Random flag does not propagate through [Trace Context Level 1][W3CCONTEXTLEVEL1] implementations, which do not recognize the flag.
+Therefore, this flag is considered meaningful only when it is set on a root span context.
+To enable sampling in this and other situations where TraceIDs lack sufficient randomness,
+OpenTelemetry defines an optional [explicit randomness value][OTELRVALUE] encoded in the [W3C TraceState field][W3CCONTEXTTRACESTATE].
+
+This specification recommends the use of either TraceID randomness or explicit trace randomness,
+which ensures that samplers always have sufficient randomness when using W3C Trace Context propagation.
+
+[W3CCONTEXTMAIN]: https://www.w3.org/TR/trace-context-2
+[W3CCONTEXTLEVEL1]: https://www.w3.org/TR/trace-context
+[W3CCONTEXTTRACEID]: https://www.w3.org/TR/trace-context-2/#randomness-of-trace-id
+[W3CCONTEXTTRACESTATE]: https://www.w3.org/TR/trace-context-2/#tracestate-header
+[W3CCONTEXTSAMPLEDFLAG]: https://www.w3.org/TR/trace-context-2/#sampled-flag
+[W3CCONTEXTRANDOMFLAG]: https://www.w3.org/TR/trace-context-2/#random-trace-id-flag
+[OTELRVALUE]: ./tracestate-handling.md#explicit-randomness-value-rv
+
+#### TraceID randomness
+
+For root span contexts, the SDK SHOULD implement the TraceID randomness requirements of the [W3C Trace Context Level 2][W3CCONTEXTTRACEID] Candidate Recommendation when generating TraceID values.
+
+#### Random trace flag
+
+For root span contexts, the SDK SHOULD set the `Random` flag in the trace flags when it generates TraceIDs that meet the [W3C Trace Context Level 2 randomness requirements][W3CCONTEXTTRACEID].
+
+#### Explicit trace randomness
+
+The SDK MAY support an optional explicit trace randomness feature.  For root span contexts, when this option is configured, the SDK inserts explicit randomness into the [`rv` sub-key of the OpenTelemetry TraceState][OTELRVALUE].
+
+#### Presumption of TraceID randomness
+
+For all span contexts, OpenTelemetry samplers SHOULD presume that TraceIDs meet the W3C Trace Context Level 2 randomness requirements, unless an explicit randomness value is present in the [`rv` sub-key of the OpenTelemetry TraceState][OTELRVALUE].
+
+#### User-defined explicit trace randomness
+
+Trace SDKs MAY permit users to setup explicit randomness by entering it into the [`rv` sub-key of the OpenTelemetry TraceState][OTELRVALUE] of the context before creating a root span.  This lets users have consistent sampling across traces.
+
+#### IdGenerator randomness
+
+If the SDK uses an `IdGenerator` extension point, the SDK SHOULD allow the extension to indicate the random trace flag for root span contexts.  When an `IdGenerator` instance does not meet the randomness requirements, users can configure explicit randomness instead, otherwise samplers may incorrectly presume TraceID randomness.
+
 ## Span Limits
 
 Span attributes MUST adhere to the [common rules of attribute limits](../common/README.md#attribute-limits).

specification/trace/tracestate-handling.md

@@ -83,3 +83,27 @@ if ok {
   // traceState was not updated.
 }
 ```
+
+## Pre-defined OpenTelemetry sub-keys
+
+The following values have been defined by OpenTelemetry.
+
+### Explicit randomness value `rv`
+
+The OpenTelemetry TraceState `rv` sub-key defines an alternative source of randomness to the use of TraceID randomness when used by samplers decisions.
+Values of `rv` MUST be exactly 14 lower-case hexadecimal digits:
+
+```
+hexdigit = DIGIT ; a-f
+```
+
+The explicit randomness value is meant to be used instead of extracting randomness from TraceIDs, therefore it contains the same number of bits as a W3C Trace Context Level 2 recommends for TraceIDs.
+
+Explicit randomness values are meant to propagate through [span contexts](../context/README.md) unmodified.
+Explicit randomness values SHOULD NOT be erased from the OpenTelemetry TraceState or modified once associated with a new TraceID, so that sampling decisions made using the explicit randomness value are consistent across signals.
+
+For example, here is a W3C TraceState value including an OpenTelemetry explicit randomness value:
+
+```
+tracestate: ot=rv:6e6d1a75832a2f
+```