Expected behavior of explicitly disabling trace sampling

[nicktrav]

I'm looking for some clarity on what should happen when a remote client indicates that a trace should not be sampled.

Specifically, consider the following example, and how it play out in the current Java implementation:

• A remote client generates a span, but the Sampler returns false

• On the local server, the tracing client sees that there is a remote span, and calls SpanBuilderImpl#startSpanInternal

• In #makeSamplingDecision, assuming no Sampler has been explicitly set on the tracer, the second if-block will run (as there is a remote parent)

• The sampler for the active tracer runs and could return true

• The current span, and all subsequent child spans (local and remote) are sampled

Therefore, it is possible to explicitly request that a trace not be sampled, yet there is a possibility that the span, and all child spans, will be sampled. This manifests as a partial trace being recorded in the tracing backend, which can be confusing to look at for someone expecting the entire trace.

Reading the opencensus spec, I see the following:

If the span is a child of a remote Span the sampling decision will be:
- If a "span-scoped" Sampler is provided, use it to determine the sampling decision.
- Else use the global default Sampler to determine the sampling decision.

While it's clear that global default Sampler will be used, I'd like to understand if it's intentional, and desirable, that a trace that is explicitly set to not sampled would potentially have child spans that are sampled and exported.

[nicktrav]

For some more context, our workaround for this is to define a custom default sampler that will honor the sampling decision of the parent. Writing this sampler made us wonder why this wasn't the default behavior.