Proposal to rename distributed context and add support for baggage.

[rghetia]

No description provided.

[reyang]

Suggested change

	- It is more of a convenient functionality to provide compatibility with Open Tracing.
	- It is more of a convenient functionality to provide compatibility with OpenTracing.

[reyang]

It is unclear when to put an item in Baggage versus TagMap.
Also, if something started as an item in TagMap, later there are needs to also put it Baggage, do we move the item or we duplicate the item?

A meta question - do we believe that OpenTelemetry should provide such general purpose Baggage so people can use it for things such as "authentication context"?

[Oberon00]

It is unclear when to put an item in Baggage versus TagMap.

As I read this OTEP, that is quite clear: An application puts a value in the TagMap if it wants to have spans, metrics, etc tagged with that value. The baggage on the other hand is used when the value should not be attached to the telemetry data but if the application itself (or another service invoked by it) wants to read the value back, for application-specific purposes.

[rghetia]

Also, if something started as an item in TagMap, later there are needs to also put it Baggage, do we move the item or we duplicate the item?

No. Two are independent. Yes, there is duplication as mentioned in Trade-offs section but it is not worth to optimize it.

A meta question - do we believe that OpenTelemetry should provide such general purpose Baggage so people can use it for things such as "authentication context"?

I agree that from observability perspective, 'Baggage' doesn't belong in OpenTelemetry. However, from POV of opentracing compatibility we need to provide some solution. It could very well be in another repo/contrib package.

[reyang]

No. Two are independent. Yes, there is duplication as mentioned in Trade-offs section but it is not worth to optimize it.

I see, now it is clear to me. Thanks.

I agree that from observability perspective, 'Baggage' doesn't belong in OpenTelemetry. However, from POV of opentracing compatibility we need to provide some solution. It could very well be in another repo/contrib package.

Makes sense. If this for compatibility, do we encourage more people to adopt it, or we suggest people to move away from it?

[yurishkuro]

Baggage is just context propagation. Context propagation is not a part of OpenTelemetry, it's the layer on top of which OpenTelemetry is built.

[toumorokoshi]

I would vote for not including a non-telemetry related concept directly into opentelemetry. If Baggage is needed for opentracing having it be in contrib makes sense.

Will SaaS vendors need to use this as well? authentication or identification seems like a common concept.

[songy23]

Please also rename DistributedContext.Entry to Tag, key and value to TagKey, TagValue.

[rghetia]

sure. My intention is to restore TagMap as defined in OC

[yurishkuro]

I am very supportive of replacing "distributed context" with "baggage".

I am not supportive of revisiting the already made decision that "tags" are a bad name for telemetry-specific baggage. In OpenTracing tags had a completely different meaning.

I am also not convinced that we need two different sets of baggage. First, it raises the questions "what if I want key X either here or there or both places, why should this mean changing how the whole key/value pair is even named". I also don't see why telemetry dimensions cannot include application baggage - that's just a matter of configuring the filters/extractors for dimensions. With this proposed separation I need one filter type for application baggage and another type for telemetry baggage - highly confusing and unnecessary. Finally, the only rationale for the separation seems to be different guarantees of propagation: never drop app baggage, could drop telemetry baggage. This is a concern orthogonal to what what the baggage key/value pair is named. It can be addressed with an extra attribute, similar to TTL, like "criticality" or "priority", instead of two separately named concepts. The priority can be a numeric value that ranks baggage items (dups are ok), so that if something needs to be truncated, we truncate items with lower priority.

[tedsuo]

Based on comments, some additional clarity about the reasoning for separating these two interfaces might be in order:

TagMap data is to be used as labels for metrics and traces. This can quickly add overhead when propagated in-band. But, because this data is write-only, how this information is transmitted remains undefined. This leaves the door open to future implementations, which do not propagate this information in-band. Instead, the labels are transmitted out-of-band and reassembled on the back-end.

Baggage data, on the other hand, is explicitly added in order to be accessed by downstream application code. Baggage must be propagated in-band in order to accomplish this goal.

For these reasons, Baggage and TagMap are totally separate devices. The Baggage interface describes a distributed context mechanism for application developers, while the TagMap interface describes a labeling mechanism for observability which hides how it is propagated.

Honestly, if we really want a clean separation between context propagation and observability, we should move these to interfaces into separate packages. They really don't have anything to do with each other, except that their implementations might both rely on Propagators under the hood.

As far as @yurishkuro's concerns about reusing the "Tag" name... how do we avoid another huge naming bikeshed? Can we call them Labels instead? I don't feel as concerned about the Tag confusion issue as much as I used to - I feel more confident that users will get that this is a different thing. Can we flip a coin? :)

[tedsuo]

Working on a context prop RFC (will be out today). In it, I'm experimenting with changing the name of TagMap to Correlations. It feels better, for a couple reasons:

• "Correlations" has clearer semantic meaning than "Tags." Since this is now a separate concept from Baggage, we can be more specific about the intent.
• The W3C propagation format will most likely be called Correlation-Context. So there is better naming alignment.
• Doesn't confuse OpenTracing users with an overloaded term.

@yurishkuro @rghetia how does that sound to you?

[yurishkuro]

Correlations definitely better than TagMap.

Only concern is how it will correspond to W3C Correlation-Context, it was not clear if it was intended as more general baggage rather than telemetry baggage.

[rghetia]

Correlations sounds better than TagMap. So there will Correlations (for Telemetry) and baggage (for non-Telemetry)?

[toumorokoshi]

how is "non-observability" defined? I would imagine here that if Baggage refers to the key-values pairs that carry with the trace, then those would contribute to observability since they help define the context of the request.

[rghetia]

non-observability should be defined as data that needs to propagate downstream along with the request for usage specific to application, testing, etc. It is not tied to whether the request is being traced/monitored or not.

[toumorokoshi]

Got it! might be good to add that definition somewhere, unless there is a definition floating in the spec or other document.

[toumorokoshi]

Is write-only desirable? I figured distributedcontext entries would be something I could pipe into metrics as I see fit.

For example, if I used this to propagate my originating service, There's cases I would want metrics to also have the originating service, as I may want to split my metric by that dimension.

[rghetia]

SDK can read it so it can be applied to metrics.

[toumorokoshi]

When you say the sdk, are you referring to the exporters?
In the case of metrics, I think a lot of the dimensions will be specified and passed in during the invocation (e.g. during the creation of a TimeSeries (currently terminology) or Handler (new RFC-based terminology)). In that world, DistributedContext would need to be readable by the user:

# create the main counter
request_count = Counter()

# in the request, get the relevant time series and add the value.
time_series = request_count.get_time_series({"service": DistributedContextSingleton.get("service"))
time_series.add(1)

Although it is possible for the exporter to have access to the values, there's no easy mechanism to tell the exporter which labels matter.

This behavior is also valuable because metrics will pre-aggregate, so they need values to determine the aggregation before the measurement even reaches the exporter.

[toumorokoshi]

I feel like there should be two RFCs: one to add explicit support for baggage and one for the renaming of DistributedContext. For me the renaming of DistributedContext seems like a good idea, and I'm not sure about baggage support.

(Just my two cents) I'm not seeing why "TagMap" is clearer than DistributedContext in terms of it's function to:

1. store several key-value pairs
2. have the context be propagated across service

TagMap certainly makes me think it has support for 1. The fact that it's propagated across process boundaries I would probably have to read somewhere. I'd say DistributedContext is clearer in that regard.

Also I don't have strong opinions on naming but I think we should have consistent semantic prefix or suffix to help the consumer understand that I can get key-value pairs for my telemetry from them. I have an issue out to try to rename "Resources" to "StaticContext", as an attempt to make it clear to the user that anything with "Context" contains key-value pairs that you can access: open-telemetry/opentelemetry-specification#239

But I don't think Context does a great job of summarizing that behavior either.

[rghetia]

I feel like there should be two RFCs: one to add explicit support for baggage and one for the renaming of DistributedContext.

SG. If we get the agreement that telemetry and non-telemetry should be separate then I can split this into two RFCs.

[rghetia]

Is it fair to say that we only need an issue to track renaming Distributed context to Correlation?
For Baggage, we may need another RFC or simply an issue after concluding on #42.

[bogdandrutu]

I am going to close this in favor of #66