Integrate OTEP-205 (messaging context propagation requirements) into the spec

[joaopgrassi]

What are you trying to achieve?

Integrate the OTEP-205 Context propagation requirements for messaging semantic conventions into the spec.

Additional context.

As stated in the OTEP, the existing messaging semantic conventions for tracing define trace examples that implicitly require the context to be propagated via the message. The OTEP's goal was to make the context propagation requirement clear and explicit so instrumentation authors have more guidance when working on messaging instrumentation.

[carlosalberto]

Re-assigning to @jsuereth as he may be interested. Let's discuss in case this needs to be reassigned to somebody else.

[joaopgrassi]

@rbailey7210 Hi! Could you please let me know what needs more information here? The OTEP already has an extensive explanation on the motivation so I'm a bit confused.

[reyang]

@joaopgrassi this issues needs more discussion, currently there is no good evidence indicating whether we should accept or reject. Please collect more feedback, and if you are stuck, follow https://github.com/open-telemetry/opentelemetry-specification/blob/main/issue-management.md#when-issues-get-stuck.

[joaopgrassi]

Hi @reyang !

I read that and still not sure though how I would promote more discussion here. The OTEP where this work is coming from contains context, rational and discussions. I can join the next spec call and bring the topic there, if that's preferred.

Nevertheless, I'll add the summary here: The current messaging semantic conventions have example diagrams and show how a trace looks like for different messaging scenarios. Neither the document nor the example diagrams talk about how exactly the context is to be propagated in such cases. Instrumentation authors are open to figure out on their own

In messaging systems, it's common that a message goes through many hops before it reaches its "final destination". The "traditional" context propagation mechanisms we have today (traceparent via headers) is not enough to cover such cases. If such mechanism is used for messaging systems, you end up with two traces: one for the producer > intermediary and another from intermediary > consumer that can't be connected to each other. This of course makes identifying problems hard, as you don't know which component received a message and can't correlate both consumer and producer traces.

What is implicit then in the current messaging semantic conventions spec is that the message itself is the mechanism which is used to propagate context. Consider one of the example in the current document

graph TD
    Producer -->|Publish message| C{Intermediary}
    C -->|Pull| D[Consumer1]
    C -->|Pull| E[Consumer2]

Loading

This would produce traces like this, using the "traditional" context propagation mechanisms

+---------------------------------------------------+
|     Producer trace                                |
|                                                   |
|     +---------------------------------------+     |
|     |  Publish message                      |     |
|     +---------------------------------------+     |
+---------------------------------------------------+
+---------------------------------------------------+
|     Consumer1 trace                               |
|                                                   |
|     +--------------------+                        |
|     |  Receive message   |                        |
|     +--------------------+                        |
|                           +--------------------+  |
|                           | Process message    |  |
|                           +--------------------+  |
+---------------------------------------------------+

So, two independent traces, no way to correlate them. With the OTEP, this issue and the linked PR we are trying to make the propagation via message explicit in the conventions. Then we could achieve a trace like this using links (not the goal of this issue)

+---------------------------------------------------------------------+
|                Producer trace                                       |
|                                                                     |
|                +---------------------------------------+            |
|      +---------|  Publish message                      |            |
|      |         +---------------------------------------+            |
|      |                                                              |
|    L |                                                              |
|    I |   +---------------------------------------------------+      |
|    N |   |     Consumer1 trace                               |      |
|    K |   |                                                   |      |
|      |   |     +--------------------+                        |      |
|      +---------|  Receive message   |                        |      |
|          |     +--------------------+                        |      |
|          |                           +--------------------+  |      |
|          |                           | Process message    |  |      |
|          |                           +--------------------+  |      |
|          |                                                   |      |
|          +---------------------------------------------------+      |
+---------------------------------------------------------------------+

[reyang]

@joaopgrassi I think there are fundamental problems we need to address regarding #2753, with the dependency and current capacity of the TC, I'm not sure if this will be treated as a high priority.

[joaopgrassi]

@reyang @rbailey7210 giving the latest guidance in #2753, I think this PR is fine to go through. It doesn't touch on any of the topics there (metrics, resouce, etc).