Unable to include trace_id in logs.

[darrelldraney]

Using router version 1.37.0 with the below configuration to include trace_id in logs for correlation with traces returns an error.

telemetry:
  exporters:
    logging:
      stdout:
        enabled: true
  instrumentation:
    spans:
      default_attribute_requirement_level: recommended
      mode: spec_compliant
      router:
        attributes:
          trace_id: true

Error on start up is:

{
    "timestamp": "2024-01-15T23:25:35.295622820Z",
    "level": "ERROR",
    "message": "Not connected to GraphOS. In order to enable these features for a self-hosted instance of Apollo Router, the Router must be connected to a graph in GraphOS (using APOLLO_KEY and APOLLO_GRAPH_REF) that provides a license for the following features:\n\nConfiguration yaml:\n* Advanced telemetry\n  .telemetry..spans.router\n\nSee https://go.apollo.dev/o/elp for more information.",
    "target": "apollo_router::state_machine",
    "resource":   {}
}

This seems to indicate that including the trace_id in logs requires a license. The documentation seems to indicate custom attributes requires a license:

Custom attributes for spans via selector is an Enterprise Feature that requires a GraphOS Enterprise plan.

Is this a bug, or Is trace_id considered a custom attribute, and there is no way to correlate logs with traces without a license?

[BrynCooke]

Any customization of attributes on spans is considered licensed functionality at this time.

Telemetry is a huge part of the Router, and has had both a significant amount of effort spent on it, but also needs further investment.

By making telemetry customization a commercial feature, we can demonstrate the need to allocate more resources to this area.

We will leave this ticket open for now to track demand from free users.

[kindermax]

Hi, I also wanted to add custom header's x-requests-id: uuid4 value to logs but as you said it is a paid feature. I tried to use standard trace_id functionality but it does not accept uuid4 values.

Talking about this one:

if let Some(from_request_header) = &propagation.request.header_name {
            propagators.push(Box::new(CustomTraceIdPropagator::new(
                from_request_header.to_string(),
            )));
        }

Being able to just pass my custom header value as a json logs field would be just awesome!

[smyrick]

Linking issues here, this is also related and could be a general solution: #5212