The weather service is coded to log its instrumentation startup:
logger.info("Setting up OpenTelemetry observability")
logger.info(f" Service: {service_name}")However, those logs to not appear in kubectl logs ...
When it receives an A2A query from Kagenti, the weather service logs
DEBUG:opentelemetry.instrumentation.openai.shared:OpenLLMetry failed to trace in _set_response_attributes, error: Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/opentelemetry/instrumentation/openai/utils.py", line 147, in sync_wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/opentelemetry/instrumentation/openai/shared/__init__.py", line 232, in _set_response_attributes
prompt_tokens_details = dict(usage.get("prompt_tokens_details", {}))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable
This error is unexpected. It has no stack trace and is thus hard to troubleshoot.
Perhaps related to the above, in Phoenix the root spans show 0 cumulative tokens, but if a root span is expanded its inner ChatOpenAI span shows token usage.
cc @Ladas
The weather service is coded to log its instrumentation startup:
However, those logs to not appear in
kubectl logs ...When it receives an A2A query from Kagenti, the weather service logs
This error is unexpected. It has no stack trace and is thus hard to troubleshoot.
Perhaps related to the above, in Phoenix the root spans show 0 cumulative tokens, but if a root span is expanded its inner ChatOpenAI span shows token usage.
cc @Ladas