[bug] Smolagents Trace doesn't Log Tool outputs that are tuples into a single span.

[njbrake]

Describe the bug
Smolagents have tools attached: if the custom tool returns an int/str, all is well and output.value gets added to the span. If it's a tuple though, then the output.value isn't added to the span, but has a new "Step 1" span added after the SimpleTool span, which makes it harder to parse.

To Reproduce
make a tool called

def return_str():
    return "IM A STRING"

then make a tool called

def return_tuple():
    return "I'M A TUPLE", "I'M A TUPLE"

Add the tools to a smolagent, make them be called, and observe how return_str function output gets put in the span, but not the return_tuple function output.

Expected behavior
I would expect for the tuple output to show up as output.value in the "SimpleTool" invocation.

Screenshots
macos

Additional context

Here's a span from when the function is a str:

{
    "name": "SimpleTool",
    "context": {
      "trace_id": "0xe45053b15efac0c1e0491ae15c9ad48c",
      "span_id": "0x4142e403271a83a3",
      "trace_state": "[]"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x0f70eb08a7fac4e5",
    "start_time": "2025-03-22T19:30:28.929903Z",
    "end_time": "2025-03-22T19:30:29.502678Z",
    "status": {
      "status_code": "OK"
    },
    "attributes": {
      "input.value": "{\"args\": [], \"sanitize_inputs_outputs\": false, \"kwargs\": {\"area_name\": \"Vigo\"}}",
      "tool.name": "get_area_lat_lon",
      "tool.description": "Get the latitude and longitude of an area from Nominatim.\n\nUses the [Nominatim API](https://nominatim.org/release-docs/develop/api/Search/).",
      "tool.parameters": "{\"area_name\": {\"type\": \"string\", \"description\": \"The name of the area.\"}}",
      "output.value": "I'M A STRING",
      "output.mime_type": "text/plain",
      "openinference.span.kind": "TOOL"
    },
    "events": [],
    "links": [],
    "resource": {
      "attributes": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.31.1",
        "service.name": "unknown_service"
      },
      "schema_url": ""
    }
  },

Here's a span from when the function is a tuple:

{
    "name": "SimpleTool",
    "context": {
      "trace_id": "0x73ec83a565342cd47b30ceb0d71566e9",
      "span_id": "0x8993ae993dedd8e3",
      "trace_state": "[]"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0xb2e904a7a812a0b5",
    "start_time": "2025-03-22T19:32:19.878726Z",
    "end_time": "2025-03-22T19:32:20.317132Z",
    "status": {
      "status_code": "OK"
    },
    "attributes": {
      "input.value": "{\"args\": [], \"sanitize_inputs_outputs\": false, \"kwargs\": {\"area_name\": \"Vigo\"}}",
      "tool.name": "get_area_lat_lon",
      "tool.description": "Get the latitude and longitude of an area from Nominatim.\n\nUses the [Nominatim API](https://nominatim.org/release-docs/develop/api/Search/).",
      "tool.parameters": "{\"area_name\": {\"type\": \"string\", \"description\": \"The name of the area.\"}}",
      "openinference.span.kind": "TOOL"
    },
    "events": [],
    "links": [],
    "resource": {
      "attributes": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.31.1",
        "service.name": "unknown_service"
      },
      "schema_url": ""
    }
  },
  {
    "name": "Step 1",
    "context": {
      "trace_id": "0x73ec83a565342cd47b30ceb0d71566e9",
      "span_id": "0xb2e904a7a812a0b5",
      "trace_state": "[]"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x5dc291cfa1536391",
    "start_time": "2025-03-22T19:32:05.247173Z",
    "end_time": "2025-03-22T19:32:20.319740Z",
    "status": {
      "status_code": "OK"
    },
    "attributes": {
      "input.value": "{\"memory_step\": \"ActionStep(model_input_messages=None, tool_calls=None, start_time=1742671925.246196, end_time=None, step_number=1, error=None, duration=None, model_output_message=None, model_output=None, observations=None, observations_images=None, action_output=None)\"}",
      "output.value": "Execution logs:\n('IM A TUPLE', 'IM A TUPLE')\nLast output from code snippet:\nNone",
      "openinference.span.kind": "CHAIN"
    },
    "events": [],
    "links": [],
    "resource": {
      "attributes": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.31.1",
        "service.name": "unknown_service"
      },
      "schema_url": ""
    }
  },

[dosubot]

The issue you're encountering with Smolagents not logging tuple outputs into a single span seems to be related to how the framework handles different data types. Currently, the framework processes output values based on their type, and it appears that tuples are not directly supported in the same way as strings or other simple data types.

The _output_value_and_mime_type_for_tool_span function in the codebase is responsible for determining how output values are added to spans. It currently handles types like "string", "boolean", "integer", "number", and "object", but it does not explicitly handle tuples ^[1].

To address this issue, you might consider modifying the function to handle tuples by converting them into a format that can be logged as a single output value, such as a JSON string. This would involve adding a case for tuples in the _output_value_and_mime_type_for_tool_span function, similar to how objects are handled.

Here's a potential modification:

def _output_value_and_mime_type_for_tool_span(
    response: Any, output_type: str
) -> Iterator[Tuple[str, Any]]:
    if output_type in (
        "string",
        "boolean",
        "integer",
        "number",
    ):
        yield OUTPUT_VALUE, response
        yield OUTPUT_MIME_TYPE, TEXT
    elif output_type == "object":
        yield OUTPUT_VALUE, safe_json_dumps(response)
        yield OUTPUT_MIME_TYPE, JSON
    elif isinstance(response, tuple):
        yield OUTPUT_VALUE, safe_json_dumps(response)
        yield OUTPUT_MIME_TYPE, JSON

This change would convert tuple outputs into a JSON string, allowing them to be logged as a single output value in the span.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[njbrake]

Cool, I would be interested to know from a human openinference developer whether dosubot is onto the right thing, since it's suggesting a change to the openinference lib. I lack the knowledge yet of the openinference lib to know whether the advice is correct or not. Thanks!

[axiomofjoy]

Thanks @njbrake, we'll take a look.