⚡️ Speed up method LiteLLMResponsesTransformationHandler.convert_chat_completion_messages_to_responses_api by 23%

[codeflash-ai]

📄 23% (0.23x) speedup for LiteLLMResponsesTransformationHandler.convert_chat_completion_messages_to_responses_api in litellm/completion_extras/litellm_responses_transformation/transformation.py

⏱️ Runtime : 7.62 milliseconds → 6.19 milliseconds (best of 255 runs)

📝 Explanation and details

The optimized code achieves a 23% speedup through several key performance optimizations:

Main Optimizations:

1. Method lookup caching: The most impactful change is caching self._convert_content_to_responses_format as a local variable _convert_content_to_responses_format. This eliminates repeated attribute lookups in the hot loop that processes each message, saving substantial time when processing many messages.

2. Reduced dictionary key checks: In the tool_calls processing section, instead of using "name" in function and "arguments" in function checks followed by separate dictionary access, the code now uses function.get("name") and function.get("arguments") with None checks. This halves the dictionary lookups from 4 to 2 per tool call.

3. List method binding: For list processing, result.append and self._convert_content_str_to_input_text are bound to local variables (append_result, convert_str) to avoid repeated method lookups during iteration.

4. Tuple membership testing: Changed the list of accepted content types to a tuple, which is faster for in operator checks due to better hash table optimization.

Performance Impact by Test Type:

• Large-scale tests show the biggest gains (33-37% faster): The method lookup caching pays off significantly when processing many messages
• Basic single message tests show moderate gains (17-24% faster): Even small message sets benefit from reduced overhead
• Edge cases with simple operations may show slight slowdowns due to the overhead of variable assignment, but this is negligible compared to gains in realistic usage

The optimizations maintain identical behavior and error handling while focusing on the most frequently executed code paths during message transformation.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 96 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

import pytest
from litellm.completion_extras.litellm_responses_transformation.transformation import \
    LiteLLMResponsesTransformationHandler

# unit tests

@pytest.fixture
def handler():
    return LiteLLMResponsesTransformationHandler()

# -------------------- Basic Test Cases --------------------

def test_basic_user_message(handler):
    # Single user message, string content
    messages = [{"role": "user", "content": "Hello"}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.26μs -> 6.18μs (17.5% faster)

def test_basic_system_message(handler):
    # System message should be extracted as instructions
    messages = [{"role": "system", "content": "You are a helpful assistant."}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.10μs -> 1.22μs (9.61% slower)

def test_basic_assistant_message(handler):
    # Assistant message, string content
    messages = [{"role": "assistant", "content": "How can I help you?"}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 8.07μs -> 6.54μs (23.5% faster)

def test_basic_tool_message(handler):
    # Tool message should convert to function_call_output
    messages = [{"role": "tool", "content": "42", "tool_call_id": "abc"}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.17μs -> 1.43μs (18.1% slower)

def test_basic_assistant_function_call(handler):
    # Assistant message with tool_calls (function call)
    messages = [{
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_1",
                "function": {"name": "get_weather", "arguments": '{"city": "London"}'}
            }
        ]
    }]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.94μs -> 2.02μs (3.96% slower)

def test_basic_multiple_messages(handler):
    # Multiple messages, mixed roles
    messages = [
        {"role": "system", "content": "System instructions."},
        {"role": "user", "content": "Hi!"},
        {"role": "assistant", "content": "Hello!"},
    ]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 11.6μs -> 9.76μs (19.1% faster)

# -------------------- Edge Test Cases --------------------

def test_edge_empty_messages(handler):
    # Empty messages list
    messages = []
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 548ns -> 696ns (21.3% slower)

def test_edge_missing_role(handler):
    # Message missing role
    messages = [{"content": "No role"}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.96μs -> 6.09μs (30.5% faster)

def test_edge_non_string_content(handler):
    # Content is not a string (e.g. integer)
    messages = [{"role": "user", "content": 123}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.75μs -> 6.25μs (24.0% faster)

def test_edge_content_list_of_strings(handler):
    # Content is a list of strings
    messages = [{"role": "user", "content": ["A", "B"]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 12.3μs -> 11.0μs (11.9% faster)

def test_edge_content_list_of_dicts(handler):
    # Content is a list of dicts (multimodal)
    messages = [{"role": "user", "content": [
        {"type": "text", "text": "Hello"},
        {"type": "image_url", "image_url": "http://img.com/img.png"}
    ]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 18.1μs -> 17.4μs (4.28% faster)
    content = result[0]["content"]

def test_edge_image_url_dict_detail(handler):
    # Content is an image_url dict with detail
    messages = [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "http://img.com/img.png", "detail": "high"}}
    ]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 19.2μs -> 18.1μs (5.83% faster)
    content = result[0]["content"][0]

def test_edge_tool_call_missing_function(handler):
    # Assistant tool_calls missing function key
    messages = [{
        "role": "assistant",
        "tool_calls": [
            {"id": "call_1"}
        ]
    }]
    with pytest.raises(ValueError):
        handler.convert_chat_completion_messages_to_responses_api(messages) # 2.88μs -> 3.08μs (6.65% slower)

def test_edge_system_message_non_string_content(handler):
    # System message with non-string content (should go to input_items)
    messages = [{"role": "system", "content": [{"type": "text", "text": "Instructions"}]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 10.5μs -> 9.21μs (13.8% faster)

def test_edge_unknown_content_type(handler):
    # Content dict with unknown type
    messages = [{"role": "user", "content": [{"type": "unknown", "text": "foo"}]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 11.1μs -> 9.65μs (14.7% faster)

def test_edge_passthrough_response_format(handler):
    # Content dict already in responses API format
    messages = [{"role": "user", "content": [{"type": "input_text", "text": "already formatted"}]}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 9.88μs -> 8.65μs (14.1% faster)

def test_edge_tool_call_output_with_none(handler):
    # Tool message with None tool_call_id
    messages = [{"role": "tool", "content": "output", "tool_call_id": None}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.14μs -> 1.33μs (14.7% slower)

def test_edge_image_url_missing_url(handler):
    # Image_url dict missing url
    messages = [{"role": "user", "content": [{"type": "image_url", "image_url": {}}]}]
    with pytest.raises(ValueError):
        handler.convert_chat_completion_messages_to_responses_api(messages) # 16.2μs -> 15.5μs (4.46% faster)

def test_edge_content_is_none(handler):
    # Message with content None
    messages = [{"role": "user", "content": None}]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.06μs -> 1.19μs (10.2% slower)

# -------------------- Large Scale Test Cases --------------------

def test_large_scale_many_messages(handler):
    # 500 user messages
    messages = [{"role": "user", "content": f"msg {i}"} for i in range(500)]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.04ms -> 757μs (36.7% faster)
    for i, item in enumerate(result):
        pass

def test_large_scale_mixed_roles(handler):
    # 100 system, 100 user, 100 assistant
    messages = (
        [{"role": "system", "content": f"sys {i}"} for i in range(100)] +
        [{"role": "user", "content": f"user {i}"} for i in range(100)] +
        [{"role": "assistant", "content": f"assistant {i}"} for i in range(100)]
    )
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 426μs -> 320μs (33.3% faster)

def test_large_scale_function_calls(handler):
    # 50 assistant messages, each with 2 tool_calls
    messages = [{
        "role": "assistant",
        "tool_calls": [
            {"id": f"call_{i}_1", "function": {"name": "func1", "arguments": "args1"}},
            {"id": f"call_{i}_2", "function": {"name": "func2", "arguments": "args2"}}
        ]
    } for i in range(50)]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 27.0μs -> 29.5μs (8.41% slower)
    for i in range(0, 100, 2):
        pass

def test_large_scale_multimodal(handler):
    # 100 user messages, each with text and image
    messages = [{
        "role": "user",
        "content": [
            {"type": "text", "text": f"text {i}"},
            {"type": "image_url", "image_url": f"http://img.com/{i}.png"}
        ]
    } for i in range(100)]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 699μs -> 636μs (9.90% faster)
    for i, item in enumerate(result):
        content = item["content"]

def test_large_scale_system_and_messages(handler):
    # 1 system, 999 user messages
    messages = [{"role": "system", "content": "Instructions"}] + [
        {"role": "user", "content": f"msg {i}"} for i in range(999)
    ]
    result, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 2.17ms -> 1.62ms (33.9% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from typing import (Any, Dict, Iterable, List, Literal, Optional, Tuple, Union,
                    cast)

# imports
import pytest
from litellm.completion_extras.litellm_responses_transformation.transformation import \
    LiteLLMResponsesTransformationHandler


# --- Unit tests ---
@pytest.fixture
def handler():
    return LiteLLMResponsesTransformationHandler()

# 1. Basic Test Cases

def test_single_user_message(handler):
    # Single user message with string content
    messages = [{"role": "user", "content": "Hello!"}]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.53μs -> 6.25μs (20.5% faster)

def test_single_system_message(handler):
    # System message should be extracted as instructions
    messages = [{"role": "system", "content": "You are a helpful assistant."}]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.06μs -> 1.21μs (12.6% slower)

def test_user_and_assistant_messages(handler):
    # User and assistant messages, both with string content
    messages = [
        {"role": "user", "content": "Hello!"},
        {"role": "assistant", "content": "Hi there!"}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 10.4μs -> 8.58μs (21.3% faster)

def test_tool_role_message(handler):
    # Tool role message should be converted to function_call_output
    messages = [
        {"role": "tool", "content": "output here", "tool_call_id": "abc123"}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.15μs -> 1.37μs (16.0% slower)

def test_assistant_with_tool_calls(handler):
    # Assistant message with tool_calls (function calls)
    messages = [
        {
            "role": "assistant",
            "tool_calls": [
                {
                    "id": "call1",
                    "function": {
                        "name": "get_weather",
                        "arguments": "{\"location\": \"London\"}"
                    }
                },
                {
                    "id": "call2",
                    "function": {
                        "name": "get_time",
                        "arguments": "{\"timezone\": \"UTC\"}"
                    }
                }
            ]
        }
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 2.22μs -> 2.45μs (9.62% slower)

def test_assistant_with_tool_calls_missing_function(handler):
    # Assistant message with tool_calls missing function dict should raise ValueError
    messages = [
        {
            "role": "assistant",
            "tool_calls": [
                {
                    "id": "call1"
                    # missing 'function'
                }
            ]
        }
    ]
    with pytest.raises(ValueError):
        handler.convert_chat_completion_messages_to_responses_api(messages) # 3.11μs -> 3.58μs (13.3% slower)

def test_multimodal_content_text_and_image(handler):
    # User message with multimodal content: text and image_url
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image:"},
                {"type": "image_url", "image_url": "https://example.com/img.png"}
            ]
        }
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 19.9μs -> 18.8μs (5.56% faster)
    # Should contain both a text and an image block
    content = input_items[0]["content"]

def test_content_list_of_strings(handler):
    # Content as list of strings
    messages = [
        {"role": "user", "content": ["Hello", "World"]}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 11.9μs -> 10.3μs (15.1% faster)

def test_content_passthrough_types(handler):
    # Content as list of dicts already in responses API format
    messages = [
        {"role": "user", "content": [
            {"type": "input_text", "text": "foo"},
            {"type": "input_image", "image_url": "bar"}
        ]}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 12.9μs -> 11.5μs (12.7% faster)

def test_content_unknown_type(handler):
    # Content with unknown type should default to input_text
    messages = [
        {"role": "user", "content": [{"type": "unknown_type", "text": "foobar"}]}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 10.6μs -> 9.66μs (9.75% faster)

# 2. Edge Test Cases

def test_empty_messages_list(handler):
    # Empty messages list
    messages = []
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 526ns -> 653ns (19.4% slower)

def test_message_with_none_content(handler):
    # Message with content=None
    messages = [
        {"role": "user", "content": None}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.01μs -> 1.16μs (12.9% slower)

def test_message_with_missing_content(handler):
    # Message missing content key
    messages = [
        {"role": "user"}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.77μs -> 6.22μs (25.0% faster)

def test_system_message_with_nonstring_content(handler):
    # System message with non-string content
    messages = [
        {"role": "system", "content": [{"type": "text", "text": "System block"}]}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 10.7μs -> 9.64μs (10.9% faster)

def test_image_url_content_with_detail(handler):
    # Image_url content with detail field
    messages = [
        {"role": "user", "content": [
            {"type": "image_url", "image_url": {"url": "https://img.com/x.png", "detail": "high"}}
        ]}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 20.6μs -> 19.8μs (4.09% faster)
    img_block = input_items[0]["content"][0]

def test_image_url_content_missing_url(handler):
    # Image_url content missing url should raise ValueError
    messages = [
        {"role": "user", "content": [
            {"type": "image_url", "image_url": {}}
        ]}
    ]
    with pytest.raises(ValueError):
        handler.convert_chat_completion_messages_to_responses_api(messages) # 14.3μs -> 13.8μs (4.07% faster)

def test_tool_call_output_missing_tool_call_id(handler):
    # Tool role message missing tool_call_id
    messages = [
        {"role": "tool", "content": "output here"}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.27μs -> 1.43μs (11.1% slower)

def test_message_with_non_list_tool_calls(handler):
    # Assistant message with tool_calls not a list
    messages = [
        {"role": "assistant", "tool_calls": "notalist"}
    ]
    # Should skip tool_calls processing and treat as normal message
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 8.02μs -> 6.54μs (22.7% faster)

def test_content_as_non_list_iterable(handler):
    # Content as tuple of strings
    messages = [
        {"role": "user", "content": ("A", "B")}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 8.54μs -> 7.72μs (10.6% faster)

def test_content_as_integer(handler):
    # Content as integer should be converted to string
    messages = [
        {"role": "user", "content": 12345}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 7.67μs -> 6.36μs (20.6% faster)

# 3. Large Scale Test Cases

def test_large_number_of_messages(handler):
    # Many messages, alternating user/assistant
    N = 500
    messages = []
    for i in range(N):
        role = "user" if i % 2 == 0 else "assistant"
        messages.append({"role": role, "content": f"msg{i}"})
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.06ms -> 784μs (35.6% faster)
    for i in range(N):
        # Check correct type for each role
        if input_items[i]["role"] == "user":
            pass
        else:
            pass

def test_large_multimodal_message(handler):
    # One message with a large list of multimodal content blocks
    N = 500
    multimodal_content = []
    for i in range(N):
        if i % 2 == 0:
            multimodal_content.append({"type": "text", "text": f"text{i}"})
        else:
            multimodal_content.append({"type": "image_url", "image_url": f"https://img.com/{i}.png"})
    messages = [
        {"role": "user", "content": multimodal_content}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 1.21ms -> 1.17ms (2.70% faster)
    content_blocks = input_items[0]["content"]
    for i, block in enumerate(content_blocks):
        if i % 2 == 0:
            pass
        else:
            pass

def test_large_tool_calls(handler):
    # Assistant message with many tool_calls
    N = 500
    tool_calls = []
    for i in range(N):
        tool_calls.append({
            "id": f"call{i}",
            "function": {
                "name": f"func{i}",
                "arguments": f"{{\"arg\": {i}}}"
            }
        })
    messages = [
        {"role": "assistant", "tool_calls": tool_calls}
    ]
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 82.5μs -> 94.3μs (12.5% slower)
    for i in range(N):
        pass

def test_large_system_and_user_messages(handler):
    # Large number of system and user messages, only first system message should be instructions
    N = 100
    messages = [{"role": "system", "content": "Instructions!"}]
    for i in range(N):
        messages.append({"role": "user", "content": f"User {i}"})
        messages.append({"role": "system", "content": [{"type": "text", "text": f"SysBlock{i}"}]})
    input_items, instructions = handler.convert_chat_completion_messages_to_responses_api(messages) # 598μs -> 492μs (21.6% faster)
    # There should be N system message blocks (non-string) and N user message blocks
    sys_blocks = [x for x in input_items if x["role"] == "system"]
    user_blocks = [x for x in input_items if x["role"] == "user"]
    for i, block in enumerate(sys_blocks):
        pass
    for i, block in enumerate(user_blocks):
        pass
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-LiteLLMResponsesTransformationHandler.convert_chat_completion_messages_to_responses_api-mhdlm11l and push.