Skip to content

Conversation

@codeflash-ai
Copy link

@codeflash-ai codeflash-ai bot commented Oct 29, 2025

📄 6% (0.06x) speedup for contains_tag in litellm/litellm_core_utils/prompt_templates/factory.py

⏱️ Runtime : 362 microseconds 340 microseconds (best of 165 runs)

📝 Explanation and details

The optimization removes the capturing group (.+?) and replaces it with a non-capturing group .+? in the regex pattern. This changes f"<{tag}>(.+?)</{tag}>" to f"<{tag}>.+?</{tag}>".

Key optimization: Eliminating the unnecessary capturing group reduces regex engine overhead. Since the function only needs to check if the pattern exists (not extract the matched content), capturing groups create unnecessary work for the regex engine - it has to store and track the captured text even though it's never used.

Why this leads to speedup: Python's regex engine allocates memory and performs additional bookkeeping for each capturing group. By removing the parentheses, the engine can focus purely on pattern matching without the overhead of capturing and storing the matched content.

Test case performance patterns: The optimization shows particularly strong gains (15-37% faster) on large content tests with long strings inside tags, where the regex engine would otherwise capture and store substantial amounts of text. For basic cases with short content, gains are more modest (1-11%) but still consistent. The few slower cases (1-5%) appear to be within measurement noise and don't indicate a real regression.

Correctness verification report:

Test Status
⚙️ Existing Unit Tests 🔘 None Found
🌀 Generated Regression Tests 87 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 1 Passed
📊 Tests Coverage 100.0%
🌀 Generated Regression Tests and Runtime
import re

# imports
import pytest  # used for our unit tests
from litellm.litellm_core_utils.prompt_templates.factory import contains_tag

# unit tests

# 1. Basic Test Cases

def test_basic_tag_present():
    # Test when tag is present in the string
    codeflash_output = contains_tag("foo", "<foo>bar</foo>") # 3.61μs -> 3.41μs (5.86% faster)

def test_basic_tag_absent():
    # Test when tag is absent in the string
    codeflash_output = contains_tag("foo", "no tags here") # 2.78μs -> 2.87μs (2.90% slower)

def test_basic_multiple_tags():
    # Test when multiple tags exist, only one matches
    codeflash_output = contains_tag("foo", "<foo>bar</foo><baz>qux</baz>") # 3.67μs -> 3.28μs (11.9% faster)
    codeflash_output = contains_tag("baz", "<foo>bar</foo><baz>qux</baz>") # 1.84μs -> 1.68μs (9.48% faster)
    codeflash_output = contains_tag("quux", "<foo>bar</foo><baz>qux</baz>") # 1.46μs -> 1.34μs (9.36% faster)

def test_basic_empty_tag_content():
    # Test when tag is present but content is empty
    # Should be False because regex requires at least one character (.+?)
    codeflash_output = contains_tag("foo", "<foo></foo>") # 2.60μs -> 2.63μs (1.33% slower)

def test_basic_tag_with_whitespace_content():
    # Test when tag content is whitespace
    codeflash_output = contains_tag("foo", "<foo> </foo>") # 3.19μs -> 3.26μs (2.21% slower)

def test_basic_tag_with_newline_content():
    # Test when tag content is newline
    codeflash_output = contains_tag("foo", "<foo>\n</foo>") # 3.29μs -> 3.43μs (4.00% slower)

def test_basic_tag_with_special_characters():
    # Test when tag content contains special characters
    codeflash_output = contains_tag("foo", "<foo>@#$%^&*</foo>") # 3.61μs -> 3.25μs (11.0% faster)

def test_basic_tag_case_sensitive():
    # Test that tag matching is case-sensitive
    codeflash_output = contains_tag("Foo", "<foo>bar</foo>") # 2.96μs -> 2.86μs (3.28% faster)
    codeflash_output = contains_tag("foo", "<Foo>bar</Foo>") # 1.32μs -> 1.33μs (1.13% slower)

# 2. Edge Test Cases

def test_edge_empty_string():
    # Test with empty string
    codeflash_output = contains_tag("foo", "") # 2.43μs -> 2.58μs (5.51% slower)

def test_edge_empty_tag():
    # Test with empty tag name
    codeflash_output = contains_tag("", "<>content</>") # 3.79μs -> 3.50μs (8.44% faster)
    codeflash_output = contains_tag("", "no tags here") # 1.31μs -> 1.29μs (1.47% faster)

def test_edge_tag_at_start_and_end():
    # Tag at the very start and end of the string
    codeflash_output = contains_tag("foo", "<foo>bar</foo>") # 3.34μs -> 3.50μs (4.60% slower)
    codeflash_output = contains_tag("foo", "<foo>bar</foo>extra") # 1.56μs -> 1.48μs (5.27% faster)
    codeflash_output = contains_tag("foo", "extra<foo>bar</foo>") # 1.07μs -> 1.15μs (6.78% slower)

def test_edge_nested_tags():
    # Nested tags should match only the outermost matching tag
    codeflash_output = contains_tag("foo", "<foo><foo>bar</foo></foo>") # 3.41μs -> 3.21μs (6.19% faster)

def test_edge_overlapping_tags():
    # Overlapping tags (invalid XML, but regex should match the first closing tag)
    codeflash_output = contains_tag("foo", "<foo>abc<foo>def</foo>ghi</foo>") # 3.34μs -> 3.25μs (2.83% faster)

def test_edge_tag_with_attributes():
    # Tag with attributes should not match (since regex expects <tag>)
    codeflash_output = contains_tag("foo", "<foo attr='bar'>baz</foo>") # 2.66μs -> 2.60μs (2.39% faster)

def test_edge_tag_with_extra_spaces():
    # Tag with extra spaces: should not match
    codeflash_output = contains_tag("foo", "<foo >bar</foo >") # 2.79μs -> 2.59μs (7.53% faster)

def test_edge_partial_tag():
    # Partial tag, e.g., missing closing tag
    codeflash_output = contains_tag("foo", "<foo>bar") # 2.56μs -> 2.52μs (1.59% faster)
    codeflash_output = contains_tag("foo", "bar</foo>") # 1.21μs -> 1.11μs (9.27% faster)

def test_edge_tag_with_newlines_and_tabs():
    # Tag content contains newlines and tabs
    codeflash_output = contains_tag("foo", "<foo>\n\tbar\t\n</foo>") # 3.39μs -> 3.44μs (1.63% slower)

def test_edge_tag_with_unicode():
    # Tag content contains unicode characters
    codeflash_output = contains_tag("foo", "<foo>你好</foo>") # 3.83μs -> 3.77μs (1.54% faster)

def test_edge_tag_with_angle_brackets_in_content():
    # Tag content contains angle brackets
    codeflash_output = contains_tag("foo", "<foo><bar></foo>") # 3.47μs -> 3.21μs (8.13% faster)

def test_edge_tag_with_similar_tag_names():
    # Tag name is a substring of another tag
    codeflash_output = contains_tag("foo", "<foobar>baz</foobar>") # 2.61μs -> 2.85μs (8.40% slower)
    codeflash_output = contains_tag("foo", "<foo>baz</foo>") # 2.10μs -> 1.94μs (8.15% faster)

def test_edge_tag_with_number():
    # Tag name is a number
    codeflash_output = contains_tag("123", "<123>abc</123>") # 3.40μs -> 3.38μs (0.770% faster)
    codeflash_output = contains_tag("123", "<abc>123</abc>") # 1.41μs -> 1.38μs (2.40% faster)

def test_edge_tag_with_dash_underscore():
    # Tag name contains dash or underscore
    codeflash_output = contains_tag("foo-bar", "<foo-bar>baz</foo-bar>") # 3.54μs -> 3.38μs (4.70% faster)
    codeflash_output = contains_tag("foo_bar", "<foo_bar>baz</foo_bar>") # 1.74μs -> 1.70μs (2.29% faster)

def test_edge_tag_with_dot():
    # Tag name contains dot
    codeflash_output = contains_tag("foo.bar", "<foo.bar>baz</foo.bar>") # 3.49μs -> 3.31μs (5.31% faster)

def test_edge_tag_with_non_alphanumeric():
    # Tag name contains non-alphanumeric character
    codeflash_output = contains_tag("foo$", "<foo$>baz</foo$>") # 3.56μs -> 3.43μs (3.76% faster)

# 3. Large Scale Test Cases

def test_large_scale_many_tags():
    # Test with a string containing many tags
    tags = [f"tag{i}" for i in range(1000)]
    string = "".join([f"<{tag}>content{i}</{tag}>" for i, tag in enumerate(tags)])
    # Test a few tags at random positions
    codeflash_output = contains_tag("tag0", string) # 4.09μs -> 4.01μs (1.90% faster)
    codeflash_output = contains_tag("tag500", string) # 8.42μs -> 8.42μs (0.048% slower)
    codeflash_output = contains_tag("tag999", string) # 12.9μs -> 12.7μs (2.22% faster)
    codeflash_output = contains_tag("notatag", string) # 10.1μs -> 10.3μs (2.13% slower)

def test_large_scale_long_content():
    # Test with a tag containing a very long content
    long_content = "a" * 1000
    string = f"<foo>{long_content}</foo>"
    codeflash_output = contains_tag("foo", string) # 11.3μs -> 9.95μs (13.6% faster)

def test_large_scale_tag_at_end():
    # Tag at the very end of a long string
    string = "x" * 999 + "<foo>bar</foo>"
    codeflash_output = contains_tag("foo", string) # 3.43μs -> 3.50μs (2.06% slower)

def test_large_scale_tag_with_many_newlines():
    # Tag content with many newlines
    string = "<foo>" + "\n" * 999 + "bar</foo>"
    codeflash_output = contains_tag("foo", string) # 11.5μs -> 9.91μs (15.9% faster)

def test_large_scale_tag_with_repeated_content():
    # Tag content repeated many times
    string = "<foo>" + ("abc" * 333) + "</foo>"
    codeflash_output = contains_tag("foo", string) # 11.2μs -> 9.69μs (15.7% faster)

def test_large_scale_many_non_matching_tags():
    # String with many non-matching tags
    string = "".join([f"<tag{i}>content</tag{i}>" for i in range(1000)])
    codeflash_output = contains_tag("foo", string) # 10.7μs -> 10.7μs (0.159% faster)

def test_large_scale_tag_with_large_whitespace():
    # Tag content is only whitespace, but very long
    string = "<foo>" + (" " * 999) + "</foo>"
    codeflash_output = contains_tag("foo", string) # 11.3μs -> 10.1μs (12.6% faster)

def test_large_scale_tag_with_large_unicode():
    # Tag content is a large unicode string
    unicode_content = "你好" * 500
    string = f"<foo>{unicode_content}</foo>"
    codeflash_output = contains_tag("foo", string) # 13.9μs -> 10.4μs (33.7% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
import re
import string  # used for generating large scale test cases

# imports
import pytest  # used for our unit tests
from litellm.litellm_core_utils.prompt_templates.factory import contains_tag

# unit tests

# 1. Basic Test Cases

def test_contains_tag_basic_present():
    # Basic: Tag is present with content
    codeflash_output = contains_tag("foo", "<foo>bar</foo>") # 3.71μs -> 4.12μs (10.2% slower)

def test_contains_tag_basic_absent():
    # Basic: Tag is absent
    codeflash_output = contains_tag("foo", "<bar>baz</bar>") # 2.80μs -> 2.83μs (1.09% slower)

def test_contains_tag_basic_multiple_tags():
    # Basic: Multiple tags, only one matches
    s = "<foo>bar</foo><baz>qux</baz>"
    codeflash_output = contains_tag("foo", s) # 3.63μs -> 3.57μs (1.65% faster)
    codeflash_output = contains_tag("baz", s) # 1.73μs -> 1.64μs (5.55% faster)
    codeflash_output = contains_tag("quux", s) # 1.50μs -> 1.40μs (6.92% faster)

def test_contains_tag_basic_nested_tags():
    # Basic: Nested tags (should match only exact tag)
    s = "<foo><bar>baz</bar></foo>"
    codeflash_output = contains_tag("foo", s) # 3.40μs -> 3.34μs (1.92% faster)
    codeflash_output = contains_tag("bar", s) # 1.67μs -> 1.68μs (0.358% slower)

def test_contains_tag_basic_empty_content():
    # Basic: Tag present but empty content (should not match, as regex requires at least one char)
    s = "<foo></foo>"
    codeflash_output = contains_tag("foo", s) # 2.40μs -> 2.66μs (9.99% slower)

def test_contains_tag_basic_whitespace_content():
    # Basic: Tag present with whitespace content
    s = "<foo> </foo>"
    codeflash_output = contains_tag("foo", s) # 3.28μs -> 3.36μs (2.56% slower)

def test_contains_tag_basic_newline_content():
    # Basic: Tag present with newline content
    s = "<foo>\n</foo>"
    codeflash_output = contains_tag("foo", s) # 3.24μs -> 3.30μs (1.97% slower)

def test_contains_tag_basic_case_sensitive():
    # Basic: Tag is case-sensitive
    s = "<Foo>bar</Foo>"
    codeflash_output = contains_tag("Foo", s) # 3.49μs -> 3.28μs (6.56% faster)
    codeflash_output = contains_tag("foo", s) # 1.46μs -> 1.54μs (5.01% slower)

# 2. Edge Test Cases

def test_contains_tag_edge_tag_in_content():
    # Edge: Tag name appears in content, not as a tag
    s = "foo <foo>bar</foo> foo"
    codeflash_output = contains_tag("foo", s) # 3.37μs -> 3.19μs (5.62% faster)
    s2 = "foo <foofoo>bar</foofoo> foo"
    codeflash_output = contains_tag("foo", s2) # 1.41μs -> 1.37μs (2.99% faster)

def test_contains_tag_edge_partial_tags():
    # Edge: Partial tags (should not match)
    s = "<foo>bar</fo>"
    codeflash_output = contains_tag("foo", s) # 3.36μs -> 3.11μs (8.07% faster)

def test_contains_tag_edge_similar_tag_names():
    # Edge: Similar tag names
    s = "<foo>bar</foo><foobar>baz</foobar>"
    codeflash_output = contains_tag("foo", s) # 3.27μs -> 2.89μs (13.1% faster)
    codeflash_output = contains_tag("foobar", s) # 2.12μs -> 2.03μs (4.08% faster)

def test_contains_tag_edge_empty_string():
    # Edge: Empty string
    codeflash_output = contains_tag("foo", "") # 2.51μs -> 2.58μs (3.02% slower)

def test_contains_tag_edge_empty_tag():
    # Edge: Empty tag name (should not match anything)
    s = "<>bar</>"
    codeflash_output = contains_tag("", s) # 3.48μs -> 3.56μs (2.25% slower)

def test_contains_tag_edge_special_characters_in_tag():
    # Edge: Special characters in tag name
    s = "<f*o^o>bar</f*o^o>"
    codeflash_output = contains_tag("f*o^o", s) # 3.36μs -> 3.54μs (5.06% slower)
    s2 = "<foo-bar>baz</foo-bar>"
    codeflash_output = contains_tag("foo-bar", s2) # 2.17μs -> 2.00μs (8.30% faster)

def test_contains_tag_edge_tag_with_digits():
    # Edge: Tag name with digits
    s = "<foo1>bar</foo1>"
    codeflash_output = contains_tag("foo1", s) # 3.51μs -> 3.42μs (2.60% faster)

def test_contains_tag_edge_tag_with_underscore():
    # Edge: Tag name with underscore
    s = "<foo_bar>baz</foo_bar>"
    codeflash_output = contains_tag("foo_bar", s) # 3.51μs -> 3.45μs (1.77% faster)

def test_contains_tag_edge_tag_with_space():
    # Edge: Tag name with space (invalid in XML/HTML, but regex allows)
    s = "<foo bar>baz</foo bar>"
    codeflash_output = contains_tag("foo bar", s) # 3.48μs -> 3.32μs (4.57% faster)

def test_contains_tag_edge_tag_with_newline_in_tag():
    # Edge: Newline in tag name (should not match)
    s = "<foo\nbar>baz</foo\nbar>"
    codeflash_output = contains_tag("foo\nbar", s) # 3.53μs -> 3.44μs (2.50% faster)

def test_contains_tag_edge_tag_incomplete():
    # Edge: Incomplete tag
    s = "<foo>bar"
    codeflash_output = contains_tag("foo", s) # 2.73μs -> 2.83μs (3.77% slower)

def test_contains_tag_edge_tag_with_attributes():
    # Edge: Tag with attributes (should not match, as regex expects exact tag name)
    s = "<foo attr='1'>bar</foo>"
    codeflash_output = contains_tag("foo", s) # 2.92μs -> 2.87μs (1.57% faster)

def test_contains_tag_edge_tag_with_extra_spaces():
    # Edge: Tag with extra spaces (should not match, as regex expects exact tag name)
    s = "< foo >bar</ foo >"
    codeflash_output = contains_tag("foo", s) # 3.19μs -> 3.17μs (0.726% faster)

def test_contains_tag_edge_tag_with_slash_in_name():
    # Edge: Tag with slash in name
    s = "<foo/bar>baz</foo/bar>"
    codeflash_output = contains_tag("foo/bar", s) # 3.93μs -> 3.47μs (13.4% faster)

def test_contains_tag_edge_tag_with_unicode():
    # Edge: Unicode tag name
    s = "<фу>бар</фу>"
    codeflash_output = contains_tag("фу", s) # 4.60μs -> 4.74μs (2.93% slower)

# 3. Large Scale Test Cases

def test_contains_tag_large_many_tags():
    # Large: Many tags, only one matches
    tags = [f"tag{i}" for i in range(100)]
    s = "".join(f"<{t}>content{i}</{t}>" for i, t in enumerate(tags))
    # Pick a tag in the middle
    codeflash_output = contains_tag("tag50", s) # 4.98μs -> 4.91μs (1.51% faster)
    # Non-existing tag
    codeflash_output = contains_tag("notatag", s) # 2.85μs -> 2.66μs (7.26% faster)

def test_contains_tag_large_long_content():
    # Large: Tag with very long content
    long_content = "a" * 1000
    s = f"<foo>{long_content}</foo>"
    codeflash_output = contains_tag("foo", s) # 11.5μs -> 9.93μs (15.5% faster)

def test_contains_tag_large_long_string_no_tag():
    # Large: Very long string without any tags
    s = "a" * 1000
    codeflash_output = contains_tag("foo", s) # 2.98μs -> 2.98μs (0.034% faster)

def test_contains_tag_large_tag_at_end():
    # Large: Tag at the end of a long string
    s = "a" * 999 + "<foo>bar</foo>"
    codeflash_output = contains_tag("foo", s) # 3.70μs -> 3.67μs (0.626% faster)

def test_contains_tag_large_multiple_same_tag():
    # Large: Multiple instances of the same tag
    s = "".join(f"<foo>{i}</foo>" for i in range(100))
    codeflash_output = contains_tag("foo", s) # 3.24μs -> 3.03μs (6.73% faster)

def test_contains_tag_large_tag_with_large_content_and_other_tags():
    # Large: Tag with large content, surrounded by other tags
    content = "x" * 500
    s = "<bar>abc</bar>" + f"<foo>{content}</foo>" + "<baz>def</baz>"
    codeflash_output = contains_tag("foo", s) # 7.67μs -> 6.84μs (12.1% faster)
    codeflash_output = contains_tag("bar", s) # 1.78μs -> 1.68μs (5.88% faster)
    codeflash_output = contains_tag("baz", s) # 1.38μs -> 1.46μs (5.34% slower)

def test_contains_tag_large_tag_with_special_char_content():
    # Large: Tag with special characters in content
    special_content = string.punctuation * 10
    s = f"<foo>{special_content}</foo>"
    codeflash_output = contains_tag("foo", s) # 6.21μs -> 5.63μs (10.2% faster)

def test_contains_tag_large_tag_with_multiline_content():
    # Large: Tag with multiline content
    multiline_content = "\n".join(["line" + str(i) for i in range(100)])
    s = f"<foo>{multiline_content}</foo>"
    codeflash_output = contains_tag("foo", s) # 8.98μs -> 7.55μs (19.0% faster)

def test_contains_tag_large_tag_with_unicode_content():
    # Large: Tag with unicode content
    unicode_content = "你好" * 500
    s = f"<foo>{unicode_content}</foo>"
    codeflash_output = contains_tag("foo", s) # 13.7μs -> 9.99μs (37.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from litellm.litellm_core_utils.prompt_templates.factory import contains_tag

def test_contains_tag():
    contains_tag('', '')
🔎 Concolic Coverage Tests and Runtime
Test File::Test Function Original ⏱️ Optimized ⏱️ Speedup
codeflash_concolic__vli01p5/tmp4j4ivq06/test_concolic_coverage.py::test_contains_tag 2.90μs 2.88μs 0.835%✅

To edit these changes git checkout codeflash/optimize-contains_tag-mhboykhc and push.

Codeflash

The optimization removes the capturing group `(.+?)` and replaces it with a non-capturing group `.+?` in the regex pattern. This changes `f"<{tag}>(.+?)</{tag}>"` to `f"<{tag}>.+?</{tag}>"`.

**Key optimization:** Eliminating the unnecessary capturing group reduces regex engine overhead. Since the function only needs to check if the pattern exists (not extract the matched content), capturing groups create unnecessary work for the regex engine - it has to store and track the captured text even though it's never used.

**Why this leads to speedup:** Python's regex engine allocates memory and performs additional bookkeeping for each capturing group. By removing the parentheses, the engine can focus purely on pattern matching without the overhead of capturing and storing the matched content.

**Test case performance patterns:** The optimization shows particularly strong gains (15-37% faster) on large content tests with long strings inside tags, where the regex engine would otherwise capture and store substantial amounts of text. For basic cases with short content, gains are more modest (1-11%) but still consistent. The few slower cases (1-5%) appear to be within measurement noise and don't indicate a real regression.
@codeflash-ai codeflash-ai bot requested a review from mashraf-222 October 29, 2025 07:46
@codeflash-ai codeflash-ai bot added ⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: Medium Optimization Quality according to Codeflash labels Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: Medium Optimization Quality according to Codeflash

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant