Prevent unbounded tool call repair buffering

[matdev83]

Summary

• cap the streaming tool call repair buffer using either the injected limit or a service-provided default
• expose the configured buffer size from ToolCallRepairService so processors can respect deployment limits
• add coverage for the new buffer trimming behaviour to ensure large streams are flushed safely

Testing

• ./.venv/Scripts/python.exe -m pytest tests/unit/core/services/test_tool_call_repair.py
• ./.venv/Scripts/python.exe -m pytest (fails: existing suite issues such as lint/mypy/test quality checks and server smoke test)

---

https://chatgpt.com/codex/tasks/task_e_68e93687b4608333b4487ec075dd0f5c

Summary by CodeRabbit

• New Features
  • Streaming tool-call output now enforces a configurable buffer limit to prevent excessive memory usage. Remaining content is emitted reliably when streams complete.
  • Added a visible configuration to read the current buffer size.
• Bug Fixes
  • Improves stability by trimming oversized partial outputs during streaming, reducing chances of stalled or bloated streams.
• Tests
  • Added unit tests to validate buffer-cap enforcement, partial flush behavior, and final flush on stream completion.

[matdev83]

✅ APPROVED - Ready to merge

Review Summary:

• PR adds buffer capping to ToolCallRepairProcessor to prevent unbounded memory usage
• Changes are well-implemented with proper configurable limits and sensible defaults (64KB)
• Added comprehensive test coverage for the new buffer trimming behavior
• All tests pass: 3017 passed, 42 skipped
• Code follows existing patterns and maintains backward compatibility
• Safety improvements are valuable for production deployments

Changes Approved:

• Buffer size configuration in ToolCallRepairProcessor constructor
• New max_buffer_bytes property in ToolCallRepairService
• Buffer trimming logic with proper UTF-8 byte handling
• Warning logging when buffer exceeds limits
• Comprehensive test coverage for edge cases

Safe to merge into dev branch.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Adds max_buffer_bytes handling to ToolCallRepairProcessor with buffer trimming and single-attempt repair per chunk, exposes max_buffer_bytes on ToolCallRepairService, and introduces unit tests for buffer cap behavior. Also adds a trivial file containing “1”.

Changes

Cohort / File(s)	Summary of Changes
Tool call repair buffering src/core/services/streaming/tool_call_repair_processor.py, src/core/services/tool_call_repair_service.py	Processor: new optional max_buffer_bytes arg with precedence, buffer trim logic via _trim_buffer, single repair attempt per chunk, final flush on done/cancel. Service: new max_buffer_bytes property exposing internal value.
Unit tests tests/unit/core/services/test_tool_call_repair.py	Adds tests validating buffer cap enforcement and final flush behavior for a single stream with set max_buffer_bytes.
Miscellaneous forcepytest.run	Adds a single line containing “1”.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Caller
  participant Processor as ToolCallRepairProcessor
  participant Service as ToolCallRepairService
  participant Sink as Downstream Consumer

  Caller->>Processor: process(chunk)
  alt is cancellation
    Processor-->>Sink: StreamingContent(is_cancellation=True)
    Processor->>Processor: Clear buffer for stream
  else not done
    Processor->>Processor: Append chunk to per-stream buffer
    Processor->>Service: repair_tool_calls(buffer)
    alt repair succeeded
      Processor-->>Sink: Emit repaired JSON
      Processor->>Processor: Clear buffer
    else no repair
      Processor->>Processor: _trim_buffer(max_buffer_bytes)\n(possible partial flush)
      opt flushed text exists
        Processor-->>Sink: Emit flushed text
      end
    end
  end

  opt final chunk (is_done)
    Processor->>Processor: Emit remaining buffer (if any)
    Processor-->>Sink: Final StreamingContent(is_done=True)
    Processor->>Processor: Clear buffer
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

I buffered bytes in moonlit stacks,
Then trimmed the tails with careful whacks.
A JSON tune, repaired just right,
Hops to the stream in tidy light.
Thump-thump—my paws approve the cap,
No overflow—just one clean lap. 🐇✨

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch codex/fix-detected-bug-in-proxy-server-b8pz3a

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dd216cf and 9bc60cf.

📒 Files selected for processing (4)

• forcepytest.run (1 hunks)
• src/core/services/streaming/tool_call_repair_processor.py (3 hunks)
• src/core/services/tool_call_repair_service.py (1 hunks)
• tests/unit/core/services/test_tool_call_repair.py (2 hunks)

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}