Skip to content

Limit test file contents in LLM prompt to prevent token overflow#2273

Merged
hiroshinishio merged 1 commit intomainfrom
wes
Feb 18, 2026
Merged

Limit test file contents in LLM prompt to prevent token overflow#2273
hiroshinishio merged 1 commit intomainfrom
wes

Conversation

@hiroshinishio
Copy link
Collaborator

@hiroshinishio hiroshinishio commented Feb 18, 2026

Summary

  • When find_test_files returns more than 5 files (e.g., grepping for "logger" returns 66), only file paths are passed to the LLM instead of full contents
  • Prevents BadRequestError (209K tokens > 200K limit) that caused OOM on customer PRs
  • Agent can still read individual files on demand via its file-reading tools

Test plan

  • Added test: test_few_test_files_include_contents_in_prompt - verifies <=5 files includes contents
  • Added test: test_many_test_files_include_paths_only_in_prompt - verifies >5 files passes paths only
  • All 22 issue_handler tests pass
  • pylint 10.0/10, pyright 0 errors

Social Media Post (GitAuto)

Grepping for "logger" in a customer repo returned 66 test files. We loaded all their contents into the prompt - 209K tokens, over the 200K limit. Lambda OOM'd and the PR went silent. Fix: if there are more than 5 test files, pass just the file paths and let the agent read what it needs.

Social Media Post (Wes)

Found a bug where grep for the word "logger" matched 66 test files in a customer repo. We stuffed all 66 file contents into the LLM prompt. 209K tokens. Hard limit is 200K. Lambda ate 2GB of RAM and got killed. Now we pass file paths instead of contents when there are too many matches. The agent reads what it needs.

When >5 test files are found by grep, pass only file paths instead of
full contents to avoid exceeding the 200K token input limit. The agent
can read individual files on demand.
@hiroshinishio hiroshinishio self-assigned this Feb 18, 2026
@hiroshinishio hiroshinishio merged commit 07f1da6 into main Feb 18, 2026
1 check passed
@hiroshinishio hiroshinishio deleted the wes branch February 18, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant