Limit test file contents in LLM prompt to prevent token overflow#2273
Merged
hiroshinishio merged 1 commit intomainfrom Feb 18, 2026
Merged
Limit test file contents in LLM prompt to prevent token overflow#2273hiroshinishio merged 1 commit intomainfrom
hiroshinishio merged 1 commit intomainfrom
Conversation
When >5 test files are found by grep, pass only file paths instead of full contents to avoid exceeding the 200K token input limit. The agent can read individual files on demand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
find_test_filesreturns more than 5 files (e.g., grepping for "logger" returns 66), only file paths are passed to the LLM instead of full contentsTest plan
test_few_test_files_include_contents_in_prompt- verifies <=5 files includes contentstest_many_test_files_include_paths_only_in_prompt- verifies >5 files passes paths onlySocial Media Post (GitAuto)
Grepping for "logger" in a customer repo returned 66 test files. We loaded all their contents into the prompt - 209K tokens, over the 200K limit. Lambda OOM'd and the PR went silent. Fix: if there are more than 5 test files, pass just the file paths and let the agent read what it needs.
Social Media Post (Wes)
Found a bug where grep for the word "logger" matched 66 test files in a customer repo. We stuffed all 66 file contents into the LLM prompt. 209K tokens. Hard limit is 200K. Lambda ate 2GB of RAM and got killed. Now we pass file paths instead of contents when there are too many matches. The agent reads what it needs.