search_for_pattern: add a snippet stage to the overflow shortening chain#1667
Open
bymebyu wants to merge 2 commits into
Open
search_for_pattern: add a snippet stage to the overflow shortening chain#1667bymebyu wants to merge 2 commits into
bymebyu wants to merge 2 commits into
Conversation
When search results exceed max_answer_chars, the tool previously fell back
directly from full match content to bare line numbers, forcing the agent to
re-read files to understand what matched. This adds an intermediate stage
returning line numbers plus the matched text truncated to 60 characters:
full matches -> {line, text-snippet} -> line numbers -> per-file counts -> summary
In an agent-level A/B benchmark (8 tasks x 5 trials, headless Claude agents),
the bare line-number overflow response caused the agent to time out in 2/5
trials on an overflow-search task; with the snippet stage it succeeded 5/5
with exact line answers.
The snippet stage's header restated "too long" (duplicating the framework's overflow banner) and said "line ranges" although it emits single line points. Replace it with a plain label consistent with the sibling shortening factories. Also add the CHANGELOG entry under Tools.
opcode81
requested changes
Jul 9, 2026
Comment on lines
628
to
+634
| def make_lines_only() -> str: | ||
| """Match locations without surrounding context""" | ||
| return f"Match lines per file:\n{self._to_json(match_lines_by_file)}" | ||
| """Match locations with line number and truncated matched text.""" | ||
| compact = { | ||
| path: [{"line": m["line"], "text": str(m["text"])[:_TEXT_TRUNCATE]} for m in lines] | ||
| for path, lines in match_lines_by_file.items() | ||
| } | ||
| return f"Match locations per file (line + matched text):\n{self._to_json(compact)}" |
Contributor
There was a problem hiding this comment.
Instead of adding a new stage to the shortening chain which adds truncation, this replaces the previous first stage, where the full first line is retained.
Also, for the new shortening stage, the LLM is not informed
- that lines are truncated to
_TEXT_TRUNCATEcharacters in general - which lines are truncated (could add an indicator like
...at the end of affected lines)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #1640 (direction approved by @opcode81 in the issue).
Summary
When a
search_for_patternresult exceedsmax_answer_chars, the shortening chainpreviously fell back from full match content directly to bare line numbers, so the
agent learned where the matches were but not what matched — and had to re-read files
to recover that. This adds one intermediate stage to the chain:
The snippet stage fits within the limit precisely when the matched lines are long (which
is exactly the case that makes full results overflow), and it degrades gracefully: if even
the snippets exceed the limit, the chain continues on to bare line numbers, per-file counts
and finally a summary, as before.
What the agent actually receives (deterministic)
Reproducible tool-level demonstration — a search with ~1,300 matches whose full result is
196,274 chars, against the default
max_answer_charsof 150,000. The task is to find the onematch belonging to a specific class (
Message777). This is the first fallback returned ineach case (the
The answer is too long (…)banner is added by the framework in both):Before — bare line numbers:
Line
17894is indistinguishable from the other ~1,300 — to tell which match isMessage777,the agent has to open the file.
After — line number + matched text:
The distinguishing text (
777) is inline, so the agent can pick the right match and issue atargeted
read_file— without re-reading the file just to discover what matched.Changes
src/serena/tools/file_tools.py— snippet stage inSearchForPatternTool(adds a{line, text}intermediate and keeps a dedicated bare-line-numbers stage after it).CHANGELOG.md— entry under Tools.The motivating agent-level A/B measurement is described in #1640.