Fix transcript parsing failing on lines exceeding 10MB #42
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace bufio.Scanner with bufio.Reader.ReadBytes to remove the 10MB line size limit. Scanner fails with token too long when transcript lines exceed its buffer size, which happens with large tool outputs (e.g., reading big files). ReadBytes has no artificial limit and will allocate as needed.
Soph
approved these changes
Jan 14, 2026
Collaborator
Soph
left a comment
Soph
left a comment
There was a problem hiding this comment.
So Georgs log had a singe line > 10MB?
Contributor
Author
Yes, it was in fact 28MBs. No idea what was in it |
4 tasks
khaong
added a commit
that referenced
this pull request
Jan 22, 2026
…pt lines Replace bufio.Scanner with bufio.Reader.ReadBytes to remove the 10MB line size limit in GetTranscriptPosition and ExtractModifiedFilesFromOffset. Scanner fails with "token too long" when transcript lines exceed its buffer size, which can happen with large tool outputs. ReadBytes has no artificial limit and will allocate as needed. Addresses PR review feedback referencing PR #42. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Entire-Checkpoint: 7295d22ef3a1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace bufio.Scanner with bufio.Reader.ReadBytes to remove the 10MB line size limit.
Scanner fails with token too long when transcript lines exceed its buffer size, which happens with
large tool outputs(e.g., reading big files).ReadBytes has no artificial limit and will allocate as needed.