fix: add timeout to tree-sitter parsing to prevent unbounded memory growth (APP-4323)#11640
Draft
warp-dev-github-integration[bot] wants to merge 1 commit into
Draft
Conversation
…rowth (APP-4323) Tree-sitter's error recovery (ts_parser__recover) can allocate unbounded memory when parsing pathological inputs. The heap profile from Sentry issue 7259255054 shows ts_parser__recover allocating 6+ GiB across multiple samples. This adds a 5-second timeout via ParseOptions progress callback. When the deadline is exceeded the parser returns None and we keep the existing (edited but un-reparsed) tree, preserving approximate highlighting until the next successful parse. This is complementary to the file-size guard in PR #11344 — it protects against pathological parse times even for files under the size threshold. Co-Authored-By: Oz <oz-agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a 5-second timeout to tree-sitter parsing to prevent unbounded memory growth from pathological error recovery.
Root Cause
Sentry issue 7259255054 shows "Excessive memory usage detected" with the heap profile revealing tree-sitter's
ts_parser__recoverallocating 6.1 GiB across just 2 samples, plusts_subtree_new_leafat 2.4 GiB. Total tree-sitter-related allocations exceeded 8.8 GiB.The problem:
SyntaxTreeState::parse_textcalledparse_with_optionswithout any timeout or cancellation mechanism. When tree-sitter encounters error recovery scenarios on certain inputs, it can allocate unbounded memory.Fix
Uses the
ParseOptionsprogress callback to enforce a 5-second deadline (PARSE_TIMEOUT). When the deadline is exceeded, the progress callback returnsfalse, causing tree-sitter to stop parsing and returnNone. The caller preserves the existing (edited but un-reparsed) syntax tree, maintaining approximate highlighting until the next successful parse.Relationship to PR #11344
This is complementary to the file-size guard in PR #11344, which skips tree-sitter entirely for files >1 MiB. This timeout protects against pathological parse times even on files under the size threshold — error recovery can spike memory on smaller inputs too.
Linked Issue
Testing
cargo check -p syntax_treepassescargo clippy -p syntax_tree -- -D warningspassescargo fmtpassesAgent Mode
CHANGELOG-BUG-FIX: Fixed excessive memory usage from tree-sitter parsing on pathological inputs
Conversation: https://staging.warp.dev/conversation/9ec872fd-2e84-464c-9871-7a23c7c998d2
Run: https://oz.staging.warp.dev/runs/019e5b94-d27c-700c-b7a8-68ba622d802b
This PR was generated with Oz.