fix: reduce memory usage from SessionContext clone and DirectoryFetcher#11626
Draft
warp-dev-github-integration[bot] wants to merge 1 commit into
Draft
fix: reduce memory usage from SessionContext clone and DirectoryFetcher#11626warp-dev-github-integration[bot] wants to merge 1 commit into
warp-dev-github-integration[bot] wants to merge 1 commit into
Conversation
Two targeted fixes for excessive memory usage (Sentry #7259255054): 1. Wrap SessionContext.cached_directory_entries in Arc<DashMap> so that SessionContext::clone() is O(1) instead of deep-copying every cached directory listing. The DashMap already provides interior mutability, so shared ownership via Arc is safe. This eliminates ~550 MB of redundant allocations observed in the heap profile when display chips clone the session context. 2. Cap DirectoryFetcher results to 1,000 items after sorting. Directories with a very large number of entries (e.g. node_modules, .git/objects) were causing unbounded Vec<DirectoryItem> growth (~930 MB in the heap profile). The sort runs first so directories and text files are prioritised over binary files. Co-Authored-By: Oz <oz-agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two targeted fixes for excessive memory usage identified via Sentry alert #7259255054.
Heap Profile Analysis (10 GB total footprint)
The heap profile from event
0e7bc8aashows three major allocation categories:Vec<StyledBufferBlock>::clone— 5.66 GB (58.4%) inCodeEditorModel::handle_content_model_event. This is an architectural issue requiringEditDelta.new_linesto be wrapped inArc— noted as follow-up work.SessionContext::clone(DashMap deep copy) — ~550 MB (5.4%) across display chip creation pathsDirectoryFetcher::refetch_directory— ~930 MB (9.7%) from unbounded directory item vectorsChanges
1. Wrap
SessionContext.cached_directory_entriesinArc<DashMap>The
SessionContextstruct derivesClone, which deep-copies theDashMap<TypedPathBuf, Arc<Vec<EngineDirEntry>>>directory cache. This is called frequently fromAgentInputFooter::create_display_chipsandPromptDisplay::reset_chips. Wrapping inArcmakes clone O(1) while preserving interior mutability.2. Cap
DirectoryFetcherresults to 1,000 itemsDirectories with very large numbers of entries (e.g.
node_modules,.git/objects, build output) caused unboundedVec<DirectoryItem>growth. Items are sorted first (directories → text files → binary files) so truncation preserves the most useful entries.Follow-up needed
The dominant allocation (
Vec<StyledBufferBlock>::cloneat 5.66 GB) requires wrappingEditDelta.new_linesinArc<Vec<StyledBufferBlock>>to avoid deep clones during content change events. This is a larger architectural change touching many files and tests.Test Plan
cargo check✅cargo clippy✅cargo fmt✅CHANGELOG-IMPROVEMENT:
CHANGELOG-BUG-FIX:
CHANGELOG-NEW-FEATURE:
CHANGELOG-IMAGE:
Conversation: https://staging.warp.dev/conversation/be54270e-a827-4115-a8f9-6049221a4fe3
Run: https://oz.staging.warp.dev/runs/019e58b0-c118-7a15-9f1a-0b2dbfd4a8c6
This PR was generated with Oz.