[NewTextureRepacker] Group textures to dump by their texture page to prevent explosive memory growth #2109
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Some games such as vivid/stasis, due to developer ignorance, have 10s to 100s of 4096x4096 texture pages. With the current design of the script, eventually all the texture pages will be loaded and held in memory by the
TextureWorker
, causing explosive memory growth (up to 7GB in my case).This PR changes the dump function to only process one texture page at a time, avoiding the excessive memory usage.
Caveats
Parallelism might be reduced, but not dying due to running out of memory is arguably more important.
Notes