Skip to content

Conversation

longnguyen2004
Copy link

Description

Some games such as vivid/stasis, due to developer ignorance, have 10s to 100s of 4096x4096 texture pages. With the current design of the script, eventually all the texture pages will be loaded and held in memory by the TextureWorker, causing explosive memory growth (up to 7GB in my case).
This PR changes the dump function to only process one texture page at a time, avoiding the excessive memory usage.

Caveats

Parallelism might be reduced, but not dying due to running out of memory is arguably more important.

Notes

@colinator27
Copy link
Member

Does this change the behavior of the script at all? I don't recall exactly what the dumping behavior in particular is used for, but if it ends up with the same result I guess it's okay. Would be nice to know how much this impacts performance in games where memory isn't as much of an issue, though.

Copy link

github-actions bot commented May 7, 2025

@longnguyen2004
Copy link
Author

The problem isn't when running the game, it's when running the script itself. For games that have very large bundled textures (vivid/stasis has 2+GB total), reading all of them into memory (which will happen due to the usage of TextureWorker in this script) will easily causes 8+GB of memory usage.

Anyway, the modified script sometimes outputs a corrupted texture file, causing the rest of the script to fail. Not sure why but I'm investigating.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants