This is an Extension for the Automatic1111 Webui, which attempts to clean up memory after each generation.
- A few Reddit posts/comments mentioned that CUDA sometimes can cause memory issues/leaks.
This Extension tries to solve that by calling
torch.cuda.empty_cache()
after each generation.
- Perform
gc.collect()
andtorch.cuda.empty_cache()
after every generation - A button to manually trigger the above
- A button to unload then reload the checkpoint
- (This is just the same built-in functions in the Actions section of the Settings tab)
You can optionally go to the System
section of Settings tab to enable Unload Checkpoint after Generation
.
This makes it so that Checkpoint is only kept in memory during generation, thus achieving the most memory reduction,
but the reloading may take a while depending on your system specs.
[1] Shout out to @kgmkm_mkgm for sharing this Extension with tens of thousands of people
[2] Apparently, this indeed does help in certain situations