Large file with a lot small files will cause "memory leak"

[Barry-DareIT]

Q	A
Library version(s) affected:	3.1.9
PHP version(s):	All
OS (with bit depth):	Effects all

Description
Due to the caching in memory of entry content that is less than 512kb, you can easily run out of memory when you try to extract a zip file with a lot of smaller files. Ran into this problem with a zip file containing over a million images of around 400 kb.

How to reproduce
Get a zip file with a lot of entries smaller than 512kb, so it will exceed your memory limit and loop through the entries and get their content.

Possible Solution
Make it possible to set the size threshold for the entry content

Additional context
None

[Ne-Lexa]

Hello.
Such a problem really exists.

The cache is used to re-retrieve the contents of the file without decryption or decoding.

I think the correct option would be to abandon caching into memory altogether, and use the php://temp stream. It will be possible to specify the limit as php://temp/maxmemory:102400.
It will be necessary to test.

In the next major version, much will be recycled.

[Barry-DareIT]

@Ne-Lexa thanks for the quick response.

I forked the project and made options for setting the threshold for the memory caching and an options to disable the caching in total. This makes it work for our specific situation.

Using the temp stream for caching will result in the same situation if someone has to work with the zip files with a lot of small entries, even with the 100kb memory limit on the temp stream.

Of course, these are edge cases and you should wonder if you want to support those.

Thanks for the library, it really helped us with processing archive files that ZipArchive is unable to process.

[Ne-Lexa]

@Barry-DareIT, I would like to more elegantly solve this problem. Memory shortages can also occur when using the addFromString method.

For the STORED compression method, it will be better to turn off cache-memoty and read the data from the open file. It is probably better to leave caching only for encrypted entries. I will test performance without caching and with limited caching when re-reading the contents of zip entries.

If we set a memory limit, it is better to do this globally for all methods and monitor the memory consumption, and store anything that exceeds the memory limit in temporary files.

[Barry-DareIT]

@Ne-Lexa makes sense. Better deny my PR and I will keep an eye out for updates to this project. Will use my fixed fork for our specific edge case.

[Ne-Lexa]

Fixed in 3.2.0 version