fix(fileRequest): use Guzzle's "sink" instead of stream #68
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I can reproduce locally such error(I was able to reproduce this even a year ago, but I was thinking that it is only on my local machine):
Allowed memory size of 536870912 bytes exhausted (tried to allocate 5000032 bytes) at /var/www/html/lib/private/Log/ExceptionSerializer.php#236After that always syncing dies, as process crashes and flag
onedrive_import_runningdoes not get reset in the database for the next background jobs run.Why does memory error occur? (a guess)
The Nextcloud HTTP client is a thin wrapper around Guzzle 7.
With Guzzle, the
stream => trueflag does not stream the response to the passed resource; it only prevents the body from being-decoded, afaik.The entire response is still buffered in memory before it is handed back as a
Psr7\Stream.When we then iterate over that stream with
fread(), every 5 MB chunk we copy is duplicated – the buffered copy inside Guzzle plus your$chunkstring – until PHP asks for another 5 MB and finally dies at ~512 MB.The easiest way is to tell Guzzle to write directly to the file handler, with sink
I have tested this changes and now I do not see
memory exhaustederrors anymore.