netlify-cms-proxy-server throws errors when collective filesize of medialibrary exceeds some limit

[airtonix]

regarding: #5472 (comment)

Disabling publish_mode as in Knogobert/ntn-boilerplate#2 (comment) resolved the issue for me.

This didn't work, I did some git bisect'ish work and discovered that my guy had been uploading 30mb images (lol).

Removing a number of these suddenly made the error go away.

interesting to note: The error only occured when trying to view the CmsCollectionFile types of collections.

Describe the bug

have lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and of images that are 30mb and over in size for the media library to load.

To Reproduce

create a massive amount of images....

Expected behavior

no error

Screenshots

see linked issue thread: #5472 (comment)

Applicable Versions:

• Netlify CMS version: [e.g. 2.0.4]
• Git provider: [e.g. GitHub, BitBucket]
• OS: [e.g. Windows 7]
• Browser version [e.g. chrome 22, safari 11]

• Node.JS version:

CMS configuration

Additional context

[martinjagodic]

Hi @airtonix, please edit the issue so that it follows the template. A link to a comment on another issue is not enough. If it's a separate concern please describe it here in detail and if it's not, this should be discussed on the main issue thread.

[airtonix]

@martinjagodic I'd have thought that :

a) the power of the internet is based on "links to things" was a strong and positively good habit
b) that the title is explanitory enough.

[martinjagodic]

Thanks for updating the issue @airtonix. The thing with links is that they expire and it's harder to understand the issue if no context is provided. Although the title may seem obvious to the poster, it helps others to have more info, even if it's trivial.

[luanraithz]

I'm facing the same problem, I have about 1.2GB of images inside the media folder, the code seams to read every image into memory.

decap-cms/packages/netlify-cms-proxy-server/src/middlewares/localFs/index.ts

Line 95 in bc59b51

const mediaFiles = await Promise.all(files.map(file => readMediaFile(repoPath, file)));

The API timeouts at 60 seconds in my case. After that the UI works just fine.

I'm wrong? Is there any workaround?

[luanraithz]

Is there a real need to load the whole content into memory? The path or URL should do the job, right?

[sznowicki]

I'd like to bump this issue as it currently hits us as well.