Restrict file size of tmp files

[katauber]

Restrict file size of tmp files used for workflows.

functional review: @dr0i
code review: @fsteeg

[dr0i]

This is not solved yet.

[dr0i]

Came up in DINI-KIM WS 2025. There was a 200GB file in tmp.
[edit 2025-05-08: kudos @zumstein for providing this breaking test]

[Phu2]

Came up in DINI-KIM WS 2025. There was a 200GB file in tmp.

This big JSON file was created by processing a specific OAI endpoint with >50k records. I can reproduce this behavior with this same endpoint. Lots of requests (per resumptionToken) are being made while the result file is still empty. At some point in time the requests come to a halt and the result file is written very fast.

When examining the file i noticed a lot of line breaks and white spaces that accumulated more and more towards the end of the file. I wonder wether the error occurs during the transformation (metafacture) or later when writing the result file (playground).

Is the incoming OAI data validated in any way?

[TobiasNx]

Could you add the FLUX Workflow, that broke the playground, that i can run it locally?