Skip to content
This repository has been archived by the owner on Dec 13, 2023. It is now read-only.

When the input and output data volume is particularly large, arbitrary downloading of external stored content leads to OOM #3763

Open
shoy160 opened this issue Sep 6, 2023 · 0 comments
Labels
type: bug bugs/ bug fixes

Comments

@shoy160
Copy link

shoy160 commented Sep 6, 2023

Describe the bug
When the input and output data volume is particularly large, arbitrary downloading of external stored content leads to OOM.

Details
Conductor version: 3.13.3
Persistence implementation: Postgres, MySQL
Queue implementation: Postgres, MySQL
Lock: Redis
Workflow definition:
Task definition:
Event handler definition:

To Reproduce
Steps to reproduce the behavior:
java/com/netflix/conductor/core/dal/ExecutionDAOFacade.java getTaskModel(String taskId) 493

Why should inputstream be converted to String first and then serialized to Map format in
java/com/netflix/conductor/core/utils/ExternalPayloadStorageUtils.java downloadPayload(String path) 78

Expected behavior
Be cautious when downloading external storage content, and do not download it unless it is necessary.

Screenshots
img_v2_dcbedb51-1c62-4ee6-ab94-1084d0a55e0g

@shoy160 shoy160 added the type: bug bugs/ bug fixes label Sep 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: bug bugs/ bug fixes
Projects
None yet
Development

No branches or pull requests

1 participant