Memory leak

[Alex-lsky]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.13.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

The memory footprint of dify,api, and work containers deployed with docker is always monotonically increasing, and memory is not released when calls are stopped, I have verified this issue in multiple versions (0.8-0.13.1) and in multiple environments (docker, k8s)

✔️ Expected Behavior

When the call is stopped, the memory usage drops

❌ Actual Behavior

The memory of the api and work is always growing until it takes up about 6G and the LLM call efficiency becomes so low that the container must be restarted to alleviate it

[dosubot]

@Alex-lsky Hello! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve this issue.

It seems like you're experiencing a memory leak issue with the Dify 'api' and 'work' containers, where memory usage continuously increases until it affects performance. Unfortunately, I couldn't find specific information about known memory leaks in the Dify documentation or previous issues. However, here are a few steps you might consider to address this problem:

1. Update to Latest Version: Ensure you are using the latest version of Dify, as updates may contain bug fixes related to memory management.

2. Monitor Resource Usage: Use monitoring tools to track memory usage over time. This can help identify patterns or specific operations that lead to increased memory usage.

3. Adjust Configuration Settings: Review and adjust configuration settings related to memory usage, such as PostgreSQL and OpenSearch parameters, to optimize resource allocation ^[1].

4. Investigate Code for Leaks: If you have access to the codebase, investigate areas where memory might not be properly released, such as long-lived objects or unclosed resources.

5. Limit Resource Usage: Set memory limits on your Docker containers to prevent them from consuming excessive resources. This can be done using Docker's resource management options.

6. Restart Strategy: Implement a restart strategy for your containers to periodically release memory, especially if a long-term fix is not immediately available.

If these steps do not resolve the issue, further investigation into the specific operations or code paths causing the memory increase may be necessary.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[liangpeihuahua]

I also encountered this problem. The memory kept increasing and I had to restart the container to solve it. My version is 0.13.1