Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: One or more background workers are no longer alive. Exiting. Please check the print statements above for the actual error message #2656

Open
RunChuan123 opened this issue Dec 24, 2024 · 1 comment
Assignees

Comments

@RunChuan123
Copy link

When I processed the data for the first time and was ready to run, it reported this error. When I changed to a100 training on colab, the memory would always increase inexplicably, causing colab to crash. It even crashed and disconnected before the error message was printed. I have tried many times. I hope someone can help me.
屏幕截图 2024-12-24 165949

@maxencewynen
Copy link

What is the command that prompted the error? Was it during planning, preprocessing or training? Do you have any logs to show associated to this?
It does look like the crash is due to your CPU RAM being overloaded. I would suggest making sure you are running critical code on your GPU and not your CPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants