Insufficient Memory Issue

[Cynthiaaa-peng]

I'd like to ask how much memory is required for the server of your code repository. I'm using a server with a total of 160GB of memory, and I've already set the batch_size to a very small value. However, when I run the "bash run_logicRL_4gpus_phase1.sh" script, it keeps prompting me that there is not enough memory.

[Tim-Siu]

I personally run the experiments with 4 A100 80GV gpus. I believe if you tune ppo_max_token_len_per_gpu, it is possible to run experimens with 4 A6000.

[Cynthiaaa-peng]

非常感谢您的解答，我尝试调整ppo_max_token_len_per_gpu至1500，仍然有报错，我看了一下应该是在rollout阶段报的CPU不够（如图），我的配置是4卡的L40（GPU共160G，CPU共125G），您能帮我看一下是否还要调整其他的参数，还是我的配置不够呢~

[Tim-Siu]

貌似verl rollout阶段cpu oom挺常见的。可以看看这个issue里的解决方案能不能解决

verl-project/verl#429
verl-project/verl#920 (comment)

理论上这个仓库里的verl和vllm版本没有memory leakage的bug，有可能125gb内存真的不太够😭