Resource needed for multi-node training #106

[Some-random]

How many GPUs/nodes do we need for 32B model training? We have tried using 2 8*H100 nodes, each with 80G memory, to run the training of 32B model. But we're getting OOM errors

[AnselCmy]

Could you please share your training configuration parameters? To address the OOM errors, there are a few potential solutions:

1. Increase the tensor model parallel size (rollout.tensor_model_parallel_size) to distribute the model across more GPUs
2. Decrease the training batch size to reduce memory usage