Training using FSDP, qLoRa on multinode #74
Description
How to train a TinyLlama-like model using FSDP, qLoRa on two (or more) nodes if each node has one (or more) GPU using train.py (main branch) ?
I am grateful in advance for any help!
{{ message }}