Can i use this script to pre-train models?

Hi there!
I am currently working on pre-training a Llama or Mistral model with clinical texts. Is there any way to use this QLoRA + FSDP script to do such training? Should I make any changes to the current code to be able to do pre-training?  