nautilus-tutorial

Material for the Nautilus Tutorial

To train a toy GPT-2 from scratch

python3 gpt2_training.py --shakespeare_files dataset/input.txt

To train a toy GPT-2 from scratch with bayesian hyperparameter sweep

python3 gpt2_training_sweep.py --shakespeare_files dataset/input.txt

To train a toy GPT-2 from scratch with bayesian hyperparameter sweep on two GPUs (DDP)

torchrun --nproc_per_node=2 gpt2_training_sweep_ddp.py --shakespeare_files dataset/input.txt

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
dataset		dataset
docs		docs
Dockerfile		Dockerfile
README.md		README.md
gpt2_training.py		gpt2_training.py
gpt2_training_sweep.py		gpt2_training_sweep.py
gpt2_training_sweep_ddp.py		gpt2_training_sweep_ddp.py
requirements.txt		requirements.txt
tutorial-pod-ddp.yaml		tutorial-pod-ddp.yaml
tutorial-pod.yaml		tutorial-pod.yaml

Provide feedback