Adds VLM Training support to SFTTrainer + VSFT script

[edbeeching]

• Modifies the SFTTrainer so that it can be used with a VLM SFT dataset
• Adds an example script vsft.py to train the llava1.5 model with an instruct dataset

TODO:

• Test PEFT support
• Run full training
• Add example to docs
• add tests

Example usage:

python examples/scripts/vsft.py \
    --model_name_or_path="llava-hf/llava-1.5-7b-hf" \
    --report_to="wandb" \
    --learning_rate=1.4e-5 \
    --per_device_train_batch_size=8 \
    --gradient_accumulation_steps=1 \
    --output_dir="data/vsft-llava-1.5-7b-hf" \
    --logging_steps=5 \
    --num_train_epochs=1 \
    --push_to_hub \
    --gradient_checkpointing \
    --remove_unused_columns=False \
    --torch_dtype=float16 \
    --fp16=True \ 
    --dataset_name=HuggingFaceH4/llava-instruct-mix-vsft \

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[lewtun]

Very clean implementation @edbeeching ! Just a few nits and a few unit/integration tests needed and this should be good to merge

[lewtun]

Let's add an integration test for this case under the existing SFTTrainer tests, along with an example training a tiny random llava model

[pcuenca]

Looking great!

[lewtun]

Suggested change

	"""
	# to evaluate, first install the lmms-eval framework: pip install git+https://github.com/EvolvingLMMs-Lab/lmms-eval.git
	# then run:
	accelerate launch --num_processes=8 -m lmms_eval \
	--model llava_hf \
	--model_args pretrained=llava-hf/llava-1.5-7b-hf \
	--tasks mmbench \
	--batch_size 1 \
	--output_path ./logs/ \
	--log_sample
	"""

[younesbelkada]

Thanks ! We should be good to go after fixing the merge conflict on main ! 🚀