Confirmed: Qwen Image LoRA training works on Apple Silicon (M3 Ultra, MPS, fp32)

[MsFixer101]

Summary

Successfully trained a Qwen Image DreamBooth LoRA on Apple Silicon using MPS backend — appears to be the first public confirmation of this working.

Hardware: Mac Studio M3 Ultra, 256GB unified memory
Software: PyTorch 2.10.0, diffusers 0.37.0.dev0 (from git), peft, accelerate
Script: examples/dreambooth/train_dreambooth_lora_qwen_image.py
Speed: ~4.5 seconds/step at 512px resolution
Result: Training completes with decreasing loss, LoRA weights saved successfully

Key Finding: fp32 Required

fp16 produces NaN loss on MPS. The script correctly disables native_amp for MPS (line ~935), but without gradient scaling, fp16 gradients underflow to NaN. Switching to --mixed_precision="no" (fp32) resolves this completely.

Precision	Loss	Speed	Status
fp16	NaN (all steps)	4.18s/step	Broken
fp32	0.150 → 0.098 (decreasing)	4.51s/step	Working

Working Command

export MODEL_NAME="Qwen/Qwen-Image"
export INSTANCE_DIR="./training-images"
export OUTPUT_DIR="./output"
export PYTORCH_ENABLE_MPS_FALLBACK=1

accelerate launch train_dreambooth_lora_qwen_image.py \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --mixed_precision="no" \
  --instance_prompt="a photo of sks dog" \
  --resolution=512 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=1 \
  --learning_rate=2e-4 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=500 \
  --offload \
  --cache_latents \
  --seed=42

What Works

• MPS device detection and placement — no issues
• VAE, text encoder (Qwen2.5-VL 7B), and transformer (20B DiT) all load and run on MPS
• --offload (text encoder + VAE to CPU when not in use)
• --cache_latents (pre-compute VAE outputs, remove VAE from training loop)
• Gradient computation and optimizer step — no unsupported MPS operations
• LoRA weight saving

What Doesn't Work

• --mixed_precision="fp16" — NaN loss (no AMP gradient scaler on MPS)
• --mixed_precision="bf16" — correctly blocked by script (MPS doesn't support bf16)
• --use_8bit_adam — bitsandbytes is CUDA-only

Speed Projections (fp32, 512px)

Steps	Estimated Time
100	~7.5 min
500	~37 min
1000	~75 min

Notes

• Zero PYTORCH_ENABLE_MPS_FALLBACK warnings during training — all ops ran natively on Metal
• The 20B model loads comfortably in 256GB unified memory with --offload
• Latent caching for 3 test images took ~3 seconds
• Accelerate config: single machine, no distributed training, no mixed precision
• Speed was much better than expected — prior SDXL MPS reports (10 min/step on M3 Max) had set low expectations, but Qwen Image's DiT architecture seems well-suited to Metal

Environment

• macOS (Darwin 24.6.0)
• Python 3.11.14
• PyTorch 2.10.0 (MPS backend)
• diffusers 0.37.0.dev0 (installed from git — required for check_min_version)
• transformers, peft, accelerate (latest pip versions)

Hope this helps others trying to train Qwen Image LoRAs on Mac!