Train and evaluate Qwen 3.5-4B + Gemma 4 E4B on Colab #101

Summary

Run the Colab training notebook for both Qwen 3.5-4B and Gemma 4 E4B. Evaluate both against ship criteria and select the winner.

Depends On

• Build Colab training notebook (Unsloth + LoRA) #98 (Colab notebook)
• Distill training pairs for summarizer and explainer #99 (summarizer + explainer training pairs)
• Distill training pairs for SHAP narrator #100 (SHAP narrator training pairs)

Tasks

• Upload training data to notebook environment (or clone from repo)
• Run notebook with both models configured — single Run All execution
• Review comparison metrics table:
  • json_valid_rate per task (target >= 95%)
  • schema_valid_rate per task (target >= 90%)
  • shap_grounding_rate for narrator (target >= 80%)
  • Inference latency (p50, p95)
• Download GGUF artifacts (6 files: 3 tasks × 2 models)
• Select winner model based on metrics
• If neither passes ship criteria: diagnose, adjust hyperparams, re-run
• Document results in experiment log
• Track compute cost (expected: $8-20)

Model Candidates

Ship Criteria

Acceptance Criteria

• At least one model passes all blocking ship criteria for all 3 tasks
• GGUF files downloaded and verified loadable in Ollama
• Experiment results documented with head-to-head metrics comparison
• Winner selected with documented rationale