8 Pull requests merged by 2 people

• Torch 2.6 support for base docker image

  #2312 merged Feb 5, 2025

• fix: drop long seq even if not sample packing

  #2211 merged Feb 4, 2025

• [feature] sweeps

  #2171 merged Feb 2, 2025

• better handling of multipack dataset length

  #2296 merged Feb 2, 2025

• set MODAL_IMAGE_BUILDER_VERSION=2024.10 to 2024.10 to test latest builder

  #2302 merged Feb 1, 2025

• KD Trainer V2

  #2303 merged Feb 1, 2025

• fix: add warning for invalid eval_steps or save_steps

  #2298 merged Jan 31, 2025

• Misc fixes 20250130

  #2301 merged Jan 31, 2025

4 Pull requests opened by 3 people

• feat(doc): Add multi-node torchrun info

  #2304 opened Feb 1, 2025

• TRL upgrade

  #2307 opened Feb 3, 2025

• chore: cleanup deprecated config elements

  #2309 opened Feb 3, 2025

• feat: add torch2.6 to ci

  #2311 opened Feb 5, 2025

3 Issues closed by 2 people

• Dependency Conflicts When Installing Axolotl

  #2313 closed Feb 6, 2025

• Zamba2AttentionDecoderLayer.forward() takes from 4 to 10 positional arguments but 11 were given

  #1799 closed Feb 4, 2025

• 'AdamW' object has no attribute 'optim_bits'

  #2191 closed Feb 3, 2025

2 Issues opened by 2 people

• Mistral-Small-3 support

  #2308 opened Feb 3, 2025

• Integrate GRPO for Replicating DeepSeek-R1 and Other Inference Models, Achieving a HuggingFace open‑r1–like Effect

  #2306 opened Feb 2, 2025

10 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• KD trainer w/ logprobs

  #2202 commented on Jan 31, 2025 • 6 new comments

• feat: add config for optional parameters in a chat message

  #2260 commented on Feb 3, 2025 • 3 new comments

• feat(doc): Improve guide to dataset types with better examples

  #2286 commented on Feb 6, 2025 • 3 new comments

• [KD] add uld and jsd

  #2253 commented on Feb 6, 2025 • 2 new comments

• when will add training of deepseek v3？ it`s a big update of llamas

  #2228 commented on Jan 31, 2025 • 0 new comments

• Training with FP8

  #755 commented on Jan 31, 2025 • 0 new comments

• "RuntimeError: Invalid device argument : did you call init? "When setting CUDA_VISIBLE_DEVICES

  #2199 commented on Feb 5, 2025 • 0 new comments

• Very High Loss (~15) and Instability with Previously-working Config From A While Ago

  #2224 commented on Feb 6, 2025 • 0 new comments

• Fix: RL base feature parity

  #2133 commented on Feb 3, 2025 • 0 new comments

• Enable flex attention support

  #2255 commented on Feb 5, 2025 • 0 new comments