Commits

Branch selector

 main

User selector

samsja

Datepicker

All time

Commit History

Commits on Nov 26, 2024

• pin pytorch version

  samsja

  authored
  71f5c2b

  Copy full SHA for 71f5c2b

Commits on Sep 12, 2024

• add better instruction

  samsja

  committed
  a5b7236

  Copy full SHA for a5b7236

Commits on Sep 5, 2024

• update example command run training.sh

  samsja

  committed
  d999451

  Copy full SHA for d999451

Commits on Aug 23, 2024

• feat: use wandb auto resuming feature

  samsja

  committed
  ad3a344

  Copy full SHA for ad3a344

• feat: not failing if the ckpt folder does not exist yet

  samsja

  committed
  bb49d16

  Copy full SHA for bb49d16

• feat: allow to pass --ckpt.resume and start from scratch if no ckpt files are present

  samsja

  committed
  49bfa9b

  Copy full SHA for 49bfa9b

Commits on Aug 22, 2024

• fix multi gpu hivemind ckpt resume

  samsja

  committed
  b605e6a

  Copy full SHA for b605e6a

Commits on Aug 21, 2024

• refactor: use init devish mesh for hybrid shard (#24)

  samsja

  authored
  f7ef006

  Copy full SHA for f7ef006

• fix torch compile log act (#23)

  Show description for 8ce08c0
  samsja

  authored
  8ce08c0

  Copy full SHA for 8ce08c0

Commits on Aug 20, 2024

• test: check ckpt resuming hivemind

  samsja

  committed
  35cd120

  Copy full SHA for 35cd120

• test: check ckpt resuming

  samsja

  committed
  af06270

  Copy full SHA for af06270

• feat: add custom logger

  samsja

  committed
  96a0e82

  Copy full SHA for 96a0e82

• refactor(testing): use torchrun during testing

  samsja

  committed
  fb18a5e

  Copy full SHA for fb18a5e

Commits on Aug 19, 2024

• refactor: move code to ckpt_utils.py

  samsja

  committed
  8b7b6a8

  Copy full SHA for 8b7b6a8

• feat: allow to keap only k ckpt

  samsja

  committed
  9cd4a43

  Copy full SHA for 9cd4a43

• feat: auto resume from latest ckpt

  samsja

  committed
  69fdb96

  Copy full SHA for 69fdb96

• refactor: rename and isolated ckpt args

  samsja

  committed
  0b95922

  Copy full SHA for 0b95922

Commits on Aug 12, 2024

• fix training script

  samsja

  committed
  1b3ea8a

  Copy full SHA for 1b3ea8a

• allow to fail when loosing a diloco workers

  samsja

  committed
  eaae47a

  Copy full SHA for eaae47a

• allow training script to automatically start the dht server

  samsja

  committed
  3e1fa45

  Copy full SHA for 3e1fa45

• fix: ckpt

  samsja

  committed
  f2be2f7

  Copy full SHA for f2be2f7

Commits on Aug 9, 2024

• support multi gpu per diloco worker in the run training script

  samsja

  committed
  fdb1acd

  Copy full SHA for fdb1acd

Commits on Aug 8, 2024

• diloco ckpt check fix

  samsja

  committed
  a53b294

  Copy full SHA for a53b294

Commits on Aug 6, 2024

• refactor checkpointing: remove date and run id

  samsja

  committed
  ba63895

  Copy full SHA for ba63895

• refactor: use None instead of literal none in hv compression

  samsja

  committed
  269cf90

  Copy full SHA for 269cf90

Commits on Aug 2, 2024

• update dht server instruction

  samsja

  committed
  8500dba

  Copy full SHA for 8500dba

• add 8bit quant all reduce support

  samsja

  committed
  32395eb

  Copy full SHA for 32395eb

Commits on Jul 19, 2024

• fix grad norm fsdp

  samsja

  committed
  e4e4f86

  Copy full SHA for e4e4f86

Commits on Jul 18, 2024

• add pytest to requirements-dev.txt

  Show description for e8ff410
  samsja

  committed
  e8ff410

  Copy full SHA for e8ff410

• remove hivemind source ignore from ruff

  Show description for 39c47b1
  samsja

  committed
  39c47b1

  Copy full SHA for 39c47b1

Commits on Jul 17, 2024

• add flash attention build to dockerfile

  samsja

  committed
  e66a1ab

  Copy full SHA for e66a1ab

• remove github submodule

  samsja

  committed
  44d6d87

  Copy full SHA for 44d6d87

Commits on Jul 11, 2024

• add arxiv link to readme

  samsja

  authored
  b459953

  Copy full SHA for b459953

• Initial commit

  Show description for 828b643
  

  samsja

  and

  Jackmin801

  committed
  828b643

  Copy full SHA for 828b643