Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 10, 2024

• add fast rotary embedding

  

  Ubuntu

  committed
  40ede3a

  Copy full SHA for 40ede3a

Commits on Mar 25, 2024

• add multinode training

  patrick-toulme

  committed
  e736351

  Copy full SHA for e736351

Commits on Mar 20, 2024

• Use llm training. working,

  patrick-toulme

  committed
  08f0eb1

  Copy full SHA for 08f0eb1

• Trainer working end to end

  patrick-toulme

  committed
  ccbe0a6

  Copy full SHA for ccbe0a6

Commits on Mar 19, 2024

• Mergin xujun

  patrick-toulme

  committed
  b8d05c4

  Copy full SHA for b8d05c4

Commits on Mar 15, 2024

• Working vocab parallel embedding. Needs libneruonxla wheel

  patrick-toulme

  committed
  805b34d

  Copy full SHA for 805b34d

• use a simple scaled hidden dim scalar

  

  Ubuntu

  committed
  327cb4f

  Copy full SHA for 327cb4f

• disable fp32 softmax for now

  

  Ubuntu

  committed
  f94e156

  Copy full SHA for f94e156

Commits on Mar 14, 2024

• change the default input partition type to DP

  penxujun

  committed
  35506f8

  Copy full SHA for 35506f8

• change mesh shape for neuron trn1

  penxujun

  committed
  cc9240b

  Copy full SHA for cc9240b

• reduce the model size, sequence length, vocab_size for FAKE dataset testing

  penxujun

  committed
  368130a

  Copy full SHA for 368130a

• init state in cpu first and move it to neuron afterwards

  penxujun

  committed
  8ac9323

  Copy full SHA for 8ac9323

Commits on Mar 12, 2024

• add with_sharding_constraint to attention

  penxujun

  committed
  1a0b9a3

  Copy full SHA for 1a0b9a3

• input data type in fuji

  penxujun

  committed
  47b21f0

  Copy full SHA for 47b21f0

• stop data shuffle

  penxujun

  committed
  d4265dc

  Copy full SHA for d4265dc

Commits on Mar 11, 2024

• Remove fp32 softmax temporarily

  patrick-toulme

  committed
  9c1a133

  Copy full SHA for 9c1a133

• Zero1 and 2D weight sharding working. no more dynamic slice

  patrick-toulme

  committed
  f48bfb0

  Copy full SHA for f48bfb0

• output sharding change required by SP

  penxujun

  committed
  99ef22c

  Copy full SHA for 99ef22c

• 4 layers fuji model test

  penxujun

  committed
  73c64a3

  Copy full SHA for 73c64a3

• add new input data sharding

  penxujun

  committed
  d13bce6

  Copy full SHA for d13bce6

Commits on Mar 8, 2024

• Run 4 layer llama 7b with zero1 and optimizer step in one graph

  patrick-toulme

  committed
  d3b11a9

  Copy full SHA for d3b11a9

Commits on Mar 5, 2024

• Sequence packing working

  patrick-toulme

  committed
  f0c6538

  Copy full SHA for f0c6538

• Optimizer step works but has to be outside of a jit

  patrick-toulme

  committed
  1a85ce2

  Copy full SHA for 1a85ce2

Commits on Mar 4, 2024

• Adds selective checkpointing

  patrick-toulme

  committed
  3663f68

  Copy full SHA for 3663f68

Commits on Feb 29, 2024

• Multinode works tested to 32 nodes. Change in run.slurm

  patrick-toulme

  committed
  f14534c

  Copy full SHA for f14534c

• Perfect 4 layer HLO

  patrick-toulme

  committed
  2f32a4e

  Copy full SHA for 2f32a4e

Commits on Feb 26, 2024

• 4 layer broken in compiler

  patrick-toulme

  committed
  2172c62

  Copy full SHA for 2172c62

Commits on Feb 23, 2024

• Fixed HLO fwd/bwd - initial 4 layer commit

  patrick-toulme

  committed
  6b5b1f9

  Copy full SHA for 6b5b1f9

Commits on Feb 21, 2024

• Full Llama 7B layer fwd bwd

  patrick-toulme

  committed
  569cee9

  Copy full SHA for 569cee9

Commits on Feb 20, 2024

• Working forward pass in tensor parallel, sequence parallel, data parallel

  patrick-toulme

  committed
  bd6b716

  Copy full SHA for bd6b716

• Add scale_key and scale_query in deberta self-attention conversion. (apple#323)

  Show description for 176298a
  tuzhucheng

  authored
  176298a

  Copy full SHA for 176298a

• Adds initial transformer test

  patrick-toulme

  committed
  bbd67b5

  Copy full SHA for bbd67b5

Commits on Feb 19, 2024

• Renames BaseBastionLaunchJob to BaseBastionManagedJob (apple#333)

  Show description for d97c1f3
  ruomingp

  authored
  d97c1f3

  Copy full SHA for d97c1f3

• Removes arg mask from mean_square_error, bilinear_mean_squared_error, and l1_loss. (apple#331)

  Show description for c9f5b19
  ruomingp

  authored
  c9f5b19

  Copy full SHA for c9f5b19

Commits on Feb 18, 2024

• Renames mask to live_targets for cross_entropy_loss and binary_cross_entropy_loss. (apple#329)

  Show description for 2fa2ca0
  ruomingp

  authored
  2fa2ca0

  Copy full SHA for 2fa2ca0

Pagination

Previous

Next