Code

Padding free dpo#2437

Code

Open

dame-cell wants to merge 57 commits intohuggingface:mainhuggingface/trl:mainfrom dame-cell:padding_free_dpodame-cell/trl:padding_free_dpoCopy head branch name to clipboard

+212-83

Commits

Commits on Nov 30, 2024

• added eos token for ppotrainer

  dame-cell

  committed
  ca99954

• remove the unnecessary stuff

  dame-cell

  authored
  fe1d5f6

• Update ppo_config.py

  dame-cell

  authored
  b15c635

Commits on Dec 1, 2024

• remove redundant EOS token fallback

  dame-cell

  committed
  1bcb3a4

• remove redundant EOS token fallback

  dame-cell

  committed
  2ef2b24

• remove some unnecessary tests stuff

  dame-cell

  committed
  42a0f73

Commits on Dec 4, 2024

• added tests and update concatenated_inputs

  dame-cell

  committed
  6130a91

• return only list and also a lot to do

  dame-cell

  authored
  6732ed2

Commits on Dec 10, 2024

• padding free not tested but getting closer

  dame-cell

  committed
  ce67292

• rebase and also reevaluate my approach

  dame-cell

  committed
  91e40aa

• merge main

  dame-cell

  committed
  8a34cb5

• fix identation

  dame-cell

  committed
  1d38632

• better tests

  dame-cell

  committed
  814d69e

• concatenated_forward now supports padding_free

  dame-cell

  committed
  7dae607

• collator now does not return attention masks

  dame-cell

  committed
  1a59d74

Commits on Dec 11, 2024

• postion ids and no attention mask works

  dame-cell

  committed
  562c52e

• update concatenated forward to support padding_free

  dame-cell

  committed
  d194054

• update concatenated forward to support padding_free

  dame-cell

  committed
  3855851

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  d9adbfb

Commits on Dec 13, 2024

• grad accumalation tests

  dame-cell

  committed
  1145006

• Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl into padding_free_dpo

  dame-cell

  committed
  24f73a4

• Resolved merge conflict in ppo_trainer.py

  dame-cell

  committed
  f6bd9e1

• Resolved merge conflict in ppo_trainer.py

  dame-cell

  committed
  bbd99cf

• Resolved merge conflict in ppo_trainer.py

  dame-cell

  committed
  ba4969d

• Resolved merge conflict in ppo_trainer.py

  dame-cell

  committed
  187b1e5

• fix identation

  dame-cell

  committed
  1d9ce3e

• comments update

  dame-cell

  committed
  8a974cc

• fix some small issue

  dame-cell

  committed
  7f0298b

• fix some small issue

  dame-cell

  committed
  2900275

• fix some small issue

  dame-cell

  committed
  58e779a

• fix some small issue

  dame-cell

  committed
  6a1e251

• update concatenate_forward to support padding_fre

  dame-cell

  committed
  f92e056

• fix some small issue

  dame-cell

  committed
  457e3a1

• fix some small issue

  dame-cell

  committed
  f1789f4

• So we need to make sure to correctlty handle the list

  dame-cell

  committed
  0103728

• by correclty updatuing concatenated_forward it works now

  dame-cell

  committed
  5837faa

Commits on Dec 14, 2024

• refactoring concatenated_forward and batched same length seq for padding_free

  dame-cell

  committed
  0321c1d

• update

  dame-cell

  committed
  a6e2163

Commits on Dec 17, 2024

• padding_free in concatenated_forward and update_test

  dame-cell

  committed
  1328fc3

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  51a2cc6

• padding_free in concatenated_forward and update_test

  dame-cell

  committed
  986ed71

• padding_free in concatenated_forward and update_test

  dame-cell

  committed
  570b79a

• Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl into padding_free_dpo

  dame-cell

  committed
  9dd9564

Commits on Dec 18, 2024

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  b781876

Commits on Dec 19, 2024

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  525ecb2

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  c8ce9c8

• Reverted PPO trainer to original version and updated DPO files

  dame-cell

  committed
  b7fad73

• Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl into padding_free_dpo

  dame-cell

  committed
  55cd219

• Updated DPO files

  dame-cell

  committed
  5e8df69

Commits on Dec 20, 2024

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  ba1ded1

Commits on Dec 21, 2024

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  0784202

• update test_dpo_trainer.py

  dame-cell

  committed
  ddfed7c

• update dpo_trainer.py

  dame-cell

  committed
  955c7e8

• update dpo_trainer.py

  dame-cell

  committed
  ba4356e

• update dpo_trainer.py

  dame-cell

  committed
  c61abb5

Commits on Dec 22, 2024

• update dpo_trainer.py

  dame-cell

  committed
  64e9909

• Merge branch 'main' into padding_free_dpo

  dame-cell

  authored
  68186e7