Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Padding free dpo #2437

Open
wants to merge 58 commits into
base: main
Choose a base branch
from
Open

Padding free dpo #2437

Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
ca99954
added eos token for ppotrainer
dame-cell Nov 30, 2024
fe1d5f6
remove the unnecessary stuff
dame-cell Nov 30, 2024
b15c635
Update ppo_config.py
dame-cell Nov 30, 2024
1bcb3a4
remove redundant EOS token fallback
dame-cell Dec 1, 2024
2ef2b24
remove redundant EOS token fallback
dame-cell Dec 1, 2024
42a0f73
remove some unnecessary tests stuff
dame-cell Dec 1, 2024
6130a91
added tests and update concatenated_inputs
dame-cell Dec 4, 2024
6732ed2
return only list and also a lot to do
dame-cell Dec 4, 2024
ce67292
padding free not tested but getting closer
dame-cell Dec 10, 2024
91e40aa
rebase and also reevaluate my approach
dame-cell Dec 10, 2024
8a34cb5
merge main
dame-cell Dec 10, 2024
1d38632
fix identation
dame-cell Dec 10, 2024
814d69e
better tests
dame-cell Dec 10, 2024
7dae607
concatenated_forward now supports padding_free
dame-cell Dec 10, 2024
1a59d74
collator now does not return attention masks
dame-cell Dec 10, 2024
562c52e
postion ids and no attention mask works
dame-cell Dec 11, 2024
d194054
update concatenated forward to support padding_free
dame-cell Dec 11, 2024
3855851
update concatenated forward to support padding_free
dame-cell Dec 11, 2024
d9adbfb
Merge branch 'main' into padding_free_dpo
dame-cell Dec 11, 2024
1145006
grad accumalation tests
dame-cell Dec 13, 2024
24f73a4
Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl i…
dame-cell Dec 13, 2024
f6bd9e1
Resolved merge conflict in ppo_trainer.py
dame-cell Dec 13, 2024
bbd99cf
Resolved merge conflict in ppo_trainer.py
dame-cell Dec 13, 2024
ba4969d
Resolved merge conflict in ppo_trainer.py
dame-cell Dec 13, 2024
187b1e5
Resolved merge conflict in ppo_trainer.py
dame-cell Dec 13, 2024
1d9ce3e
fix identation
dame-cell Dec 13, 2024
8a974cc
comments update
dame-cell Dec 13, 2024
7f0298b
fix some small issue
dame-cell Dec 13, 2024
2900275
fix some small issue
dame-cell Dec 13, 2024
58e779a
fix some small issue
dame-cell Dec 13, 2024
6a1e251
fix some small issue
dame-cell Dec 13, 2024
f92e056
update concatenate_forward to support padding_fre
dame-cell Dec 13, 2024
457e3a1
fix some small issue
dame-cell Dec 13, 2024
f1789f4
fix some small issue
dame-cell Dec 13, 2024
0103728
So we need to make sure to correctlty handle the list
dame-cell Dec 13, 2024
5837faa
by correclty updatuing concatenated_forward it works now
dame-cell Dec 13, 2024
0321c1d
refactoring concatenated_forward and batched same length seq for padd…
dame-cell Dec 14, 2024
a6e2163
update
dame-cell Dec 14, 2024
1328fc3
padding_free in concatenated_forward and update_test
dame-cell Dec 17, 2024
51a2cc6
Merge branch 'main' into padding_free_dpo
dame-cell Dec 17, 2024
986ed71
padding_free in concatenated_forward and update_test
dame-cell Dec 17, 2024
570b79a
padding_free in concatenated_forward and update_test
dame-cell Dec 17, 2024
9dd9564
Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl i…
dame-cell Dec 17, 2024
b781876
Merge branch 'main' into padding_free_dpo
dame-cell Dec 18, 2024
525ecb2
Merge branch 'main' into padding_free_dpo
dame-cell Dec 19, 2024
c8ce9c8
Merge branch 'main' into padding_free_dpo
dame-cell Dec 19, 2024
b7fad73
Reverted PPO trainer to original version and updated DPO files
dame-cell Dec 19, 2024
55cd219
Merge branch 'padding_free_dpo' of https://github.com/dame-cell/trl i…
dame-cell Dec 19, 2024
5e8df69
Updated DPO files
dame-cell Dec 19, 2024
ba1ded1
Merge branch 'main' into padding_free_dpo
dame-cell Dec 20, 2024
0784202
Merge branch 'main' into padding_free_dpo
dame-cell Dec 21, 2024
ddfed7c
update test_dpo_trainer.py
dame-cell Dec 21, 2024
955c7e8
update dpo_trainer.py
dame-cell Dec 21, 2024
ba4356e
update dpo_trainer.py
dame-cell Dec 21, 2024
c61abb5
update dpo_trainer.py
dame-cell Dec 21, 2024
64e9909
update dpo_trainer.py
dame-cell Dec 22, 2024
68186e7
Merge branch 'main' into padding_free_dpo
dame-cell Dec 22, 2024
90f50e4
Merge branch 'main' into padding_free_dpo
dame-cell Jan 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update concatenate_forward to support padding_fre
  • Loading branch information
@dame-cell
dame-cell committed Dec 13, 2024
commit f92e0561214048735862ec1613b207d1b8dc80d9
Loading