Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛠️ Update tests and fix PPO #2463

Merged
merged 13 commits into from
Dec 12, 2024
Merged

🛠️ Update tests and fix PPO #2463

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
bd7494d
[bugfix] critic not update
NINGBENZHE Dec 9, 2024
8066364
Update ppo_trainer.py
NINGBENZHE Dec 9, 2024
65428b7
Update ppo_trainer.py
NINGBENZHE Dec 9, 2024
f2c4d79
Merge branch 'main' into main
kashif Dec 11, 2024
450c847
add failing test
kashif Dec 11, 2024
bcc47fa
test both policy and critic
kashif Dec 12, 2024
20ccd53
Merge remote-tracking branch 'NINGBENZHE/main' into ppo-test
kashif Dec 12, 2024
2699ddb
formatting
kashif Dec 12, 2024
216116e
fix tests
kashif Dec 12, 2024
f0ad702
formatting
kashif Dec 12, 2024
7f1e4e9
Update tests/test_ppo_trainer.py
kashif Dec 12, 2024
00936a6
fix test
qgallouedec Dec 12, 2024
f2005ec
Merge branch 'ppo-test' of https://github.com/kashif/trl into pr/kash…
qgallouedec Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
formatting
  • Loading branch information
@kashif
kashif committed Dec 12, 2024
commit 2699ddb7ebc5b3123ab55a7a428b648d26d10713
8 changes: 2 additions & 6 deletions trl/trainer/ppo_trainer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,9 +271,7 @@ def __init__(
# sync random states for DataLoader(shuffle=True) before `accelerator.prepare`
# see https://gist.github.com/vwxyzjn/2581bff1e48e185e0b85b6dfe1def79c
torch.manual_seed(args.seed)
self.model, self.optimizer, self.dataloader = accelerator.prepare(
self.model, self.optimizer, self.dataloader
)
self.model, self.optimizer, self.dataloader = accelerator.prepare(self.model, self.optimizer, self.dataloader)
torch.manual_seed(self.local_seed) # reset the local seed again

self.eval_dataloader = DataLoader(
Expand Down Expand Up @@ -749,9 +747,7 @@ def create_model_card(
if not self.is_world_process_zero():
return

if hasattr(self.model.config, "_name_or_path") and not os.path.isdir(
self.model.config._name_or_path
):
if hasattr(self.model.config, "_name_or_path") and not os.path.isdir(self.model.config._name_or_path):
base_model = self.model.config._name_or_path
else:
base_model = None
Expand Down
Loading