Rename trainer arg `tokenizer` to `processing_class` #2162

qgallouedec · 2024-10-03T09:52:42Z

What does this PR do?

Follows huggingface/transformers#32385
Fixes #2161

Ensure backward compatibility for DPO and SFT only

>>> from datasets import load_dataset
>>> from transformers import AutoModelForCausalLM, AutoTokenizer
>>> from trl import DPOConfig, DPOTrainer
>>> model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
>>> dataset = load_dataset("trl-lib/Capybara-Preferences", split="train")
>>> training_args = DPOConfig(output_dir="Qwen2.5-0.5B-DPO")
>>> trainer = DPOTrainer(model=model, args=training_args, train_dataset=dataset, tokenizer=tokenizer)
/fsx/qgallouedec/miniconda3/envs/trl/lib/python3.11/site-packages/huggingface_hub/utils/_deprecation.py:101: FutureWarning: `tokenizer` is deprecated and will be removed in version 0.14.0 for `DPOTrainer.__init__`. Use `processing_class` instead.
  return f(*args, **kwargs)

TODO

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a GitHub issue? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

HuggingFaceDocBuilderDev · 2024-10-03T09:56:17Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

…ggingface/trl into tokenizer_to_processing_class

trl/trainer/dpo_trainer.py

trl/trainer/ppov2_trainer.py

alvarobartt · 2024-10-04T14:27:29Z

trl/trainer/ppov2_trainer.py

@@ -599,7 +604,7 @@ def repeat_generator():

    def generate_completions(self, sampling: bool = False):
        args = self.args
-        tokenizer = self.tokenizer
+        processing_class = self.processing_class


I may be missing something but is this required? Cannot we just use self.processing_class?

You're right. Same for args. It will probably need some refactoring in the future

I've seen that in other places too so maybe there's a rationale that I don't see for that? Not sure, but sure we'll keep it in mind

trl/trainer/rloo_trainer.py

Co-authored-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

yananchen1989 · 2024-10-08T17:03:14Z

Traceback (most recent call last):
File "/workspace/trl/examples/scripts/sft.py", line 93, in
trainer = SFTTrainer(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
return f(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/deprecation.py", line 165, in wrapped_func
return func(*args, **kwargs)
File "/workspace/trl/trl/trainer/sft_trainer.py", line 409, in init
super().init(
TypeError: Trainer.init() got an unexpected keyword argument 'processing_class'

yananchen1989 · 2024-10-08T17:06:03Z

trl version: 0.12.0.dev0

BUILDERlym · 2024-10-08T20:09:04Z

Traceback (most recent call last):
File "/workspace/trl/examples/scripts/sft.py", line 93, in
trainer = SFTTrainer(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
return f(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/deprecation.py", line 165, in wrapped_func
return func(*args, **kwargs)
File "/workspace/trl/trl/trainer/sft_trainer.py", line 409, in init
super().init(
TypeError: Trainer.init() got an unexpected keyword argument 'processing_class'

same issue. checked the local source code, indeed no argument 'processing_class'. why's that?
seems like current version still uses 'tokenizer'

kashif · 2024-10-08T20:16:48Z

you need to use the main version of transformers

BUILDERlym · 2024-10-08T20:19:14Z

you need to use the main version of transformers

got it, thanks

we use positional args to obtain model and optimizer, however, this has the un-needed tokenizer argument between them due to recent changes, the tokenizer arg is now renamed to processing_class, see: + huggingface/trl#2162 + huggingface/transformers#32385 leading to unexpected breakdown of scanner the line relevant to us is here: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_callback.py#L523 since we anyway don't depend on this arg, switch out to using model and opt from the kwargs

update doc

e30f39f

qgallouedec linked an issue Oct 3, 2024 that may be closed by this pull request

AttributeError: property 'tokenizer' of 'DPOTrainer' object has no setter #2161

Closed

4 tasks

qgallouedec added 4 commits October 3, 2024 10:19

bco

0d8a793

bco

f03185f

cpo

d1c5c3c

revert some cpo changes

35564bf

qgallouedec mentioned this pull request Oct 3, 2024

🩹 [Hotfix] Add setter for tokenizer #2163

Merged

5 tasks

qgallouedec and others added 17 commits October 3, 2024 15:32

dpo

42cf70b

0.14

802e1cb

online dpo

e61afce

Merge branch 'main' into tokenizer_to_processing_class

2d2f350

Merge branch 'tokenizer_to_processing_class' of https://github.com/hu…

0625889

…ggingface/trl into tokenizer_to_processing_class

gkd

bb57af7

explicit args gkd

8aad102

kto

5cc7ef3

Merge branch 'main' into tokenizer_to_processing_class

6609e07

drop deprecated beta

c1bdfab

kto type hint

560e61b

nash-md

97af75b

orpo

a8fba85

reward

cad7c1e

sft

dc10655

Merge branch 'main' into tokenizer_to_processing_class

8055683

xpo

753a79a

qgallouedec marked this pull request as ready for review October 4, 2024 12:24

qgallouedec requested review from alvarobartt, kashif, edbeeching and lewtun October 4, 2024 12:35

qgallouedec added 6 commits October 4, 2024 12:47

iterative sft

d8ca7c0

correct type gkd

f70c950

rloo

aff4853

fix gkd import

6ddef1d

ppo

9d662f2

sft stack llama

bc33bf6

alvarobartt reviewed Oct 4, 2024

View reviewed changes

qgallouedec and others added 6 commits October 4, 2024 16:36

Update trl/trainer/dpo_trainer.py

4b1912f

Co-authored-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

Update trl/trainer/rloo_trainer.py

1a4b002

Co-authored-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

Update trl/trainer/ppov2_trainer.py

cf75054

Co-authored-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

Update trl/trainer/rloo_trainer.py

9eabad9

Co-authored-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

Merge branch 'main' into tokenizer_to_processing_class

f24e3eb

update cpo

19e949a

edbeeching approved these changes Oct 7, 2024

View reviewed changes

alvarobartt approved these changes Oct 7, 2024

View reviewed changes

qgallouedec merged commit 47d08a9 into main Oct 7, 2024
9 of 10 checks passed

qgallouedec deleted the tokenizer_to_processing_class branch October 7, 2024 07:39

qgallouedec mentioned this pull request Oct 28, 2024

"tokenizer" attribute is now deprecated on Transformers Trainer class #2290

Closed

qgallouedec mentioned this pull request Nov 11, 2024

👈 Add tokenizer arg back and add deprecation guidelines #2348

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rename trainer arg `tokenizer` to `processing_class` #2162

Rename trainer arg `tokenizer` to `processing_class` #2162

qgallouedec commented Oct 3, 2024 •

edited

Loading

HuggingFaceDocBuilderDev commented Oct 3, 2024

alvarobartt Oct 4, 2024

qgallouedec Oct 4, 2024

alvarobartt Oct 4, 2024

yananchen1989 commented Oct 8, 2024

yananchen1989 commented Oct 8, 2024

BUILDERlym commented Oct 8, 2024 •

edited

Loading

kashif commented Oct 8, 2024

BUILDERlym commented Oct 8, 2024

Rename trainer arg tokenizer to processing_class #2162

Rename trainer arg tokenizer to processing_class #2162

Conversation

qgallouedec commented Oct 3, 2024 • edited Loading

What does this PR do?

TODO

Before submitting

Who can review?

HuggingFaceDocBuilderDev commented Oct 3, 2024

alvarobartt Oct 4, 2024

Choose a reason for hiding this comment

qgallouedec Oct 4, 2024

Choose a reason for hiding this comment

alvarobartt Oct 4, 2024

Choose a reason for hiding this comment

yananchen1989 commented Oct 8, 2024

yananchen1989 commented Oct 8, 2024

BUILDERlym commented Oct 8, 2024 • edited Loading

kashif commented Oct 8, 2024

BUILDERlym commented Oct 8, 2024

Rename trainer arg `tokenizer` to `processing_class` #2162

Rename trainer arg `tokenizer` to `processing_class` #2162

qgallouedec commented Oct 3, 2024 •

edited

Loading

BUILDERlym commented Oct 8, 2024 •

edited

Loading