Skip to content

Update tokenizer parameter in sfttrainer across multiple examples - #2664

Merged
githubnemo merged 5 commits into
huggingface:mainfrom
gapsong:sfttrainer_update
Jul 23, 2025
Merged

Update tokenizer parameter in sfttrainer across multiple examples#2664
githubnemo merged 5 commits into
huggingface:mainfrom
gapsong:sfttrainer_update

Conversation

@gapsong

@gapsong gapsong commented Jul 22, 2025

Copy link
Copy Markdown
Contributor

Update tokenizer parameter in sfttrainer across multiple examples
#2663

@gapsong

gapsong commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

@BenjaminBossan

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@githubnemo githubnemo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix :)

Generally looks good but I think instances of Trainer/Seq2SeqTrainer need updating as well, I found these but there may be more:

  • docs/source/quicktour.md
  • docs/source/task_guides/lora_based_methods.md
  • examples/conditional_generation/peft_prompt_tuning_seq2seq_with_generate.ipynb

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@gapsong

gapsong commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

@githubnemo

@githubnemo

Copy link
Copy Markdown
Collaborator

@gapsong check git grep 'tokenizer=', I think there are a few more occurences

- Updated the Trainer initialization in corda_finetuning.py to use processing_class instead of tokenizer.
- Changed the execution_count to null in image_classification_peft_lora.ipynb.
- Modified the tokenizer parameter to processing_class in image_classification_peft_lora.ipynb.
- Adjusted the tokenizer parameter to processing_class in peft_bnb_whisper_large_v2_training.ipynb.
- Updated the README.md in lorafa_finetune to reflect the change from tokenizer to processing_class in Trainer initialization.
@gapsong

gapsong commented Jul 23, 2025

Copy link
Copy Markdown
Contributor Author

Ok found some more. The rest are from GPTQconfigs and datasetcollators.
@githubnemo

@githubnemo githubnemo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! :)

@githubnemo
githubnemo merged commit a795199 into huggingface:main Jul 23, 2025
11 of 14 checks passed
BenjaminBossan pushed a commit to BenjaminBossan/peft that referenced this pull request Jul 28, 2025
…ggingface#2664)

* REFAC Update tokenizer parameter to processing_class in SFTTrainer instances across multiple examples

* REFAC Replace tokenizer parameter with processing_class in Trainer instances across documentation and examples

* Refactor tokenizer parameter to processing_class in various examples

- Updated the Trainer initialization in corda_finetuning.py to use processing_class instead of tokenizer.
- Changed the execution_count to null in image_classification_peft_lora.ipynb.
- Modified the tokenizer parameter to processing_class in image_classification_peft_lora.ipynb.
- Adjusted the tokenizer parameter to processing_class in peft_bnb_whisper_large_v2_training.ipynb.
- Updated the README.md in lorafa_finetune to reflect the change from tokenizer to processing_class in Trainer initialization.

* REFAC Update tokenizer parameter to processing_class in Seq2SeqTrainer instantiation

* REFAC Replace tokenizer parameter with processing_class in README and notebook examples
iambogeumkim pushed a commit to iambogeumkim/peft that referenced this pull request Jun 21, 2026
…ggingface#2664)

* REFAC Update tokenizer parameter to processing_class in SFTTrainer instances across multiple examples

* REFAC Replace tokenizer parameter with processing_class in Trainer instances across documentation and examples

* Refactor tokenizer parameter to processing_class in various examples

- Updated the Trainer initialization in corda_finetuning.py to use processing_class instead of tokenizer.
- Changed the execution_count to null in image_classification_peft_lora.ipynb.
- Modified the tokenizer parameter to processing_class in image_classification_peft_lora.ipynb.
- Adjusted the tokenizer parameter to processing_class in peft_bnb_whisper_large_v2_training.ipynb.
- Updated the README.md in lorafa_finetune to reflect the change from tokenizer to processing_class in Trainer initialization.

* REFAC Update tokenizer parameter to processing_class in Seq2SeqTrainer instantiation

* REFAC Replace tokenizer parameter with processing_class in README and notebook examples
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants