fix: GPT OSS Conversion Script Enhancements #42901
Conversation
Rocketknight1
left a comment
Rocketknight1
left a comment
There was a problem hiding this comment.
Conversion script updates look good! I'm a little wary about the change in the core convert_slow_tokenizer.py, though, so I might need a second opinion from an expert there
| with open(chat_template_path, "w") as f: | ||
| json.dump({"chat_template": chat_template}, f, indent=2) | ||
| print("Saving chat template...") | ||
| chat_template_path = os.path.join(save_dir, "chat_template.json") |
There was a problem hiding this comment.
We prefer raw chat_template.jinja in modern conversions!
There was a problem hiding this comment.
Would you like me to change the filename to chat_template.jinja?
There was a problem hiding this comment.
I changed the extension name as requested.
| tokenizer.add_special_tokens( | ||
| [AddedToken(token, normalized=False, special=True) for token in self.extra_special_tokens] | ||
| ) | ||
| if self.extra_special_tokens is not None: | ||
| tokenizer.add_special_tokens( | ||
| [AddedToken(token, normalized=False, special=True) for token in self.extra_special_tokens] | ||
| ) |
There was a problem hiding this comment.
I'm not sure why we need changes in the core code! cc @itazap @ArthurZucker before I can approve this
There was a problem hiding this comment.
It hard-crashes otherwise because self.extra_special_tokens can be None, and is in the conversion script.
There was a problem hiding this comment.
Bumping for @itazap and @ArthurZucker feedback.
There was a problem hiding this comment.
Great! How can we get this over the line? Would love to see this change in Transformers 5.0.0 release.
…pt' into fix_conversion_script
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gpt_oss |
|
View the CircleCI Test Summary for this PR: https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42901&sha=e48a61 |
4 participants
What does this PR do?
GPT OSS Models have a conversion script which allows transforming from the distributed quantized weights to a Hugging Face model in BFloat16. This also converts aspects of the tokenizer.
As a side note, it also fixes a minor bug in the TikToken tokenizer which caused a hard crash when iterating over a NoneType object when converting the tokenizer.
Fixes # (issue)
This resolves an open issue in the Hugging Face Accelerate repository by making the conversion script functional: huggingface/accelerate#3882 (comment)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@ArthurZucker @Cyrilvallez