New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix] Fix gemma 3 conv template stop token #3191

Merged

MasterJH5574 merged 1 commit into mlc-ai:main from CharlieFRuan:fix-0327-gemma3-conv

Mar 27, 2025

Contributor

CharlieFRuan commented Mar 27, 2025 •

edited

Loading

Currently for all the gemma3 instruction models, we reused the previous gemma conv template. However, there is a difference in the stop token.

In gemma2, the stop tokens are 1, 107: https://huggingface.co/google/gemma-2-2b-it/blob/main/generation_config.json
In gemma3, the stop tokens are 1, 106: https://huggingface.co/google/gemma-3-1b-it/blob/main/generation_config.json

This PR add a new gemma3_instruction conv template, and will manually update the already-uploaded gemma 3 models on HF.

In gemma3, 107 is \n. So currently whenever the model tries to change to a new line, it ends generation.


          [Fix] Fix gemma 3 conv template stop token

5f7d7ed

Contributor Author

CharlieFRuan commented Mar 27, 2025

cc @MasterJH5574 @joshua-j-hong

MasterJH5574 approved these changes

View reviewed changes

MasterJH5574 merged commit e89a484 into mlc-ai:main

2 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet