Skip to content

Add support for GLM-Edge and GLM-Edge-V series models #10573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 34 commits into from
Feb 2, 2025
Merged

Add support for GLM-Edge and GLM-Edge-V series models #10573

merged 34 commits into from
Feb 2, 2025

Conversation

piDack
Copy link
Contributor

@piDack piDack commented Nov 29, 2024

This pull request support for the GLM-Edge-Chat 1.5B & 4B and GLM-Edge-V 2B & 5B series of models within the llama.cpp.

Note: The current model pretrain -> gguf only supports using the transformers version 4.47.0.dev0.

@github-actions github-actions bot added testing Everything test related examples python python script changes labels Nov 29, 2024
ngxson
ngxson reviewed Nov 29, 2024
View reviewed changes
@arch-btw
Copy link
Contributor

Works great.

./llama-llava-cli -m ggml-model-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --temp 0.1 --image bee.jpg -p "<|system|>\n You are a helpful AI assistant. <image><|user|>\n What is in the image? <|assistant|>\n"

glm

@piDack
Copy link
Contributor Author

piDack commented Dec 19, 2024

Is there anyone available to review the code?

@arch-btw
Copy link
Contributor

@piDack As this would be adding support for GlmForCausalLM for these vision models, I'm curious to know if we could create a more modular or generic implementation that could also be used for the other GlmForCausalLM model(s)?

I'm asking because glm-4-9b-chat-hf is currently broken with the new transformers-only implementation:

python convert_hf_to_gguf.py /home/Models/glm-4-9b-chat-hf --outtype f32
INFO:hf-to-gguf:Loading model: glm-4-9b-chat-hf
ERROR:hf-to-gguf:Model GlmForCausalLM is not supported

The version with the custom python files still works but if we're moving away from that (related discussion), it might be best to support GlmForCausalLM in general.

Are there any parts of this PR that could be refactored or generalized for broader applicability so that we can support both and maybe upcoming models? Thank you.

@piDack
Copy link
Contributor Author

piDack commented Jan 7, 2025

@piDack As this would be adding support for GlmForCausalLM for these vision models, I'm curious to know if we could create a more modular or generic implementation that could also be used for the other GlmForCausalLM model(s)?

I'm asking because glm-4-9b-chat-hf is currently broken with the new transformers-only implementation:

python convert_hf_to_gguf.py /home/Models/glm-4-9b-chat-hf --outtype f32
INFO:hf-to-gguf:Loading model: glm-4-9b-chat-hf
ERROR:hf-to-gguf:Model GlmForCausalLM is not supported

The version with the custom python files still works but if we're moving away from that (related discussion), it might be best to support GlmForCausalLM in general.

Are there any parts of this PR that could be refactored or generalized for broader applicability so that we can support both and maybe upcoming models? Thank you.

I will tried to do it

@arch-btw arch-btw mentioned this pull request Jan 16, 2025
Closed
ngxson
ngxson reviewed Jan 29, 2025
View reviewed changes
ngxson
ngxson reviewed Jan 29, 2025
View reviewed changes
@piDack piDack requested a review from ngxson January 30, 2025 12:50
ngxson
ngxson reviewed Jan 30, 2025
View reviewed changes
Copy link
Collaborator

@ngxson ngxson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

many places still having inconsistent style

ngxson
ngxson reviewed Jan 30, 2025
View reviewed changes
ngxson
ngxson reviewed Jan 30, 2025
View reviewed changes
ngxson
ngxson reviewed Jan 30, 2025
View reviewed changes
ngxson
ngxson reviewed Jan 30, 2025
View reviewed changes
@ngxson
Copy link
Collaborator

ngxson commented Jan 30, 2025 •
edited
Loading

@ggerganov Could you have a look quickly on llama.cpp? I gave my approval for the rest of changes.

ggerganov
ggerganov reviewed Jan 31, 2025
View reviewed changes
@piDack piDack requested a review from ggerganov February 1, 2025 01:42
@piDack
Copy link
Contributor Author

piDack commented Feb 2, 2025

I believe it's ready to be merged into the master branch.

ggerganov
ggerganov approved these changes Feb 2, 2025
View reviewed changes
@ggerganov ggerganov merged commit 0cec062 into ggml-org:master Feb 2, 2025
47 checks passed
@ggerganov ggerganov mentioned this pull request Feb 2, 2025
Closed
21 tasks
tinglou pushed a commit to tinglou/llama.cpp that referenced this pull request Feb 13, 2025
@piDack @ngxson
@ggerganov @tinglou
llama : add support for GLM-Edge and GLM-Edge-V series models (ggml-o…
6b5f5f8 
…rg#10573)

* add glm edge chat model

* use config partial_rotary_factor as rope ratio

* support for glm edge model

* vision model support

* remove debug info

* fix format

* llava.cpp trailing whitespace

* remove unused AutoTokenizer

* Update src/llama.cpp for not contain <|end|> or </s>

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add edge template

* fix chat template

* fix confict

* fix confict

* fix ci err

* fix format err

* fix template err

* 9b hf chat support

* format

* format clip.cpp

* fix format

* Apply suggestions from code review

* Apply suggestions from code review

* Update examples/llava/clip.cpp

* fix format

* minor : style

---------

Co-authored-by: liyuhang <yuhang.li@zhipuai.cn>
Co-authored-by: piDack <pcdack@hotmail.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: liyuhang <yuhang.li@aminer.cn>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
orca-zhang pushed a commit to orca-zhang/llama.cpp that referenced this pull request Feb 26, 2025
@piDack @ngxson
@ggerganov @orca-zhang
llama : add support for GLM-Edge and GLM-Edge-V series models (ggml-o…
e26e218 
…rg#10573)

* add glm edge chat model

* use config partial_rotary_factor as rope ratio

* support for glm edge model

* vision model support

* remove debug info

* fix format

* llava.cpp trailing whitespace

* remove unused AutoTokenizer

* Update src/llama.cpp for not contain <|end|> or </s>

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add edge template

* fix chat template

* fix confict

* fix confict

* fix ci err

* fix format err

* fix template err

* 9b hf chat support

* format

* format clip.cpp

* fix format

* Apply suggestions from code review

* Apply suggestions from code review

* Update examples/llava/clip.cpp

* fix format

* minor : style

---------

Co-authored-by: liyuhang <yuhang.li@zhipuai.cn>
Co-authored-by: piDack <pcdack@hotmail.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: liyuhang <yuhang.li@aminer.cn>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Feb 26, 2025
@piDack @ngxson
@ggerganov @NeoZhangJianyu
llama : add support for GLM-Edge and GLM-Edge-V series models (ggml-o…
be4e8dd 
…rg#10573)

* add glm edge chat model

* use config partial_rotary_factor as rope ratio

* support for glm edge model

* vision model support

* remove debug info

* fix format

* llava.cpp trailing whitespace

* remove unused AutoTokenizer

* Update src/llama.cpp for not contain <|end|> or </s>

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add edge template

* fix chat template

* fix confict

* fix confict

* fix ci err

* fix format err

* fix template err

* 9b hf chat support

* format

* format clip.cpp

* fix format

* Apply suggestions from code review

* Apply suggestions from code review

* Update examples/llava/clip.cpp

* fix format

* minor : style

---------

Co-authored-by: liyuhang <yuhang.li@zhipuai.cn>
Co-authored-by: piDack <pcdack@hotmail.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: liyuhang <yuhang.li@aminer.cn>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
mglambda pushed a commit to mglambda/llama.cpp that referenced this pull request Mar 8, 2025
@piDack @ngxson
@ggerganov @mglambda
llama : add support for GLM-Edge and GLM-Edge-V series models (ggml-o…
54b97b5 
…rg#10573)

* add glm edge chat model

* use config partial_rotary_factor as rope ratio

* support for glm edge model

* vision model support

* remove debug info

* fix format

* llava.cpp trailing whitespace

* remove unused AutoTokenizer

* Update src/llama.cpp for not contain <|end|> or </s>

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add edge template

* fix chat template

* fix confict

* fix confict

* fix ci err

* fix format err

* fix template err

* 9b hf chat support

* format

* format clip.cpp

* fix format

* Apply suggestions from code review

* Apply suggestions from code review

* Update examples/llava/clip.cpp

* fix format

* minor : style

---------

Co-authored-by: liyuhang <yuhang.li@zhipuai.cn>
Co-authored-by: piDack <pcdack@hotmail.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: liyuhang <yuhang.li@aminer.cn>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
@Tianyue-Zhao Tianyue-Zhao mentioned this pull request Apr 17, 2025
Open
4 tasks
@ngxson ngxson mentioned this pull request Apr 23, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggerganov ggerganov ggerganov approved these changes

@ngxson ngxson ngxson approved these changes

Assignees
No one assigned
Labels
examples python python script changes testing Everything test related
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@piDack @arch-btw @ngxson @ggerganov