Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GLM-4 and Later GLM Model (Draft) #31977

Closed
wants to merge 86 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
9cf74d7
add GLM-4
zRzRzRzRzRzRzR Jul 11, 2024
bef7fd9
GLM-4 FastTokenizer
zRzRzRzRzRzRzR Jul 11, 2024
c986fac
tokenizer fix
zRzRzRzRzRzRzR Jul 11, 2024
2da5d32
rename
zRzRzRzRzRzRzR Jul 11, 2024
675e7a1
pad token
zRzRzRzRzRzRzR Jul 11, 2024
304e4ef
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 11, 2024
0b241f2
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 12, 2024
fa44041
Fix past_key_values
duzx16 Jul 14, 2024
24dec6b
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 14, 2024
5d2bf5e
Merge branch 'glm-4' of github.com:zRzRzRzRzRzRzR/transformers into g…
duzx16 Jul 14, 2024
63d49c9
Fix flash attention
duzx16 Jul 14, 2024
0a5adf3
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 15, 2024
51cbf5d
add update
zRzRzRzRzRzRzR Jul 15, 2024
86b5004
Merge branch 'glm-4' of https://github.com/zRzRzRzRzRzRzR/transformer…
zRzRzRzRzRzRzR Jul 15, 2024
9a553e5
test with glm
zRzRzRzRzRzRzR Jul 15, 2024
4d45b21
fix test
zRzRzRzRzRzRzR Jul 15, 2024
85cfe41
add discription
zRzRzRzRzRzRzR Jul 15, 2024
860c7ee
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 15, 2024
c83ec2d
update glm
zRzRzRzRzRzRzR Jul 16, 2024
2608010
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 16, 2024
1719000
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 18, 2024
3f0452e
rewrite tokenizer
zRzRzRzRzRzRzR Jul 18, 2024
33d2ca3
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 19, 2024
084988e
fix some test
zRzRzRzRzRzRzR Jul 19, 2024
0cb1531
fix testing
zRzRzRzRzRzRzR Jul 19, 2024
e49718f
Fix RMSNorm initialization
duzx16 Jul 20, 2024
a362206
Fix position ids when passing input_embeds
duzx16 Jul 20, 2024
08b43d9
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 20, 2024
3c5322d
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 23, 2024
dd06993
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 24, 2024
8cc0381
Fix dtype error
duzx16 Jul 24, 2024
a35997e
Merge branch 'glm-4' of github.com:zRzRzRzRzRzRzR/transformers into g…
duzx16 Jul 24, 2024
621d32f
Fix output_layer for classification models
duzx16 Jul 24, 2024
48d1704
fix gradient
zRzRzRzRzRzRzR Jul 24, 2024
5881ed5
remove some skip test
zRzRzRzRzRzRzR Jul 24, 2024
c920ad9
fix small test
zRzRzRzRzRzRzR Jul 24, 2024
21781b3
Fix prepare_inputs_for_generation
duzx16 Jul 24, 2024
9599200
Merge branch 'glm-4' of github.com:zRzRzRzRzRzRzR/transformers into g…
duzx16 Jul 24, 2024
a9b1d0d
fix
zRzRzRzRzRzRzR Jul 25, 2024
0631615
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 25, 2024
9f33751
add converter
zRzRzRzRzRzRzR Jul 25, 2024
2663a13
fix PEP 8
zRzRzRzRzRzRzR Jul 25, 2024
aad19db
remove test
zRzRzRzRzRzRzR Jul 25, 2024
1e9183c
index
zRzRzRzRzRzRzR Jul 25, 2024
e8b90a1
fix doctested
zRzRzRzRzRzRzR Jul 25, 2024
65e1996
remove init
zRzRzRzRzRzRzR Jul 25, 2024
266ce77
fix copied error
zRzRzRzRzRzRzR Jul 25, 2024
cd9c304
fix mlp differ
zRzRzRzRzRzRzR Jul 25, 2024
ba30dad
fix copied eerror
zRzRzRzRzRzRzR Jul 25, 2024
afb1423
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 25, 2024
48aaba1
test_hidden_states_output = False
zRzRzRzRzRzRzR Jul 25, 2024
33d976f
Merge branch 'glm-4' of https://github.com/zRzRzRzRzRzRzR/transformer…
zRzRzRzRzRzRzR Jul 25, 2024
0675202
fix
zRzRzRzRzRzRzR Jul 25, 2024
19b0939
Update modeling_glm.py
zRzRzRzRzRzRzR Jul 25, 2024
b2b6c0f
Update __init__.py
zRzRzRzRzRzRzR Jul 25, 2024
6760791
fix glm type error
zRzRzRzRzRzRzR Jul 25, 2024
515d9d9
fix
zRzRzRzRzRzRzR Jul 25, 2024
9951c92
ruff problem
zRzRzRzRzRzRzR Jul 25, 2024
547ac95
Update convert_slow_tokenizer.py
zRzRzRzRzRzRzR Jul 25, 2024
9ba6cf7
Add explanations in English
zRzRzRzRzRzRzR Jul 25, 2024
9fb6405
reformate
zRzRzRzRzRzRzR Jul 25, 2024
e37bb49
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 25, 2024
25aec29
Update configuration_glm.py
zRzRzRzRzRzRzR Jul 25, 2024
58d344a
Merge branch 'glm-4' of https://github.com/zRzRzRzRzRzRzR/transformer…
zRzRzRzRzRzRzR Jul 25, 2024
073b811
fix
zRzRzRzRzRzRzR Jul 25, 2024
c0e6ae9
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 25, 2024
6ac085f
fix glm dummy
zRzRzRzRzRzRzR Jul 25, 2024
f140603
Merge branch 'glm-4' of https://github.com/zRzRzRzRzRzRzR/transformer…
zRzRzRzRzRzRzR Jul 25, 2024
65f471d
add doc
zRzRzRzRzRzRzR Jul 26, 2024
7ad819f
fix init
zRzRzRzRzRzRzR Jul 26, 2024
f86af8e
Update __init__.py
zRzRzRzRzRzRzR Jul 26, 2024
c179377
Update dummy_vision_objects.py
zRzRzRzRzRzRzR Jul 26, 2024
41338d7
add_start_docstrings
zRzRzRzRzRzRzR Jul 26, 2024
dba6d1e
fix GLM_START_DOCSTRING
zRzRzRzRzRzRzR Jul 26, 2024
82b0c7f
1
zRzRzRzRzRzRzR Jul 26, 2024
a6b6f4e
Update perf_infer_gpu_one.md
zRzRzRzRzRzRzR Jul 26, 2024
d1a5ee1
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 26, 2024
c99610e
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 27, 2024
b283adc
flash attn
zRzRzRzRzRzRzR Jul 27, 2024
4cc618e
stiil need fix rotary_emb
zRzRzRzRzRzRzR Jul 27, 2024
b476dd0
fix GLMSelfAttension
zRzRzRzRzRzRzR Jul 27, 2024
aab2386
remove _get_unpad_data
zRzRzRzRzRzRzR Jul 27, 2024
550a692
fix GLMSelfAttention
zRzRzRzRzRzRzR Jul 27, 2024
6492ac3
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Jul 30, 2024
c3d4636
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Aug 9, 2024
70b7ff4
Merge branch 'huggingface:main' into glm-4
zRzRzRzRzRzRzR Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
1
  • Loading branch information
zRzRzRzRzRzRzR committed Jul 26, 2024
commit 82b0c7fc7d6f22e3224efe398970282069dc3307
2 changes: 1 addition & 1 deletion src/transformers/models/glm/modeling_glm.py
Original file line number Diff line number Diff line change
Expand Up @@ -998,7 +998,7 @@ def forward(
add_lm_head ([`bool`], *optional*, defaults to `False`):
Whether or not to add a language modeling head on top of the model. The language modeling head is composed
of two dense layers.
"""
""",
)
class GLMModel(GLMPreTrainedModel):
"""
Expand Down