Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issues during StableLM models conversion due to config.json changes #5703

Merged
merged 8 commits into from
Feb 25, 2024

Conversation

aahouzi
Copy link
Contributor

@aahouzi aahouzi commented Feb 24, 2024

Type of Change

[user1@dnp-test4 llama.cpp]$ python convert-hf-to-gguf.py $STABLELM
Loading model: a2eb1af48d006a3f6ec66a3a434979f4c817e2d0
Traceback (most recent call last):
  File "/home/user1/aahouzi/llama.cpp/convert-hf-to-gguf.py", line 1933, in <module>
    main()
  File "/home/user1/aahouzi/llama.cpp/convert-hf-to-gguf.py", line 1914, in main
    model_instance = model_class(dir_model, ftype_map[args.outtype], fname_out, args.bigendian)
  File "/home/user1/aahouzi/llama.cpp/convert-hf-to-gguf.py", line 50, in __init__
    self.model_arch = self._get_model_architecture()
  File "/home/user1/aahouzi/llama.cpp/convert-hf-to-gguf.py", line 285, in _get_model_architecture
    raise NotImplementedError(f'Architecture "{arch}" not supported!')
NotImplementedError: Architecture "StableLmForCausalLM" not supported!

Description

  • This PR fixes the issue, by adapting to the new changes in Stability naming. After the change:
[user1@dnp-test4 llama.cpp]$ python convert-hf-to-gguf.py $STABLELM
Loading model: a2eb1af48d006a3f6ec66a3a434979f4c817e2d0
gguf: This GGUF file is for Little Endian only
Set model parameters
Set model tokenizer
gguf: Adding 100000 merge(s).
gguf: Setting special token type bos to 100257
gguf: Setting special token type eos to 100257
gguf: Setting special token type unk to 100257
Exporting model to '/home/user1/.cache/huggingface/hub/models--stabilityai--stablelm-2-1_6b/snapshots/a2eb1af48d006a3f6ec66a3a434979f4c817e2d0/ggml-model-f16.gguf'
gguf: loading model part 'model.safetensors'
output.weight, n_dims = 2, torch.bfloat16 --> float16
token_embd.weight, n_dims = 2, torch.bfloat16 --> float16
blk.0.attn_norm.bias, n_dims = 1, torch.bfloat16 --> float32
blk.0.attn_norm.weight, n_dims = 1, torch.bfloat16 --> float32
blk.0.ffn_down.weight, n_dims = 2, torch.bfloat16 --> float16
blk.0.ffn_gate.weight, n_dims = 2, torch.bfloat16 --> float16
blk.0.ffn_up.weight, n_dims = 2, torch.bfloat16 --> float16
...
blk.9.attn_k.bias, n_dims = 1, torch.bfloat16 --> float32
blk.9.attn_k.weight, n_dims = 2, torch.bfloat16 --> float16
blk.9.attn_output.weight, n_dims = 2, torch.bfloat16 --> float16
blk.9.attn_q.bias, n_dims = 1, torch.bfloat16 --> float32
blk.9.attn_q.weight, n_dims = 2, torch.bfloat16 --> float16
blk.9.attn_v.bias, n_dims = 1, torch.bfloat16 --> float32
blk.9.attn_v.weight, n_dims = 2, torch.bfloat16 --> float16
output_norm.bias, n_dims = 1, torch.bfloat16 --> float32
output_norm.weight, n_dims = 1, torch.bfloat16 --> float32
Model successfully exported to '/home/user1/.cache/huggingface/hub/models--stabilityai--stablelm-2-1_6b/snapshots/a2eb1af48d006a3f6ec66a3a434979f4c817e2d0/ggml-model-f16.gguf'

Expected Behavior & Potential Risk

  • N/A

How has this PR been tested?

  • N/A

Dependency Change?

  • N/A

convert-hf-to-gguf.py Outdated Show resolved Hide resolved
convert-hf-to-gguf.py Outdated Show resolved Hide resolved
Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
convert-hf-to-gguf.py Outdated Show resolved Hide resolved
convert-hf-to-gguf.py Outdated Show resolved Hide resolved
aahouzi and others added 3 commits February 24, 2024 22:19
Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
@aahouzi
Copy link
Contributor Author

aahouzi commented Feb 24, 2024

@cebtenzzre one test in CI fails with:

ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

@cebtenzzre
Copy link
Collaborator

one test in CI fails with:

Yeah, this is something @crasm added in #4585 but it's not important - the CI just doesn't have enough disk space to run the test.

convert-hf-to-gguf.py Outdated Show resolved Hide resolved
convert-hf-to-gguf.py Outdated Show resolved Hide resolved
aahouzi and others added 2 commits February 25, 2024 10:33
Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>
Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>
@ggerganov ggerganov merged commit 69917df into ggerganov:master Feb 25, 2024
22 of 23 checks passed
@aahouzi aahouzi deleted the fix_stablelm_issue branch February 25, 2024 10:15
crasm added a commit that referenced this pull request Mar 1, 2024
Reduces peak tmpfs usage and should prevent the check from failing from
running out of space.

Fixes the 'No space left on device' issue mentioned in #5703.
crasm added a commit that referenced this pull request Mar 2, 2024
Reduces peak tmpfs usage and should prevent the check from failing from
running out of space.

Fixes the 'No space left on device' issue mentioned in #5703.
hazelnutcloud pushed a commit to hazelnutcloud/llama.cpp that referenced this pull request Mar 10, 2024
…#5826)

Reduces peak tmpfs usage and should prevent the check from failing from
running out of space.

Fixes the 'No space left on device' issue mentioned in ggerganov#5703.
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Mar 13, 2024
* Fix issues during StableLM models conversion

* Fix hard coded layer_norm_eps

* Support layer_norm_eps for LlavaStableLM

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* Add missing parenthesis

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* Support rotary_factor for LlavaStableLM

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* fix typo

* Add StableLMEpochForCausalLM for safety

Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>

* Add StableLMEpochForCausalLM for safety 2

Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>

---------

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
Co-authored-by: Jared Van Bortel <jared@nomic.ai>
Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Mar 13, 2024
…#5826)

Reduces peak tmpfs usage and should prevent the check from failing from
running out of space.

Fixes the 'No space left on device' issue mentioned in ggerganov#5703.
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
* Fix issues during StableLM models conversion

* Fix hard coded layer_norm_eps

* Support layer_norm_eps for LlavaStableLM

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* Add missing parenthesis

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* Support rotary_factor for LlavaStableLM

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

* fix typo

* Add StableLMEpochForCausalLM for safety

Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>

* Add StableLMEpochForCausalLM for safety 2

Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>

---------

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
Co-authored-by: Jared Van Bortel <jared@nomic.ai>
Co-authored-by: compilade <113953597+compilade@users.noreply.github.com>
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
…#5826)

Reduces peak tmpfs usage and should prevent the check from failing from
running out of space.

Fixes the 'No space left on device' issue mentioned in ggerganov#5703.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants