Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

download and convert the 7B1 model to ggml FP16 format fails! #35

Open
distalx opened this issue Dec 18, 2023 · 0 comments
Open

download and convert the 7B1 model to ggml FP16 format fails! #35

distalx opened this issue Dec 18, 2023 · 0 comments

Comments

@distalx
Copy link

distalx commented Dec 18, 2023

As describe into readme file, when I try to run the convert-hf-to-ggml.py script I'm getting the following error.

Loading model:  bigscience/bloomz-7b1
pytorch_model.bin:  68%|████████████████████████████████████████████████████████████████████████████████████▎                                       | 9.62G/14.1G [16:33<07:47, 9.68MB/s]
Traceback (most recent call last):
  File "/home/e/Downloads/bloomz.cpp/convert-hf-to-ggml.py", line 84, in <module>
    model = AutoModelForCausalLM.from_pretrained(model_name, config=config, torch_dtype=torch.float16 if ftype == 1 else torch.float32, low_cpu_mem_usage=True)
  File "/home/e/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/home/e/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3268, in from_pretrained
    resolved_archive_file = cached_file(
  File "/home/e/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1461, in hf_hub_download
    http_get(
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 569, in http_get
    raise EnvironmentError(
OSError: Consistency check failed: file should be of size 14138162687 but has size 9616967235 (pytorch_model.bin).
We are sorry for the inconvenience. Please retry download and pass `force_download=True, resume_download=False` as argument.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.
Loading model:  bigscience/bloomz-7b1
pytorch_model.bin:  75%|████████████████████████████████████████████████████████████████████████████████████████████▋                               | 10.6G/14.1G [17:45<05:59, 9.92MB/s]
Traceback (most recent call last):
  File "/home/e/Downloads/bloomz.cpp/convert-hf-to-ggml.py", line 84, in <module>
    model = AutoModelForCausalLM.from_pretrained(model_name, config=config, torch_dtype=torch.float16 if ftype == 1 else torch.float32, low_cpu_mem_usage=False)
  File "/home/e/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/home/e/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3268, in from_pretrained
    resolved_archive_file = cached_file(
  File "/home/e/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1461, in hf_hub_download
    http_get(
  File "/home/e/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 569, in http_get
    raise EnvironmentError(
OSError: Consistency check failed: file should be of size 14138162687 but has size 10568333935 (pytorch_model.bin).
We are sorry for the inconvenience. Please retry download and pass `force_download=True, resume_download=False` as argument.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

I'm running : python3 convert-hf-to-ggml.py bigscience/bloomz-7b1 ./models

I believe this fails for some reason.
AutoModelForCausalLM.from_pretrained(model_name, config=config, torch_dtype=torch.float16 if ftype == 1 else torch.float32, low_cpu_mem_usage=True)

I do have enough disk space so I'm not sure why it downloading fails around 10 Gb. Also, my .cache directory has both unfinished files.

My operating system is Ubuntu 22.04.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant