Skip to content

Bug: Latest version of convert_hf_to_gguf not compatible with gguf 0.9.1 from pip #8925

Closed
@Ru13en

Description

@Ru13en

What happened?

Hey,
I've been trying to run the convert_hf_to_gguf.py in python 3.12 in my development enviroment.
While trying to run a model conversion I got a error from GGUFWriter.
I investigated the pip package from the official pip repo:
https://files.pythonhosted.org/packages/9f/1e/6bd024e0138d663cd333e8fde4c03d343e8be680d43b86537ef1497a7e32/gguf-0.9.1.tar.gz
The wheel has the same issue.
And it seems that the published version does not contain the latest changes from master.
Can you guys bump the version of gguf and publish it to pip?
Thanks

Name and Version

version: 3541 (be55695)
built with gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9) for x86_64-redhat-linux

What operating system are you seeing the problem on?

Linux Red Hat 11.2.1-9

Relevant log output

Traceback (most recent call last):
  File "/app/./convert_hf_to_gguf.py", line 3823, in <module>
    main()
  File "/app/./convert_hf_to_gguf.py", line 3817, in main
    model_instance.write()
  File "/app/./convert_hf_to_gguf.py", line 401, in write
    self.prepare_metadata(vocab_only=False)
  File "/app/./convert_hf_to_gguf.py", line 352, in prepare_metadata
    total_params, shared_params, expert_params, expert_count = self.gguf_writer.get_total_parameter_count()
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'GGUFWriter' object has no attribute 'get_total_parameter_count'

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-unconfirmedhigh severityUsed to report high severity bugs in llama.cpp (Malfunctioning hinder important workflow)stale

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions