[Bug] The specified tag is not a valid quantization scheme.

[Mushoz]

Bug description.
When trying to pull a specific quantization tag for a model through Ollama I was getting the following error: The specified tag is not a valid quantization scheme.

At first I thought it was an Ollama bug, so I reported it to their issue tracker. However, it seems that it's really a Huggingface bug instead. I can replicate the issue with curl:

curl -vL -H Accept:application/vnd.docker/distribution.manifest.v2+json hf.co/v2/bartowski/Replete-LLM-V2.5-Qwen-32b-GGUF/manifests/IQ4_NL

Gives me the exact same error, while:

curl -vL -H Accept:application/vnd.docker/distribution.manifest.v2+json hf.co/v2/bartowski/Replete-LLM-V2.5-Qwen-32b-GGUF/manifests/Q4_K_M

Does not. It seems like Huggingface does not recognize IQ4_NL as a valid quantization scheme.

Upon further testing, IQ2_XXS suffers from the exact same problem, while IQ2_XS does work fine. This test was done with this repo: https://huggingface.co/bartowski/Meta-Llama-3.1-70B-Instruct-GGUF

Describe the expected behaviour
If the Quant is valid and exists within the specified repo, it should pull successfully.

[Vaibhavs10]

Hi @Mushoz - Thanks for the issue; we're looking into this with @ngxson 🤗

[Vaibhavs10]

Hi @Mushoz - can you retry now, this should be fixed.