[CANN] Any possibility for the Unsloth dynamic quant of R1 to work on Ascend cards?

[Dango233]

Ascends NPUs seems to be a great alternative (to Macstudio and epyc) to run quantized R1.
For example: Atlas 300I Duo offers 140TFLOPS fp16 408GB/s mem bandwidth + 96G Vram.
2 of this card onto a PC could run the quantized 671B R1 relatively well I would say.

However, as shown in https://github.com/ggerganov/llama.cpp/blob/master/docs/backend/CANN.md, there is no deepseek architecture support yet, and low bit quantization seems to be not validated yet.

@hipudding Do you have plan on porting low-bit quantized R1 to Ascend cards, via gguf-cann backend?

That seems a pretty valid use case to me...

[cecuca]

@hipudding is there something we can help you with to make it happen?