This repository was archived by the owner on Jul 13, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Jul 13, 2026. It is now read-only.
[feature]: Support CPU accelerate by using GGUF #87
Description
Activity
- changed the title
[-][feature]: Support CPU accelerate[/-][+][feature]: Support CPU accelerate by using GGUF[/+]on May 2, 2024 Hugging Face transformers already support gguf. However, only several model architectures. So, we will do some test first. If it is ok we will suppert CPU accelerate smoothly. More detail see our discussion
Currently, we support CPU inference accelerate using llama.cpp. However, we will keep working on kimchima repo. We need to implement the CPT and fine-tune in kimchima
https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf?source=post_page-----7d1fa0b0b623--------------------------------