Closed
Description
This is a new 1.6B code model: https://huggingface.co/smallcloudai/Refact-1_6B-fim
We should look into adding support into llama.cpp
similar as to how we did for Falcon and Baichuan:
- llm : add Falcon support #2717
- Feature: support baichuan serial models, by now, including Baichuan-7… #3009
I haven't looked into the architecture yet, but I'm hoping it is similar to GPT, which we already know how to handle OK thanks to the Falcon experience. I see it also uses Alibi which we should have support for, but we haven't tested extensively, so there might be issues there.