Skip to content

llama : add Refact support #3061

Closed
@ggerganov

Description

@ggerganov

This is a new 1.6B code model: https://huggingface.co/smallcloudai/Refact-1_6B-fim

We should look into adding support into llama.cpp similar as to how we did for Falcon and Baichuan:

I haven't looked into the architecture yet, but I'm hoping it is similar to GPT, which we already know how to handle OK thanks to the Falcon experience. I see it also uses Alibi which we should have support for, but we haven't tested extensively, so there might be issues there.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions