[REQ] Add support for phi-3 mini model

[surenderssm]

What kind of request is this?

New feature

What is your request or suggestion?

I would like to add a support for phi-3 mini model.

https://huggingface.co/microsoft/Phi-3-mini-128k-instruct

I am very new to this ecosystem, however I have a strong desire to learn and contribute in this space, pointers or references will be enough for me to get started. Let me know your thoughts!

Are you willing to submit PRs to contribute to this feature request?

• Yes, I am willing to implement it.

[sozercan]

@surenderssm thanks for opening an issue! I just added the phi 3 3.7b model. see https://sozercan.github.io/aikit/premade-models

this is not the 128k version but the 4k one. if you are interested in contributing, it'll be similar to #210 but for 128k gguf file. I didn't see the gguf for the 128k one at the time

[sozercan]

Looks like llama.cpp doesn't support 128k yet due to longrope scaling type: ggerganov/llama.cpp#6849

[sozercan]

Looks like ggerganov/llama.cpp#7225 added support for 128k version. aikit v0.9.0 includes the llama.cpp version that has this feature.