feat(model): Support llama.cpp server deploy

[fangyinc]

Description

1. Support llama.cpp server inference
2. API Server support /v1/completions
3. Support native generate function

How Has This Been Tested?

Install dependencies

pip install -e ".[llama_cpp_server]"

If you want to accelerate the inference speed, and you have a GPU, you can install the following dependencies:

CMAKE_ARGS="-DGGML_CUDA=ON" pip install -e ".[llama_cpp_server]"

Download the model

Here, we use the qwen2.5-0.5b-instruct model as an example. You can download the model from the Huggingface.

wget https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/qwen2.5-0.5b-instruct-q4_k_m.gguf?download=true -O /tmp/qwen2.5-0.5b-instruct-q4_k_m.gguf

Modify configuration file

In the .env configuration file, modify the inference type of the model to start llama.cpp inference.

LLM_MODEL=qwen2.5-0.5b-instruct
LLM_MODEL_PATH=/tmp/qwen2.5-0.5b-instruct-q4_k_m.gguf
MODEL_TYPE=llama_cpp_server

Start the DB-GPT server

python dbgpt/app/dbgpt_server.py

Snapshots:

Include snapshots for easier review.

Checklist:

• My code follows the style guidelines of this project
• I have already rebased the commits and make the commit message conform to the project standard.
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Any dependent changes have been merged and published in downstream modules

[Aries-ckt]

LGTM

[csunny]

LGTM