feat(model): Support llama3.1 models

[fangyinc]

Description

Support Llama3.1 models

How Has This Been Tested?

We assume that your models are stored in the models directory, e.g., models/Meta-Llama-3.1-8B-Instruct.

Then modify your .env file:

LLM_MODEL=meta-llama-3.1-8b-instruct
# LLM_MODEL=meta-llama-3.1-70b-instruct
# LLM_MODEL=meta-llama-3.1-405b-instruct
## you can also specify the model path
# LLM_MODEL_PATH=models/Meta-Llama-3.1-8B-Instruct
## Quantization settings
# QUANTIZE_8bit=False
# QUANTIZE_4bit=True
## You can configure the maximum memory used by each GPU.
# MAX_GPU_MEMORY=16Gib

Then you can run the following command to start the server:

dbgpt start webserver

Snapshots:

Include snapshots for easier review.

Checklist:

• My code follows the style guidelines of this project
• I have already rebased the commits and make the commit message conform to the project standard.
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Any dependent changes have been merged and published in downstream modules

[Aries-ckt]

r+

[csunny]

r+