The LMSYS team created the initial version of FastChat. The Shale Protocol team continuously re-engineers it in order to build the infrastructure that supports a production-ready inference API for open-source LLMs.
See more at https://shaleprotocol.com
Thanks to LMSYS team❤️
We are focused to support Llama2 at scale now. If you want any other models, please contact.
- OpenHermes-2.5-Mistral-7B
- Gemma-7b-it
Sync upstream changes
Sync upstream changes
Support llama2 at scale.
Support "Llama-2-13b-chat-hf" and make it the default for API.
- Fixed issues working with AutoGPT and gpt-engineer etc.
- Added support for longchat-7b-16k.
- Added support for CodeT5p and Falcon-7b models.
- API key database and rate limit enforcement
- Deployable on Kubernetes