Skip to content

Commit

Permalink
Merge pull request #87 from kerthcet/feat/add-projects
Browse files Browse the repository at this point in the history
Add NanoFlow
  • Loading branch information
InftyAI-Agent authored Sep 22, 2024
2 parents d8c1034 + ec6077f commit 6f47447
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
| **[MInference](https://github.com/microsoft/minference)** | ![Stars](https://img.shields.io/github/stars/microsoft/minference.svg) | ![Release](https://img.shields.io/github/release/microsoft/minference) | ![Contributors](https://img.shields.io/github/contributors/microsoft/minference) | To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy. | |
| **[MLC LLM](https://github.com/mlc-ai/mlc-llm)** | ![Stars](https://img.shields.io/github/stars/mlc-ai/mlc-llm.svg) | ![Release](https://img.shields.io/github/release/mlc-ai/mlc-llm) | ![Contributors](https://img.shields.io/github/contributors/mlc-ai/mlc-llm) | Universal LLM Deployment Engine with ML Compilation | |
| **[MLServer](https://github.com/SeldonIO/MLServer)** | ![Stars](https://img.shields.io/github/stars/SeldonIO/MLServer.svg) | ![Release](https://img.shields.io/github/release/SeldonIO/MLServer) | ![Contributors](https://img.shields.io/github/contributors/SeldonIO/MLServer) | MLServer aims to provide an easy way to start serving your machine learning models through a REST and gRPC interface, fully compliant with KFServing's V2 Dataplane spec. | |
| **[Nanoflow](https://github.com/efeslab/Nanoflow)** | ![Stars](https://img.shields.io/github/stars/efeslab/nanoflow.svg) | ![Release](https://img.shields.io/github/release/efeslab/nanoflow) | ![Contributors](https://img.shields.io/github/contributors/efeslab/nanoflow) | A throughput-oriented high-performance serving framework for LLMs | |
| **[Ollama](https://github.com/ollama/ollama)** | ![Stars](https://img.shields.io/github/stars/ollama/ollama.svg) | ![Release](https://img.shields.io/github/release/ollama/ollama) | ![Contributors](https://img.shields.io/github/contributors/ollama/ollama) | Get up and running with Llama 3, Mistral, Gemma 2, and other large language models. | edge |
| **[OpenLLM](https://github.com/bentoml/OpenLLM)** | ![Stars](https://img.shields.io/github/stars/bentoml/openllm.svg) | ![Release](https://img.shields.io/github/release/bentoml/openllm) | ![Contributors](https://img.shields.io/github/contributors/bentoml/openllm) | Operating LLMs in production | |
| **[OpenVINO](https://github.com/openvinotoolkit/openvino)** | ![Stars](https://img.shields.io/github/stars/openvinotoolkit/openvino.svg) | ![Release](https://img.shields.io/github/release/openvinotoolkit/openvino) | ![Contributors](https://img.shields.io/github/contributors/openvinotoolkit/openvino) | OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference | |
Expand Down

0 comments on commit 6f47447

Please sign in to comment.