A high-throughput and memory-efficient inference and serving engine for LLMs

SGLang is a fast serving framework for large language models and vision language models.

A Datacenter Scale Distributed Inference Serving Framework

An easy to use PyTorch to TensorRT converter

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

A full-featured, hackable Next.js AI chatbot built by Vercel

The official Node.js / Typescript library for the OpenAI API

LLM Inference benchmark

Development repository for the Triton language and compiler

A low-latency & high-throughput serving engine for LLMs

An ultra lightweight inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

A template to create any LLM Inference Web Apps using Python only

OneFlow backend for 🤗 Diffusers and ComfyUI

提取微信聊天记录，将其导出成HTML、Word、CSV文档永久保存，对聊天记录进行分析生成年度聊天报告

The first open source triton inference engine for Stable Diffusion, specifically for sdxl

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

framework for plug and play of various transformers (vision and nlp) with FSDP