🤗 smolagents: a barebones library for agents. Agents write python code to call tools and orchestrate other agents.

🙌 OpenHands: Code Less, Make More

Letta (formerly MemGPT) is a framework for creating LLM services with memory.

A high-throughput and memory-efficient inference and serving engine for LLMs

Development repository for the Triton language and compiler

Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.

A Native-PyTorch Library for LLM Fine-tuning

A JupyterLab extension for displaying dashboards of GPU usage.

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

NVIDIA GPU metrics exporter for Prometheus leveraging DCGM

NVIDIA GPU Operator creates/configures/manages GPUs atop Kubernetes

DSPy: The framework for programming—not prompting—foundation models

Open source platform for the machine learning lifecycle

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frame…

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A modular graph-based Retrieval-Augmented Generation (RAG) system

Example models using DeepSpeed

A toolkit to run Ray applications on Kubernetes

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Code examples and resources for DBRX, a large language model developed by Databricks

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch

High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

A Data Streaming Library for Efficient Neural Network Training

Ongoing research training transformer models at scale