ggml

LLM inference in C/C++

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

llama and other large language models on iOS and MacOS offline using GGML library.

Go manage your Ollama models

ggml speech-to-text inference for 16+ model families

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose

KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM

Running any GGUF SLMs/LLMs locally, on-device in Android

Suno AI's Bark model in C/C++ for fast text-to-speech generation

Real-time 3D full-body reconstruction from a single camera, Multiperson BVH output, Pure C++ runtime, ONNX + ggml, 70-joint skeleton with hands.

Self-evaluating interview for AI coders

C++ ggml runtime hub for multilingual ASR and TTS models: Cohere Transcribe, Parakeet TDT, Voxtral, Canary 1B v2, etc, plus universal forced alignment, and more

Run inference on MPT-30B using CPU

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

CLIP inference in plain C/C++ with no extra dependencies

Large Language Models for All, 🦙 Cult and More, Stay in touch !