TinyML & Edge AI: On-device inference, model quantization, embedded ML, ultra-low-power AI for microcontrollers and IoT devices.
Auditable offline edge intelligence for low-cost edge devices, with benchmark evidence and public board proof on ESP32-C3.
Flutter starter example app to get started with NobodyWho, a library designed to run LLMs locally and efficiently on any device.
Custom llama.cpp fork with character intelligence engine: control vectors, attention bias, head rescaling, attention temperature, fast weight memory
iOS + Android app that runs local LLMs on-device + routstr cloud LLMs for anonymous inference
Production Android AI with ExecuTorch 1.0 - Deploy PyTorch models to mobile with NPU acceleration and 50KB footprint
Mobile AI: iOS CoreML, Android TFLite, on-device inference, ONNX, TensorRT, and ML deployment for smartphones.
High-performance Android SDK for on-device LLM inference (GGUF). Privacy-focused, offline-first, and powered by llama.cpp with a clean Kotlin Coroutines API.
The Private Agent OS — search files, run AI agents, connect to 10,000+ tools via the complete protocol stack (MCP, AG-UI, A2UI, A2A). Zero cloud. Zero telemetry. On-device inference.
Neural acoustic echo cancellation for Apple platforms using CoreML — Swift package with 128/256/512-unit DTLN-aec models
React Native SDK for local LLM inference and on-device AI on iOS and Android.
Ad generation via offline LLMs with on-device inference, optionally managed by a self-hosted CMS.
Real-time SAM2 segmentation on edge devices - 40x faster C++ inference with ONNX Runtime for iOS/Android deployment
Run small LLMs directly in your web browser, no cloud computing needed.
Swift wrapper for Apple's BNNS graph API — run compiled CoreML models (.mlmodelc) on CPU with zero-copy buffer management
Open source Node.js runtime for local LLM inference, on-device AI, and private model execution.
Privacy-first age check that keeps face processing on the device but still gives the server something it can verify.
Deep technical writing on edge AI, on-device inference, llama.cpp, GGML, and mobile AI engineering
Add a description, image, and links to the on-device-inference topic page so that developers can more easily learn about it.
To associate your repository with the on-device-inference topic, visit your repo's landing page and select "manage topics."