LocalLLM

Private AI for Web Applications

Run powerful language models locally in your web apps with zero latency, complete privacy, and native macOS performance.

_{Requires macOS 15+ with Apple Intelligence}

What is LocalLLM?

LocalLLM brings Apple's on-device AI models directly to web applications through a simple JavaScript API. It bridges the gap between powerful local language models and modern web development, enabling developers to build AI-powered experiences without compromising user privacy or dealing with API costs and latency.

Why LocalLLM?

For Users:

• 🔒 Complete Privacy - Your data never leaves your device
• 🚀 Zero Latency - Instant responses with no network delays
• 💰 No API Costs - Use powerful AI models without subscription fees
• ⚡ Native Performance - Optimized for Apple Silicon

For Developers:

• 🌐 Simple Integration - OpenAI-compatible JavaScript API
• 📡 Real-time Streaming - Token-by-token response streaming
• 🔧 Framework Agnostic - Works with React, Vue, vanilla JS, etc.
• 🛠️ TypeScript Support - Full type definitions included

Quick Start

1. Install LocalLLM

Download and run the macOS installer. It will:

• Install the native binary and Chrome extension
• Set up Safari extension (if needed)
• Verify system requirements

2. Use in Your Web App

// Check availability
if (await window.localLLM.available()) {
  console.log('LocalLLM is ready!');
}

// Generate text (OpenAI-compatible)
const response = await window.localLLM.chat.completions.create({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain quantum computing' }
  ]
});
console.log(response.choices[0].message.content);

// Stream responses
const stream = await window.localLLM.chat.completions.create({
  messages: [
    { role: 'user', content: 'Write a story' }
  ],
  stream: true
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process(content);
}

3. Test Your Integration

Open any website with LocalLLM installed and use the browser's developer console to test the API immediately.

Development

This project uses a shared codebase architecture for Chrome and Safari extensions with native macOS integration.

Setup

# Install dependencies
pnpm install

# Start development with file watching
pnpm dev

Architecture

• shared/ - Common extension logic (edit these files)
• macOS-container-app/ - Native macOS app with Safari extension
• native-foundation-models-extension/ - Chrome extension (generated files)
• scripts/ - Build automation and sync tools

The sync script automatically generates platform-specific files from shared sources. See CLAUDE.md for detailed development guidelines.

Links

• 🌐 Website & Demos
• 🔗 Chrome Extension
• 📚 Full Documentation

Contributing

Contributions are welcome! Please:

1. Read CLAUDE.md for project architecture and development guidelines
2. Check existing issues or create new ones
3. Fork and create a pull request with your changes
4. Test on both Chrome and Safari before submitting

Development Guidelines

• Edit shared code in shared/ directory only
• Use pnpm dev for development with file watching
• Follow the existing code style and patterns
• Ensure changes work on both Chrome and Safari

License

MIT License - see LICENSE file for details.

---

Created by @zats • Built with 🖤 for the developer community