Coppermind is a client-side hybrid search engine that runs entirely on your device. It combines the semantic understanding of Vector Search with the precision of Keyword Search (BM25), fused using Reciprocal Rank Fusion.
Unlike traditional search engines that rely on heavy server-side infrastructure, Coppermind compiles to a single, high-performance binary that runs in your browser (WASM), on your desktop, or on your phone. It brings transformer-based embeddings (JinaBERT) to the edge, ensuring your data never leaves your machine.
- Rust-First: UI (Dioxus), ML inference (Candle), search algorithms, and storage - all written in Rust and compiled to WASM or native code
- Hybrid Search: Combines semantic similarity (vector search) with keyword matching (BM25) using Reciprocal Rank Fusion
- Browser ML: Runs JinaBERT embeddings client-side with Candle
- Cross-Platform: Single Rust codebase targets web (WASM), desktop (macOS/Linux/Windows), and iOS
- Platform-Specific Features:
- Web: Background embedding via Web Workers, IndexedDB storage
- Desktop: Web crawler with parallel requests, native GPU acceleration (Metal/CUDA), redb storage
- CLI Tool: Search your indexed documents from the command line with
cm "query" - Fully Local: All processing happens on your device - no cloud APIs, works offline
# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Dioxus CLI
cargo install dioxus-cli --locked# Clone repository
git clone https://github.com/emersonmde/coppermind.git
cd coppermind
# Download ML models (~262MB model + 695KB tokenizer)
./download-models.sh# Web platform (development server)
dx serve -p coppermind
# Opens http://localhost:8080
# Desktop platform (native app with web crawler)
dx serve -p coppermind --platform desktop
# iOS platform (experimental)
dx serve -p coppermind --platform ios
# Production build (web)
dx bundle -p coppermind --release
# Production build (desktop)
dx bundle -p coppermind --release --platform desktop
# Install CLI tool
cargo install --path crates/coppermind-cliThe cm CLI lets you search your indexed documents from the terminal:
# Search your index
cm "rust embeddings"
cm "machine learning" -n 5
cm "query" --json
# Show help
cm --helpThe CLI shares the same index as the desktop app, so documents indexed in the GUI are searchable from the command line.
Quarantine:
- Open the DMG and drag Coppermind to your Applications folder
- Open Terminal and run:
sudo xattr -rd com.apple.quarantine /Applications/Coppermind.app
- Launch Coppermind from Applications
| Feature | Web | Desktop | iOS |
|---|---|---|---|
| Hybrid Search (Vector + BM25) | ✅ | ✅ | ✅ |
| Local Embedding (JinaBERT) | ✅ | ✅ | ✅ |
| Web Worker (background ML) | ✅ | N/A | N/A |
| Web Crawler | ❌ (CORS) | ✅ | ❌ |
| GPU Acceleration | ❌ | ✅ (Metal/CUDA) | |
| Storage | IndexedDB | redb | redb |
| Text Chunking | Markdown + Sentence | Markdown + Code + Sentence | Markdown + Code + Sentence |
- Dioxus 0.7 - Reactive UI framework (React-like component model, cross-platform rendering)
- Candle 0.8 - ML inference framework (Hugging Face Rust ML library)
- JinaBERT-v2-small-en - Embedding model (512-dimensional, ALiBi positional embeddings)
- tokenizers-rs 0.20 - Tokenization (Hugging Face Transformers tokenizer in Rust)
- instant-distance 0.6 - Vector search (HNSW approximate nearest neighbor, rayon parallel indexing)
- bm25 2.3 - Keyword search (Okapi BM25 ranking with TF-IDF)
- Reciprocal Rank Fusion - Result fusion (rank-based merging of vector and keyword results)
- scraper 0.22 - HTML parsing (CSS selector-based extraction)
- reqwest 0.12 - HTTP client (async fetching with TLS)
- BFS traversal - Crawl strategy (breadth-first with cycle detection, same-origin filtering)
- text-splitter 0.18 - Smart chunking (ICU4X sentence segmentation)
- pulldown-cmark 0.12 - Markdown parsing (structure-aware chunking)
- tree-sitter - Code parsing (syntax-aware chunking, desktop/iOS only)
- IndexedDB - Web storage (browser-native key-value store, zero bundle cost)
- redb 2.4 - Desktop/iOS storage (pure Rust B-tree database, ACID transactions)
Coppermind combines two complementary search approaches. Vector search uses instant-distance's HNSW (Hierarchical Navigable Small World) implementation to find documents semantically similar to your query - catching paraphrases, synonyms, and conceptual matches that keyword search would miss. Keyword search uses the BM25 algorithm (the same ranking function used by Elasticsearch and Lucene) to find exact keyword matches, ensuring precise terms aren't buried by semantic noise. These two result sets are merged using Reciprocal Rank Fusion (RRF), a rank-based fusion algorithm that operates purely on document positions. This means you can combine vector similarity scores (0-1 range) with BM25 scores (unbounded) without normalizing their scales - RRF simply ranks documents by their positions in each list.
Traditional browser-based ML uses JavaScript frameworks like TensorFlow.js or ONNX Runtime Web, but Coppermind takes a different approach: Candle, a minimalist Rust ML framework from Hugging Face. The JinaBERT embedding model (safetensors format) is loaded directly in the browser, tokenized with tokenizers-rs (the same Rust tokenizer library used by Hugging Face Transformers), and executed in WebAssembly. This delivers near-native performance without sending data to external APIs, keeping documents private and enabling offline operation. The model outputs 512-dimensional embeddings with a maximum sequence length of 2048 tokens (configurable up to 8192), balancing context window and memory usage.
The desktop version includes a built-in web crawler for indexing documentation sites and web content. CORS (Cross-Origin Resource Sharing) restrictions prevent web browsers from fetching arbitrary URLs, so this feature is only available in native desktop builds. The crawler implements BFS (breadth-first search) traversal with cycle detection (deduplicates URLs by normalizing trailing slashes), same-origin filtering (restricts crawling to the starting domain and path), and configurable parallel requests (1-16 concurrent fetches). HTML parsing uses scraper (built on html5ever) with CSS selectors to extract visible text while filtering scripts and styles. Crawled pages are automatically chunked and indexed for semantic search.
Documents are split into semantically meaningful chunks before embedding to improve search relevance and reduce token counts. The chunking strategy is selected automatically based on file type:
- Markdown (pulldown-cmark): Structure-aware splitting that preserves headings, lists, and code blocks
- Code (tree-sitter, desktop/iOS only): Syntax-aware splitting that respects function boundaries, classes, and module structure (supports Rust, Python, JavaScript, Java, C/C++, Go)
- Text (text-splitter): ICU4X sentence segmentation for natural language
On web (WASM), code files fall back to text chunking since tree-sitter requires native code compilation. Desktop and iOS get full syntax-aware chunking.
Dioxus is a React-like UI framework for Rust that provides a familiar component model with hooks, props, and reactive state management - but compiles to native code instead of running in a JavaScript runtime. Unlike JavaScript frameworks that require bundlers, transpilers, and runtime overhead, Dioxus components are statically typed at compile time and generate optimized WASM or native binaries with zero-cost abstractions. The framework handles platform-specific rendering transparently: DOM manipulation on web, native windows on desktop, and mobile views on iOS/Android. This means you write UI logic once with full Rust type safety, and Dioxus handles the cross-platform rendering.
Platform-specific features are handled with conditional compilation (#[cfg(target_arch = "wasm32")], #[cfg(feature = "desktop")]). The Rust compiler generates optimized code for each target:
- Web (WASM): Compact binaries, CPU-only inference, Web Worker for background embedding
- Desktop: Native executables, full GPU acceleration (Metal on macOS, CUDA on Linux/Windows), web crawler support
- iOS: Native app, CPU-only inference (Accelerate framework), no web crawler (could be added but requires handling app sandboxing)
Storage, asset loading, and threading are also platform-specific: IndexedDB/HTTP fetch/Web Workers on web vs. redb/direct file access/native threads on desktop/iOS.
Documents and search indexes are persisted locally using platform-appropriate backends via the DocumentStore trait. Web uses IndexedDB, the browser-native key-value store with zero bundle cost and excellent performance for structured data. Desktop/iOS uses redb, a pure Rust B-tree database providing ACID transactions and fast O(log n) lookups without external dependencies.
The storage layer tracks document sources with content hashes (SHA-256), enabling intelligent re-upload handling - unchanged files are skipped, modified files are updated in-place, and removed files are cleaned up. Vector index deletions use tombstone marking with automatic compaction when the ratio exceeds 30%, maintaining index efficiency across document updates.
Coppermind includes a scientific evaluation framework for measuring search quality:
# Run search quality evaluation
cargo run -p coppermind-eval --release
# Run with RRF ablation study
cargo run -p coppermind-eval --release -- --ablation rrfThe evaluation uses standard IR metrics (NDCG, MAP, MRR, Precision, Recall) with statistical significance testing. See crates/coppermind-eval/README.md for details.
Performance benchmarks for indexing throughput and search latency:
# Run all benchmarks
cargo bench -p coppermind-corecrates/
├── coppermind-core/ # Platform-independent search engine library
├── coppermind/ # Cross-platform GUI application (Dioxus)
├── coppermind-cli/ # Command-line search tool
└── coppermind-eval/ # Search quality evaluation framework
MIT License - see LICENSE for details.