|
| 1 | +# TAGLINE |
| 2 | + |
| 3 | +Peer-to-peer MoE model swarm for distributed LLM inference |
| 4 | + |
| 5 | +# TLDR |
| 6 | + |
| 7 | +**Serve** a local model directory on the swarm (tracker + bytes + optional expert compute) |
| 8 | + |
| 9 | +```lumabri serve --model [/path/to/model]``` |
| 10 | + |
| 11 | +**Chat** against a tracker, using colibri engine binaries from a local tree |
| 12 | + |
| 13 | +```lumabri chat --tracker [host:7300] --engines-dir [/path/to/colibri/c]``` |
| 14 | + |
| 15 | +Open the interactive front end with **saved swarm settings** (no flags) |
| 16 | + |
| 17 | +```lumabri``` |
| 18 | + |
| 19 | +Generate an **operator keypair** for a signed swarm |
| 20 | + |
| 21 | +```lumabri key --out [swarm]``` |
| 22 | + |
| 23 | +Serve and **join** an existing tracker while donating disk space |
| 24 | + |
| 25 | +```lumabri serve --model [./slice] --join [host:7300] --model-name [model] --donate [GB]``` |
| 26 | + |
| 27 | +Chat against a **local** model directory (no mirror, no second copy) |
| 28 | + |
| 29 | +```lumabri chat --local [/path/to/model] --engines-dir [/path/to/colibri/c]``` |
| 30 | + |
| 31 | +# SYNOPSIS |
| 32 | + |
| 33 | +**lumabri** [_subcommand_] [_options_] |
| 34 | + |
| 35 | +# DESCRIPTION |
| 36 | + |
| 37 | +**lumabri** runs huge mixture-of-experts (MoE) language models across a swarm of peers using the [colibri](https://github.com/JustVugg/colibri) engine. Pure C, no runtime dependencies beyond a colibri build for the engine binaries. One machine can share a model; others chat without downloading the full weights up front. Bytes that inference actually touches are fetched from peers, verified, and kept in a local mirror under **~/.lumabri** so later prompts can run offline from cache. |
| 38 | + |
| 39 | +**serve** starts a tracker (index of who holds which files) and a maintainer that answers byte-range reads. With phase 2 enabled it can also run expert execution for peers. **chat** boots a colibri engine against a model path that may exist only on the swarm: an **LD_PRELOAD** shim fetches missing blocks, and optional expert offload sends activations to peers that hold those experts. With no arguments, **lumabri** prompts for swarm address and key once and remembers them in **~/.lumabri/config**. |
| 40 | + |
| 41 | +Integrity modes include open swarms (per-MiB hashes, optional operator signatures, spot-check of expert results) and private swarms gated by **LUMABRI_TOKEN**. Design goals emphasize byte-identical output whether experts run locally or remotely, and a hardware floor that includes CPU-only machines. |
| 42 | + |
| 43 | +# SUBCOMMANDS |
| 44 | + |
| 45 | +**serve** |
| 46 | + |
| 47 | +> Share a model: run tracker/maintainer (and usually expert execution). Flags include **--model**, **--port**, **--join**, **--donate**, **--model-name**, **--key**, **--advertise**, **--cache** / **--exec-cache**, **--no-exec**. |
| 48 | +
|
| 49 | +**chat** |
| 50 | + |
| 51 | +> Connect to a swarm or a local model. Flags include **--tracker**, **--engines-dir**, **--local**, **--role**, **--donate**, **--model-dir**. In-chat commands include **/swarm** and **/model**. |
| 52 | +
|
| 53 | +**key** |
| 54 | + |
| 55 | +> Create an Ed25519 operator keypair (**--out** path) for signed swarms. |
| 56 | +
|
| 57 | +# ENVIRONMENT |
| 58 | + |
| 59 | +**LUMABRI_TOKEN** |
| 60 | + |
| 61 | +> Shared invite secret for private swarms; required on every node when set. |
| 62 | +
|
| 63 | +**LUMABRI_PUBKEY** |
| 64 | + |
| 65 | +> Operator public key (hex) for verifying signed model hashes out of band. |
| 66 | +
|
| 67 | +**LUMABRI_REQUIRE_HASH** |
| 68 | + |
| 69 | +> When set to 1, refuse fetches without integrity data. |
| 70 | +
|
| 71 | +**LUMABRI_VERIFY** |
| 72 | + |
| 73 | +> Spot-check fraction for phase-2 expert results on a second replica. |
| 74 | +
|
| 75 | +**LUMABRI_PREFETCH** |
| 76 | + |
| 77 | +> Readahead block count for cold mirrors (default 2). |
| 78 | +
|
| 79 | +# CAVEATS |
| 80 | + |
| 81 | +Requires **Linux**, **gcc**, **GNU make**, and a matching **colibri** checkout for engine binaries (**make phase2-all ENGINE=...** for peer expert execution). First cold answers are network-bound while the working set is mirrored. Misconfigured **--engines-dir** or missing keys can cause huge downloads or unverified models; the interactive default path is intentionally safer than raw flags. Tracker/maintainer/executor ports (commonly **7300–7302**, plus offsets per extra model) must be reachable for multi-host swarms. Status is a working prototype: features such as speculative drafting and key rotation remain incomplete. |
| 82 | + |
| 83 | +# SEE ALSO |
| 84 | + |
| 85 | +[ollama](/man/ollama)(1), [llama.cpp](/man/llama-cli)(1) |
| 86 | + |
| 87 | +# RESOURCES |
| 88 | + |
| 89 | +```[Source code](https://github.com/JustVugg/lumabri)``` |
| 90 | + |
| 91 | +<!-- verified: 2026-08-10 --> |
0 commit comments