This repository contains the knowledge-base extraction pipeline only.
The ARC application described further down — a FastAPI service with an htmx UI, a reverse-BOM solver and a vision inventory client — is not published here. It is not in the working tree and it is not in any commit of this repository. Earlier versions of this README documented it as if it shipped; that was wrong.
What you actually get:
| Directory | What it is | State |
|---|---|---|
kb/pipeline/schemas/ |
CDPO ontology (Component–Device–Project) as Pydantic v2 models, including the firmware-genome model | works, importable |
kb/pipeline/extractors/ |
LLM extraction adapters + prompts, cost estimator, final-pack builder (tar + zstd) | runnable with API keys and input data |
kb/pipeline/scripts/ |
Scrapers (iFixit, Instructables, Hackaday, YouTube transcripts), validators, index builder, packager | runnable |
scripts/bootstrap-kb.sh |
Bulk corpus fetch (Kiwix ZIMs, KiCad libraries, archive.org, Appropedia, Wikidata, OpenRepair) | runnable |
docs/ |
Deployment and firmware-validation notes | reference only |
There are no tests in this repository and the two GitHub Actions workflows
have failed on every run since the repository went public, because they install
product/server/requirements.txt — a path that has never existed here.
kb/pipeline/scripts/validate_schema.py also imports product/server/kb/index.py
and therefore cannot run in this repository.
Every "AI for hardware" tool tells you what to buy. A general-purpose chat model will invent part numbers. Offline encyclopedias are searchable but do not reason over an inventory.
The goal of ARC is a machine-readable, offline body of knowledge that can answer "what can I build from the parts I already have" without a network connection. The pipeline in this repository is the part that builds that body of knowledge.
- Reverse-BOM — "I have X, want Y" → pick components from a scrap pile and return a build plan.
- Forward-BOM — "I want Y" → return the BOM and which household devices typically contain those parts.
- Discovery — "I have X" → return buildable projects for that inventory.
These describe the intended product. None of them is implemented in this repository.
The KB is generated by the pipeline in kb/pipeline/. It is not published.
There is no release asset, and kb/output/ is empty in a fresh clone. The
counts below come from the author's local build and cannot be verified from this
repository.
| Collection | Records (local build) |
|---|---|
| Components (canonical) | 55,414 |
| Substitutions (chains) | 10,000 |
| Devices (teardown patterns) | 5,000+ |
| Materials (DIY recipes) | 1,242 |
| Projects (recipes) | 1,225+ |
| Tools (with build-from-scrap paths) | 716 |
| Safety profiles | 500 |
| Phenomena (physics for solver) | 301 |
| Skills (with prerequisites) | 203 |
| Goals (top-level objectives) | 50 |
| Regional profiles (mains / radio bands / etc.) | 50 |
| Firmware genome (configurable templates) | 40 |
| Total | ~78,869 |
kb/pipeline/scripts/package.sh writes the bundle as ark-kb-v0.1.tar.zst.
Older READMEs advertised a download at
releases/latest/download/arc-kb.tar.zst (and ark-kb.tar.zst in the Russian
version). That link 404s — release v0.1.0 carries no assets — and has been
removed rather than renamed.
git clone https://github.com/hermandoronin/arc-computer.git
cd arc-computer
pip install pydantic zstandard httpx
# Inspect the data model
python -c "from kb.pipeline.schemas.cdpo import *; print('CDPO models loaded')"
# Fetch the raw corpora the extractors work on (large, needs disk space)
bash scripts/bootstrap-kb.sh
# Run an extractor over the fetched corpus (needs an LLM API key)
python kb/pipeline/extractors/run_device_extractor.py --helpThere is no server to start and no UI to open.
- Data model — Pydantic v2 (
kb/pipeline/schemas/cdpo.py) - Extraction — LLM adapters over scraped corpora, prompts in
kb/pipeline/extractors/prompts/ - Packaging — tar + zstd (
kb/pipeline/scripts/package.sh) - Validation — cross-reference, coverage, anti-laziness regex, firmware compile via
simavr/avr-gcc/arm-none-eabi-gcc
arc-computer/
├── README.md you are here
│
├── kb/
│ ├── STRATEGY.md two-zone storage plan
│ ├── pipeline/
│ │ ├── schemas/cdpo.py canonical Pydantic data model
│ │ ├── extractors/ LLM extraction adapters + prompts
│ │ └── scripts/ scrapers, validators, packagers
│ └── output/ generated KB (gitignored, not published)
│
├── docs/ deployment + firmware validation notes
└── scripts/ corpus bootstrap helper
| Document | What |
|---|---|
kb/STRATEGY.md |
KB storage tiering (hot zone on device, cold on dev disk) |
docs/DEPLOYMENT.md |
Deployment notes for the unpublished server |
docs/FIRMWARE-VALIDATION.md |
simavr / qemu / avr-gcc setup |
- CDPO data model — Pydantic schemas including the firmware genome
- Extraction pipeline — scrapers, LLM extractors, validators, packager
- Publish the KB — release asset, reproducible build
- Publish the solver — the FastAPI service and UI described above
- v0.2 — content packs (Marine, HAM, Homestead, 3D-printer salvage)
- v0.3 — Raspberry Pi 5 kit with pre-loaded KB, vision inventory
- v1.0 — multi-language KB, mesh between devices
People who already build things themselves: off-grid homesteaders, sailors and liveaboards, remote-region engineers and repair shops, field NGO staff and expedition crews, maker hobbyists.
See CONTRIBUTING.md.
- Code style —
ruff(Python 3.13+) - KB entries follow
kb/pipeline/schemas/cdpo.py - Conventional Commits
Issue templates: bug · feature · knowledge-base entry
Community standards: CODE_OF_CONDUCT.md (Contributor Covenant 2.1) · Security disclosures: SECURITY.md
| Scope | License | File |
|---|---|---|
| Code (pipeline, extractors, validators) | Apache-2.0 | LICENSE |
| Knowledge-base content | CC-BY-4.0 | LICENSE-KB |
| Adapters and scrapers | MIT | LICENSE-ADAPTERS |
Per-record provenance metadata in the KB JSON files indicates upstream sources (iFixit content under CC-BY-NC-SA, manufacturer datasheets, etc.) — respect the upstream licence when redistributing.
- Issues — open one on GitHub
Built for people who build, fix, and grow things — anywhere, anytime.