Sorthive: RK3588 camera transport and CM5 image bringup

.gitattributes

@@ -1,2 +1,5 @@
 software/sorter/backend/bundled_models/**/*.onnx filter=lfs diff=lfs merge=lfs -text
 software/sorter/backend/bundled_models/**/*.bin filter=lfs diff=lfs merge=lfs -text
+software/training/rknn_bundles/**/*.rknn filter=lfs diff=lfs merge=lfs -text
+software/sorteros/build/overlay/usr/lib/librknnrt.so filter=lfs diff=lfs merge=lfs -text
+software/sorteros/flash/rkbin/*.bin filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -31,4 +31,17 @@ software/sorteros/v3/build/.env
 /test.db
 /test-profile-parts.db
 
+# Local runtime state, logs, backups
+/backups/
+log/
+logs/
+local_state.sqlite*
+.playwright-mcp/
+/sorter-*.png
+/software/sorter/frontend/sorter-*.png
+/software/sorter/backend/aruco_config.json
+/software/sorter/backend/data/
+/software/sorteros/flash/*.img
+/software/training/rknn_bundles/
+
 .DS_Store

docs/_data/nav.yml

@@ -44,8 +44,6 @@ sections:
         pages:
           - title: Your first sort run
             url: /sorter/tutorials/first-sort-run/
-          - title: Camera calibration
-            url: /sorter/camera-calibration/
           - title: Troubleshooting
             url: /sorter/troubleshooting/
       - title: Under the hood

docs/getting-started.md

@@ -24,7 +24,7 @@ Sorter V2 is an open-source LEGO sorting machine. Feed bulk LEGO into a hopper,
 | Classification (Brickognize API) | Working | 98.5% accuracy on supported parts, 0.56s avg response |
 | Object detection (chamber zone) | Working | NanoDet + YOLO11s trained, benchmarked on Pi 5 + Orange Pi 5 |
 | Host software (Python backend) | WIP | Coordinator, state machines, vision manager functional |
-| SvelteKit UI | WIP | Setup wizard, camera calibration, runtime dashboard in progress |
+| SvelteKit UI | WIP | Setup wizard, camera controls, runtime dashboard in progress |
 | Hive community platform | WIP | Upload pipeline, shared profiles, crowd verification |
 | Electronics / PCB | WIP | Feeder + distribution board schematics in review (Rev 0.3) |
 | Hardware CAD | WIP | V2 CAD mostly complete in Onshape, validation in progress |

docs/lab/classification-research/index.md

@@ -60,29 +60,22 @@ Synthetic rendering is viable for augmentation. Varying angles, lighting, and po
 
 ## Color classification
 
-ML is **unreliable** for color detection due to lighting variation across machines and environments. The recommended approach avoids ML entirely:
-
-1. Place a color calibration card (reference colors) in frame.
-2. Apply classic white-balance correction.
-3. Use histogram matching against known reference patches.
-
-This is more reliable than any ML-based color detection the team tested.
+ML is **unreliable** for color detection due to lighting variation across machines and environments. The sorter currently treats color as a separate vision problem from part-shape classification, with camera exposure, white balance, and lighting kept under manual operator control.
 
 ## Trade-offs
 
 - **Remote API vs local model** — Brickognize API works and is fast (0.56s), but requires internet. A local model is planned for offline operation. Strategy: dual-mode — local for common parts, API for rare/uncertain.
 - **Embeddings vs classifier** — Embeddings need fewer examples per class but are fragile to bad vectors. Classifiers need more training data but are more robust. With enough data, classifiers win.
 - **Breadth vs depth** — Supporting all 70k parts requires impractical amounts of data. Focusing on top 5k gives 85%+ coverage with achievable effort.
-- **ML color vs calibration-based color** — ML color detection sounds elegant but fails across different lighting conditions. Calibration cards are boring but reliable.
+- **ML color vs controlled capture** — ML color detection sounds elegant but fails across different lighting conditions. The practical path is to keep the capture environment controlled before adding any dedicated color classifier.
 
 ## What this is not
 
 - **Not aiming for universal coverage.** Rare and obscure parts (post-2020 limited editions, regional exclusives) are out of scope for the initial classifier.
-- **Not doing color via ML.** Color detection uses classical computer vision (calibration cards, white balance), not learned models.
+- **Not doing color via ML.** Color handling is kept separate from the part classifier until the capture pipeline is stable enough for a dedicated color strategy.
 - **Not a self-contained local system yet.** The primary classifier (Brickognize) is a remote API. Local self-hosting is planned but not shipped.
 
 ## Where to go next
 
 - [Object detection research]({{ '/lab/object-detection/' | relative_url }}) — the detector that finds pieces in the chamber before classification
 - [Sorter architecture]({{ '/sorter/architecture/' | relative_url }}) — how classification fits into the sorting pipeline
-- [Camera calibration]({{ '/sorter/camera-calibration/' | relative_url }}) — the color calibration workflow that feeds into classification

docs/sorter/index.md

@@ -17,7 +17,6 @@ permalink: /sorter/
 ## Operate
 
 - **[Your first sort run]({{ '/sorter/tutorials/first-sort-run/' | relative_url }})** — the end-to-end happy path: pick a profile, feed the machine, check a bin, stop cleanly.
-- **[Camera calibration]({{ '/sorter/camera-calibration/' | relative_url }})** — focus with a Siemens Star, then auto-calibrate color with a SpyderCheckr 24.
 - **[Sorter troubleshooting]({{ '/sorter/troubleshooting/' | relative_url }})** — symptom-led entries for install, first-run, and runtime problems.
 
 ## Under the hood

docs/sorter/installation/sorter-os.md

@@ -8,8 +8,8 @@ kicker: Installation — SorterOS
 lede: Flash SorterOS onto an SD card and configure it for first boot. The recommended way to run Sorter on an Orange Pi 5.
 permalink: /sorter/installation/sorter-os/
 audience: self-hosting operator
-applies_to: sorteros v3.x
-last_verified: 2026-05-19
+applies_to: sorteros v4.x
+last_verified: 2026-06-09
 ---
 
 <div class="notice notice-warn">
@@ -32,40 +32,27 @@ SorterOS is a purpose-built OS image for Sorter, based on the official Ubuntu im
 
 Go to **[github.com/basicallysource/sorter-v2/releases](https://github.com/basicallysource/sorter-v2/releases)**, find the latest SorterOS release, and download the `.zip` file from its assets.
 
-## Step 2 — Configure WiFi and SSH
-
-If you do not need to configure WiFi, hostname, SSH auth key, or Tailscale auth key, you can flash the `.zip` file directly — skip ahead to Step 3.
-
-Otherwise, decompress the `.zip` file on your computer first, then open the `.img` file in **[SorterOS Setup](https://setup.basically.website)** to set those options before flashing. Everything runs client-side in the browser — nothing is sent to a server.
-
-**Saving the configured image:**
-- **Chrome** — SorterOS Setup can overwrite the original `.img` file directly, so no extra disk space is needed.
-- **Other browsers** — a new copy is downloaded. The image is about 8 GB, so make sure you have the space. Once you have the configured copy you can delete the original.
-
-The Pi needs an internet connection to complete first-boot initialization and to run Sorter on an ongoing basis. If you are not comfortable working in a terminal, configure WiFi here before flashing — the base image has no desktop environment, so WiFi cannot be set up through a GUI after booting.
-
-Alternatively, plug the Pi into your router via Ethernet and it will come online automatically without any WiFi configuration.
-
-If you ever need to change your WiFi network or SSH key, you can run the image through SorterOS Setup again — it will overwrite the previous configuration.
-
-## Step 3 — Flash to SD card
+## Step 2 — Flash to SD card
 
 1. Open **[Balena Etcher](https://etcher.balena.io/)**
-2. Click **Flash from file** and select either the `.zip` file (if you skipped setup) or the `.img` file (if you ran it through SorterOS Setup)
+2. Click **Flash from file** and select the downloaded `.zip` file
 3. Click **Select target** and choose your SD card
 4. Click **Flash**
 
-Wait for Etcher to finish writing and verifying. Do not remove the card until it reports success.
+Wait for Etcher to finish writing and verifying. Do not remove the card until it reports success. The image ships generic — there is nothing to configure before flashing.
+
+## Step 3 — Boot and connect to WiFi
 
-## Step 4 — Boot
+Insert the SD card into the Orange Pi 5 and power it on.
 
-Insert the SD card into the Orange Pi 5 and power it on. SorterOS completes first-boot setup automatically, then starts the Sorter backend and UI. This takes less than 5 minutes if everything is working.
+- **Ethernet** — if the Pi is wired to your router, it comes online automatically without any WiFi configuration.
+- **WiFi** — when the Pi boots without a network connection, it opens a temporary hotspot named **`SorterOS-Setup-…`**. Join it from your phone or laptop; a captive portal opens automatically where you pick your WiFi network and enter its password. The Pi then shuts down the hotspot and joins your network.
 
-Once first-boot initialization completes and the Pi has finished downloading its dependencies, the Sorter UI is available at:
+The Pi needs an internet connection to complete first-boot initialization and to run Sorter on an ongoing basis. Once first-boot setup completes and the Pi has finished downloading its dependencies — less than 5 minutes if everything is working — the Sorter UI is available at:
 
 **[http://sorter.local:5173/](http://sorter.local:5173/)**
 
-This uses mDNS — the device you're browsing from must be on the same network as the Pi. If you set a custom hostname in SorterOS Setup, substitute that name for `sorter`.
+This uses mDNS — the device you're browsing from must be on the same network as the Pi.
 
 ## SSH access
 

software/hive/.gitignore

@@ -30,6 +30,11 @@ backend/sorthive.db
 
 # Local runtime data
 backend/data/
+# Production bind-mount data (postgres datadir, parts.db, uploads). On the prod
+# host this lives inside the repo working tree, so it MUST stay ignored — a
+# `git stash -u` / `git clean -fdx` would otherwise wipe the live database.
+# See scripts/deploy.sh, which also never runs those commands.
+/data/
 
 # Node / Svelte
 node_modules/

software/hive/backend/alembic/versions/b1c2d3e4f5a6_add_machine_config_backups.py

@@ -0,0 +1,48 @@
+"""add machine_config_backups
+
+Revision ID: b1c2d3e4f5a6
+Revises: e9f0a1b2c3d4
+Create Date: 2026-06-04
+
+Versioned snapshots of a machine's settings (machine_params.toml + curated
+local_state), pushed by the sorter. The sorter content-hashes before sending,
+so a new row lands only when the config actually changed.
+"""
+
+from __future__ import annotations
+
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+
+revision: str = "b1c2d3e4f5a6"
+down_revision: Union[str, None] = "e9f0a1b2c3d4"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "machine_config_backups",
+        sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("machine_id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("version", sa.Integer(), nullable=False),
+        sa.Column("content_hash", sa.String(), nullable=False),
+        sa.Column("payload", sa.JSON().with_variant(postgresql.JSONB(), "postgresql"), nullable=False),
+        sa.Column("trigger", sa.String(), nullable=False, server_default="config_change"),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+        sa.ForeignKeyConstraint(["machine_id"], ["machines.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("machine_id", "version", name="uq_machine_config_backups_machine_version"),
+    )
+    op.create_index("ix_machine_config_backups_machine_id", "machine_config_backups", ["machine_id"])
+    op.create_index("ix_machine_config_backups_created_at", "machine_config_backups", ["created_at"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_machine_config_backups_created_at", table_name="machine_config_backups")
+    op.drop_index("ix_machine_config_backups_machine_id", table_name="machine_config_backups")
+    op.drop_table("machine_config_backups")

software/hive/backend/alembic/versions/c4d5e6f7a8b9_add_experimental_to_detection_models.py

@@ -0,0 +1,46 @@
+"""add experimental flag to detection_models
+
+Revision ID: c4d5e6f7a8b9
+Revises: b1c2d3e4f5a6
+Create Date: 2026-06-08
+
+First-class flag so a model can be marked experimental. Experimental models are
+hidden from the default Browse on both Hive and the sorters (opt back in with
+?include_experimental=true) so operators don't install a test model by accident.
+server_default false backfills existing rows.
+"""
+
+from __future__ import annotations
+
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+revision: str = "c4d5e6f7a8b9"
+down_revision: Union[str, None] = "b1c2d3e4f5a6"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "detection_models",
+        sa.Column(
+            "experimental",
+            sa.Boolean(),
+            nullable=False,
+            server_default=sa.false(),
+        ),
+    )
+    op.create_index(
+        "ix_detection_models_experimental",
+        "detection_models",
+        ["experimental"],
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_detection_models_experimental", table_name="detection_models")
+    op.drop_column("detection_models", "experimental")

software/hive/backend/app/main.py

@@ -14,6 +14,7 @@
     api_keys,
     auth,
     leaderboard,
+    machine_config_backups,
     machine_lookup,
     machine_models,
     machine_parts,
@@ -70,6 +71,7 @@ async def lifespan(_app: FastAPI):
 app.include_router(admin.router)
 app.include_router(admin_parts.router)
 app.include_router(machines.router)
+app.include_router(machine_config_backups.router)
 app.include_router(machine_lookup.router)
 app.include_router(profiles.router)
 app.include_router(upload.router)

software/hive/backend/app/models/__init__.py

@@ -22,6 +22,7 @@ class Base(DeclarativeBase):
 from app.models.sorting_profile_ai_message import SortingProfileAiMessage  # noqa: E402, F401
 from app.models.machine_profile_assignment import MachineProfileAssignment  # noqa: E402, F401
 from app.models.machine_set_progress import MachineSetProgress  # noqa: E402, F401
+from app.models.machine_config_backup import MachineConfigBackup  # noqa: E402, F401
 from app.models.detection_model import DetectionModel, DetectionModelVariant  # noqa: E402, F401
 from app.models.user_api_key import UserApiKey  # noqa: E402, F401
 from app.models.teacher_job import TeacherJob, TeacherJobItem  # noqa: E402, F401

software/hive/backend/app/models/detection_model.py

@@ -32,6 +32,10 @@ class DetectionModel(Base):
     scopes = Column(JSON_VARIANT, nullable=True)
     training_metadata = Column(JSON_VARIANT, nullable=True)
     is_public = Column(Boolean, nullable=False, default=True)
+    # Experimental models are first-class but hidden from the default Browse on
+    # both Hive and the sorters, so operators don't install a test model by
+    # accident. Opt back in with ?include_experimental=true.
+    experimental = Column(Boolean, nullable=False, default=False, server_default="false")
     # Human-friendly handle drawn from a curated LEGO-color word list — like Ubuntu's
     # codenames. Picked by :func:`app.services.codenames.next_codename` alphabetically
     # at publish time; persistent so people can refer to "Bronze beats Aqua by 1.5 %"
@@ -62,6 +66,7 @@ class DetectionModel(Base):
         Index("ix_detection_models_codename", "codename", unique=True),
         Index("ix_detection_models_model_family", "model_family"),
         Index("ix_detection_models_is_public", "is_public"),
+        Index("ix_detection_models_experimental", "experimental"),
     )