Firmware: wireless upload, several custom patterns per build, and rebuilding only what changed

[engmung]

Three firmware-side pieces of work, now that building and flashing from the browser works end to end (#230). They are listed together because they touch the same files, but they are independent and can land separately.

1. Wireless upload — the device pulls, nothing pushes

Flashing over USB works and takes about twenty seconds, so this is convenience rather than a blocker. It matters for a Patternflow mounted on a wall, an installation, or anyone with more than one device.

What cannot work, and why — worth writing down, because two of these look plausible until you try them:

• OSC is not a transport. It carries knob and state messages over UDP. It can trigger an update ("there is one waiting"), but it is not how 1.1 MB travels.
• ArduinoOTA — what the firmware has today — is LAN push. The uploader has to reach the device on UDP 3232. A browser cannot send UDP at all, and the build server cannot reach a device behind someone's home NAT.
• Browser straight to the device over the LAN is blocked. An HTTPS page cannot fetch http://192.168.x.x; mixed content stops it, and the community is HTTPS.

What does work: the device fetches over HTTPS. An outbound connection traverses NAT with nothing to configure, and neither the browser nor the server needs to reach the device.

The hardware makes the pairing pleasant — there is a display and four encoders:

1. Long-press into an update screen. The device shows a short code and starts polling.
2. On the web, after a build, enter that code.
3. The server maps the code to the build; the device's next poll returns a URL, downloads, flashes, reboots.

Device-shows / human-types is the right direction: typing six characters on a keyboard beats turning a knob to dial them in.

What it needs

• HTTPUpdate (or esp_https_ota) in the firmware, plus the update-mode screen and polling loop
• A code→build endpoint on the server, short-lived and rate-limited
• TLS done properly. setInsecure() is not acceptable here — a man in the middle could push arbitrary firmware. Pin the root CA and verify the image hash (HTTPUpdate supports an MD5 check).
• Rollback on a failed update: the partition table already has app0/app1 and otadata, but the app must mark itself valid after a successful boot or a bad image can strand a device that is hard to reach.

Note the chicken-and-egg: this capability has to arrive by USB once. It ships in a stock release, and everyone who flashes that gets wireless updates from then on.

2. More than one custom pattern per build

Worth correcting a common assumption: the firmware already has three custom slots (custom1.h…custom3.h, three entries in customPatterns[]), and the build service's assembler already handles up to three. The reason only one arrives is that the web UI sends one — see #233.

So this is mostly a web change. On this side:

• Decide whether three is the right number. Nothing structural stops more: the app image is ~1.1 MB in a 3 MB partition, so space is not the constraint. Build time is — every additional pattern lands in the same translation unit (see below).
• Make sure the on-device pattern list stays sensible with several customs — ordering, and long-press navigation not becoming tedious.

3. Rebuild only the pattern that changed

A warm build is 14–16 s on the Pi, nearly all of it recompiling one translation unit. Arduino concatenates the sketch, and every pattern header is textually included into it, so changing one custom slot recompiles all ~34 presets along with it. Extra cores do not help: it is one file.

That is fine today. It stops being fine as the preset library grows, since build time tracks the total amount of pattern code rather than what changed.

A shape worth considering. Splitting every pattern into its own .cpp is the obvious answer, but it is a large refactor and would break the header-only .h format that community patterns are already written and shared in. A smaller change gets most of the win:

• Give the custom slots fixed namespaces (Custom1, Custom2, Custom3) and have the assembler rename the submitted pattern into its slot. The displayed name comes from the NAME constant, not the namespace, so nothing changes for the author.
• With fixed namespaces, pattern_registry.h never changes between builds — today it is regenerated every time, which is itself enough to force a full recompile.
• Compile each custom slot as its own translation unit (a generated customN.cpp that includes the submitted .h), leaving the presets in the main unit where they belong, since they do not change.

Result: a build recompiles one small file and relinks, instead of recompiling the entire pattern library. Rough guess 15 s → 5 s, most of what remains being the link.

• Confirm the assumption first by timing a link-only rebuild — if linking is 10 of the 15 seconds, this is not worth doing
• Keep accepting the existing .h format; community patterns already in circulation must not need editing
• Sanity check: separate translation units mean no accidental reliance on other patterns' symbols leaking through the shared unit

Trigger for doing it at all: when a warm build passes ~30 s. There is no reason to touch this while it sits at 15.

[engmung]

Better idea for §1: use the web server the device already runs

Rather than a code-and-poll pairing flow, upload straight to patternflow.local. The device already runs WebServer httpServer(80) (src/core_audio_ws.h) and already answers on mDNS — the only missing piece is Update.h, which is about fifty lines.

This removes the hardest part of the original plan: TLS.

browser ──HTTPS──▶ community server   (download the .bin — already implemented)
browser ──HTTP───▶ patternflow.local  (LAN, no certificates involved)

Having the device pull over HTTPS means pinning a root CA on an ESP32, handling certificate expiry, and paying the TLS handshake in RAM. Taking the bytes over plain LAN HTTP and handing them to Update.write() avoids all of it. Given the payload is a firmware image, "no certificate to get wrong" is worth a lot.

The artifact is already the right shape. The build service emits the application image only (~1.1 MB) and serves the bootloader and partition table statically — which is exactly what Update.h expects for U_FLASH. No server-side change at all, and the Download .bin button already exists.

Mixed content is not a problem here, though it looks like it should be: an HTTPS page cannot POST to http://patternflow.local, but the user opening patternflow.local directly and dropping a file there is a page talking to its own origin.

Three ways to do it, in increasing cost

	TLS on device	Server change	What the user does
A. Drop the .bin on patternflow.local	none	none	download, then drag
B. Paste the firmware URL, device fetches it	required	none	copy a URL, paste it
C. Pairing code + polling (the original sketch above)	required	yes	type a short code

A first. It is the smallest, it keeps working when the community server is down or unreachable, and B and C can be layered on later without redoing it. The build URLs are already capability URLs, so B is a small addition once TLS exists.

Problems worth solving deliberately

• mDNS is not universal. Fine on macOS and iOS, fine on Windows 10+, unreliable on Android. The device should show its IP on the matrix as a fallback rather than assuming .local resolves.
• An unauthenticated /update lets anyone on the Wi-Fi flash the device. The fix doubles as the UX: only open the endpoint while the device is in an update mode entered physically from an encoder. That makes it both a safety catch and the "ready for it now" signal.
• Rollback. A failed update on a wall-mounted device is the worst case. The partition table already has app0/app1 and otadata; the app must mark itself valid after a successful boot so a bad image rolls back instead of stranding the device.

Unchanged: this capability has to arrive by USB once, then updates are wireless from there.

[engmung]

§1 shipped: the device updates itself from the browser

Everything is on dev (tip 63ffcc4). The shape is option A from the comment above — drop a .bin on the device's own page — plus the pieces around it.

Firmware (src/core_web_update.h, src/web_update_index.h): /update rides the web server the device already runs on port 80. GET serves a drop-zone page; POST streams the multipart body straight into Update.h (U_FLASH), replies, and reboots ~1.2 s later so the response flushes. A failed or interrupted upload leaves the running firmware untouched — the boot partition only switches after a complete, verified image. Flash progress is drawn on the panel from inside the upload handler, because the whole POST is consumed in one handleClient() call and the main loop never runs while flashing. After 8 s of healthy uptime the app calls esp_ota_mark_app_valid_cancel_rollback() — an honest no-op on the stock Arduino bootloader (rollback isn't compiled in); Update.h's own checks are the real protection.

Access model — deliberately different from the sketch above. PF_WEBUPDATE_ALWAYS_ARMED defaults to 1: /update accepts firmware at any time, no trip to the device. Stated plainly, anyone on the Wi-Fi can then flash it — the same exposure ArduinoOTA's no-password default has always had, and drop-whenever is the UX this project wants. Building with 0 restores the physical arming switch: uploads are refused (403) unless the UPDATE screen is open on the device (hold K2 → NETWORK, turn K4); leaving it (or 10 min idle) disarms.

mDNS reality, exactly as §1 predicted: Android can't resolve .local, so the UPDATE screen shows the raw IP alongside patternflow.local/update, and the web pages repeat the fallback.

Console: / is now a landing page (audio sync · firmware update · remote-compute placeholder), the audio UI moved to /audio, all pages restyled to the patternflow.work design system, and every console page is Cache-Control: no-store — the HTML ships inside the firmware and changes with it, and a response truncated by the post-flash reboot must never stick in a cache.

Mobile: accept="*/*" so Android offers a real chooser, and — better — the build modal grew Send over Wi-Fi: the device page receives ?src=<bin URL>, fetches the image itself, and flashes. No file ever touches the phone. The firmware download route serves public CORS for that fetch. Live verification of that last hop is tracked in #236.

Dogfooded: after the first USB flash, every firmware iteration of this very feature went onto the hardware through /update (~8 s per 1.16 MB image).

Remaining for §1 → #236: verify the Wi-Fi handoff end to end on the community host, and cut a stock release so devices in the field pick the capability up over USB once.

🤖 Generated with Claude Code