Repository navigation
Conversation
2 tasks done
Scope the playing-simulation feature (issue #41): a virtual CDJ that advances a track's beat grid at pitch and broadcasts live CDJ status + per-beat packets, controllable at runtime. Doubles as a software test rig for the monitor/overlay/MPRIS/history, which today only work against the physical rig. Grounds it in the current code (we parse every 0x0a field, so emitting is the inverse; MarshalStatusCDJ is static-idle today; no 0x28 beat emitter; no deck engine), lays out Phase 1 (status/beat emulation, no audio) vs Phase 2 (audio + master tempo), the one real RE task (the 0x28 beat packet), the load-bearing validation (byte-diff vs a real CDJ, test-rig first), the risks (track-end-load-bug territory; confusing live gear; device-number collision), and the open decisions.
…core) proto.MarshalStatusCDJPlaying overlays the dynamic fields of a playing CDJ status (play state, loaded track, pitch, BPM, beat-in-track, beat-in-bar, sync/master/on-air flags) onto the known-good idle template built by MarshalStatusCDJ. It sets exactly the fields ParseCDJStatus reads back, so a built status round-trips through our own parser and can drive our own monitor/overlay/MPRIS/history. The NXS2 mirror copies of pitch/BPM we do not parse are left at template values; byte-exact fidelity against a real CDJ is a Phase-0 (capture + diff) task. device.SimDeck is the engine: a clock over a beat grid that advances a playhead at pitch and reports a proto.CDJPlayState. load/play/pause/cue/seek/ setpitch, beat-in-track and beat-in-bar derived from the grid + downbeat, end-of-track handling, effective BPM. The time source is injectable; the clock, pitch scaling, pause-freeze, end clamp, and downbeat offset are unit tested with no hardware. Also factor proto.PlayStateName out of the CDJStatus method so the deck can name states. Decisions locked in the design note: status-only first, isolated/test-rig first, control via API+CLI behind --simulate. No wiring into the live broadcast loop yet (next step), so nothing emits on the network; this is pure, tested machinery.
…late) Add a --simulate flag (implies --mode cdj) that attaches a device.SimDeck to the VirtualDevice. In CDJ mode the status broadcast loop then emits a dynamic playing status built from the deck snapshot (with an incrementing sequence counter) in place of the static idle status; with no track loaded the deck reports NoTrack, so behaviour is unchanged until a track is loaded. The safe profile (not tempo master, not on-air) is the default, per the isolated test-rig decision. No control surface yet: the next step adds /api/sim and the 'vynull sim' CLI to load/play/pause/cue/seek/pitch the deck. Nothing new emits on a live network until --simulate is passed.
Drive the virtual playing deck at runtime:
/api/sim/status GET the deck's state (SimDeckStatus JSON)
/api/sim/load POST {track_id|file_path} -> pull the beat grid from
analysis and cue the track
/api/sim/play|pause|cue|eject POST transport
/api/sim/seek POST {position_ms}
/api/sim/pitch POST {pitch_pct}
Every call returns the current status. 503 when the server was not started
with --simulate; 404 for an unknown action or an unanalyzed track.
'vynull sim <status|load|play|pause|cue|seek|pitch|eject>' is the matching
CLI client over that API, with --json for pipelines; seek takes ms or m:ss,
pitch takes a percent. Factor the one-track resolver (ID or unique query)
out of 'vynull load' so 'sim load' reuses it.
Covered by an httptest end-to-end flow (load->play->pitch->pause->seek) plus
the error paths; no device claim needed. Full suite green.
listenStatus drops packets from our own source IP, so a --simulate deck broadcasts on the wire but is invisible to this instance's own UI. Parse the status bytes we just built and feed them to the monitor so the virtual deck shows up locally: /api/players, the TUI, the now-playing card (it resolves from our own DB, and selectNowPlaying only needs playing + a title), MPRIS, and history. Parsing the very bytes we broadcast keeps the local view identical to what a peer sees on the wire. Only active under --simulate. This closes the single-host test-rig loop: load/play a track via 'vynull sim' and watch our own overlay and monitor follow it.
Add an EMULATOR tab to the web UI over the /api/sim endpoints. It probes /api/sim/status and reveals itself only when the server runs with --simulate (the endpoint 503s otherwise), so the tab is invisible in normal operation. The panel is a virtual CDJ face: a transport (play/pause/cue/eject with the right buttons enabled per state), a seek bar (drag to scrub, released value posts /api/sim/seek), a pitch slider with a reset, and a live readout showing state, the track title (resolved from the library), base to effective BPM, pitch, and the bar-beat dots. A search box filters the library (or takes a bare track ID + Enter) and a click loads the track. It polls the deck at 2 Hz while visible so the readout follows playback; the deck already feeds the monitor, so the now-playing card and overlay move in step. No new server code: pure client over the existing API. Page JS parses clean (node --check); go build/vet/test green.
/api/sim/load called Analysis.Get, which only returns an in-memory or already-path-mapped result. After a server restart the in-memory map and the track->path map are empty, so Get returned nil for a track analyzed in an earlier session and the load 404'd with 'no analysis for track' even though a valid .gob cache (or the file itself) was right there. Switch to the server's getOrAnalyze helper (as the waveform/analysis endpoints already do): it resolves the file path from the library/PDB, SetPaths it so the on-disk cache loads, and only analyzes synchronously as a last resort. A long-library track now loads instantly from cache; an un-analyzed one is analyzed on the spot.
SimDeck.Eject did '*d = SimDeck{...}' while holding d.mu, which replaced the
locked mutex with a fresh zero-value one; the deferred d.mu.Unlock then
unlocked a never-locked mutex and panicked with 'sync: unlock of unlocked
mutex', taking down the server.
Reset the track/grid/transport/playhead fields in place instead, leaving the
mutex (and pitch + the sync/master/on-air flags) intact. Add a test that
ejects a playing deck, confirms it clears, ejects again, and reloads, which
reproduces the panic on the old code.
Replace the single SimDeck on VirtualDevice with a SimManager holding up to 4
decks keyed by Pro DJ Link player number (1-4). The broadcast loop now emits
one dynamic status per deck under its own number and feeds each into the
monitor, so several virtual CDJs appear as distinct players. --simulate starts
with one deck as Player 1.
SimManager: Add (0 = lowest free number), Remove, Renumber (keeps the loaded
track + transport), Get, Len, Snapshots, Statuses; all numbers validated to
1-4 and unique, capped at 4.
API routing splits into manager-level and per-deck:
GET /api/sim/status list every deck + the cap
POST /api/sim/add {number?}
POST /api/sim/remove {number}
POST /api/sim/renumber {number, to}
/api/sim/{n}/{load|play|pause|cue|seek|pitch|eject|status}
A removed deck stops broadcasting and ages out of the monitor within 5s. Each
deck's track still resolves against our own DB (TrackDevice = our source
number, so externalSource stays false). Covered by SimManager unit tests and
reworked API tests (per-deck flow + add/remove/renumber/full/collision).
Web UI (single-deck panel) still to update for the multi-deck layout.
Rework the EMULATOR tab for the SimManager. A toolbar shows the deck count and an ADD CDJ button (disabled at the cap of 4); each virtual deck renders as a card with a player-number dropdown (renumber to any free 1-4 slot), a remove button, and the same transport/seek/pitch/readout as before. A shared loader at the bottom searches the library (or takes a track ID) and loads the picked track into the deck chosen in an 'into Player N' selector. Cards rebuild only when the set of player numbers changes (add/remove/ renumber); otherwise values update in place each 500ms poll, with per-deck seek/pitch drag guards so a drag is not fought by the poll. All actions go through /api/sim and refresh from a fresh list, so the view stays consistent. Pure client; page JS parses clean (node --check).
Rework the sim subcommand for the SimManager routes. New manager subcommands: vynull sim status table of every virtual deck (--json for scripts) vynull sim add [number] add a deck (omit number = lowest free) vynull sim remove <number> vynull sim renumber <old> <new> Transport subcommands (load/play/pause/cue/seek/pitch/eject) target one deck via --deck N, defaulting to the lowest-numbered deck when omitted, so the common one-deck case still reads 'vynull sim play'. --deck is stripped before the shared flag parser (which would reject it). load reuses resolveOneTrack (ID or unique query); seek takes ms or m:ss; pitch takes a percent. Smoke-tested arg handling (routing, defaults, usage + bad-input errors).
Each virtual deck now announces itself with a 0x06 keep-alive on port 50000 every 1.5s (alongside our own device's), so device trackers list the virtual CDJs as players and associate their 0x0a status. beat-link's DeviceFinder (and real gear) track devices from these keep-alives; without them a status-only emulator stayed invisible to those tools. All decks share our MAC/IP with distinct device numbers, the same one-IP-many-numbers pattern rekordbox uses for its claimed slots, so no per-deck claim handshake is needed. Add SimManager.Numbers() for the loop to enumerate player numbers without snapshotting. Only active under --simulate; our own number is announced once.
The EMULATOR beat dots updated only on the 500ms status poll, so against a sub-500ms beat period they jumped and visibly skipped. Extrapolate each playing deck's current beat from its effective BPM in a requestAnimationFrame loop, re-anchoring to the authoritative beat_in_bar on every poll so it self-corrects. The beat now ticks in time with the music. Split the beat dots into their own .sim-beat span (separate from the BPM/pitch text) so the animator can update them without rebuilding the readout. The loop runs only while the EMULATOR tab is visible.
The smoothed beat dots still flashed through a beat every few seconds: the server's beat_in_bar reflects the deck position at sample time, but the client pinned that value to the arrival timestamp. When a sample landed just before a beat boundary, the current beat was held an extra fraction and the next poll jumped forward, flashing one beat quickly. Expose the exact sub-beat phase from the deck (beat_fraction, 0..1, computed from the grid in SimDeck.Status) and seed the animator's crossing count with it: crossings = floor(frac + elapsedBeats). The next flip now happens at the true wall-clock moment, so the dots neither freeze nor jump; re-anchoring each poll stays continuous. Fraction covered by a unit test.
In --simulate the keep-alive loop announced BOTH our own device number (the cdj-mode source identity, default 3) and each virtual deck. Trackers like prolink-tools and beat-link list anything that sends a 0x06 keep-alive, so our source showed up as a phantom CDJ with no track or status alongside the real virtual decks. Route the startup and periodic keep-alives through simPresence: in --simulate it announces only the virtual deck numbers, never our own source number; otherwise behaviour is unchanged (own 0x06 in both modes; the rekordbox startup 0x02 burst is untouched). The device still binds its services on its own number; it just no longer advertises a presence that has no deck behind it. The one-shot claim sequence uses claim packets (0x00/0x02), not 0x06, so it never registered our number as a tracked device.
Add a per-request debug log of the message type and its first argument, which for menu/metadata requests encodes player, slot, and track type. Makes it easy to see exactly what a client (a real CDJ, or a tool like prolink-tools) asks for when diagnosing metadata resolution. Debug level so normal sessions stay quiet; run with --log-level debug to watch.
…racks prolink-tools showed our virtual CDJ's play state but left its overlay blank: it fetches a loaded track's metadata from the device named as the track's source, and our decks named device 3 (our cdj-mode identity) as the source while we no longer announced it, so there was nothing to query (confirmed from logs: zero dbserver/NFS fetch attempts). Model the real rekordbox-link topology instead: --simulate now runs in rekordbox mode (device 17, a discoverable metadata source), and the virtual CDJ decks play tracks linked from it (TrackDevice = 17, rekordbox slot). A metadata client then queries device 17's dbserver (us) for each deck's track; our own monitor still resolves locally since TrackDevice == our SelfDevice. - config: --simulate implies rekordbox mode (was cdj). - statusBroadcastLoop: a dedicated simBroadcastLoop emits per-deck 0x0a regardless of base mode, without waiting on a CDJ link; the base rekordbox identity broadcasts no player status of its own. - keep-alives: announce our rekordbox base (not a phantom now, it is a rekordbox source) plus each virtual CDJ deck (announceSimDecks). - sim load: source slot is rekordbox, not USB, to match the source device. Full suite green.
…erge) A CDJ status packet carries the player number at BOTH 0x21 and 0x24. MarshalStatusCDJ only set 0x24 and left the idle template's 0x01 at 0x21 (the template was captured from player 1). beat-link and prolink-tools key a status update by 0x21, so every virtual deck's status was attributed to player 1 and their track/play-state merged in those tools. Our own monitor reads 0x24, so it was unaffected and the bug stayed hidden. Set buf[0x21] = deviceNumber in MarshalStatusCDJPlaying (the --simulate path only; the hardware-validated idle status is untouched). Tests assert both 0x21 and 0x24 carry the deck number for players 1-4.
Now-playing overlays (prolink-tools, and real rigs) pick the live deck by its
on-air flag, so with the safe profile (on-air off) their overlay stays empty
even though device-status shows full metadata. Expose control of it:
- SimDeck.SetOnAir / SetMaster; SimDeckStatus gains on_air/master/sync.
- POST /api/sim/{n}/onair {on} and /api/sim/{n}/master {on}.
- 'vynull sim onair|master [on|off] [--deck N]'; status line + list show flags.
- EMULATOR web cards get ON-AIR / MASTER toggles (on-air lit red, master
accent), reflecting and setting the deck's flags.
Still off by default (safe profile); the user opts a deck on-air to drive a
now-playing overlay. Full suite green; page JS parses clean.
The dbserver reported art ID 0 for any track whose embedded cover had not been extracted yet (extraction was lazy, triggered only by the web /api/artwork endpoint), so a metadata client like prolink-tools requested artwork 0 and got nothing — the now-playing overlay showed no cover for those tracks. Factor the web endpoint's lazy extraction into Server.EnsureArtwork(trackID) and share it with the dbserver via a hook: - Metadata (0x2002) reports the track ID as the artwork handle when it has no art ID yet, so the client actually requests art (0 made it skip). - The artwork handler (0x2003/0x2103), after the existing cache/ArtID lookups, calls EnsureArtwork to extract the embedded cover on demand and retries; the result still passes the oversize-downscale guard. Extraction is deferred to the art request (not metadata), keeping the metadata response fast. Helps real CDJs too: a lazily-analyzed track now serves its cover without a prior web-UI visit. Full suite green.
The Library page's LOAD-to-player menu already lists the virtual decks (they
appear in /api/players via the monitor feed), but loadTrack always posted to
/api/load, whose 0x19 command can't reach a deck that isn't a real peer. Route
a target that is a virtual deck to /api/sim/{n}/load instead (looked up from
/api/sim/status), so you can send a library track straight onto a virtual CDJ.
Real-CDJ loads are unchanged. The play-interrupt confirm still applies, since
virtual decks are in currentPlayers too.
Each virtual deck can now advertise its own model name, so a CDJ-3000 and
an XDJ-RX3 can run side by side. The name is the model string other devices
and clients display; blank falls back to the host device's name. It lives on
SimDeck (persisting across load/eject, like a physical device's) and flows
into both the per-deck keep-alive and status packets.
A + ADD DEVICE modal (player number + model name) replaces the instant add,
and sets the name at creation so the deck advertises the right model from its
very first announce. This matters because clients latch a device's identity
when they first see it, so naming it after it is already on the network is too
late. The per-card model field and `vynull sim name <model>` edit a live deck.
Control surfaces: POST /api/sim/{n}/identity {name}; /api/sim/add also takes a
name; `vynull sim name`; the modal and per-card field in the web panel. The
toolbar label and add button are device-generic.
Virtual devices stay CDJs on the wire: a track-playing deck can't meaningfully
present as a mixer or rekordbox (it emits a CDJ status packet), so there is no
device-type control for now.
Tests cover the model name default, set, persistence across load/eject, and
snapshot propagation.
Seed per-model emulation with a built-in table of known players. A DeviceProfile carries the identity a client reads off the network, and the table (device/profiles.go) lists CDJ-3000, CDJ-2000NXS2/nexus/2000, and XDJ-XZ/RX3/1000MK2/AZ, keyed to match link-spec's per-model goldens so a profile can later be validated against its <model>-status golden. GET /api/sim/profiles serves the table, and the web panel offers the models as autocomplete suggestions (a datalist on the ADD DEVICE modal and the per-card model-name field); any free-text name still works. A profile carries only Name for now. Firmware (status offset 0x7c, currently the fixed "1.85") and capability fields are deliberately left empty until they can be filled from captures and the <model>-status goldens and plumbed into the keep-alive/status, rather than asserting versions we have not verified.
vynulldev
force-pushed
the
cdj-emulator
branch
from
October 10, 2026 20:22
512638d to
cc3ec83
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Phase 1 of the CDJ emulator from issue #41 (playing-simulation mode): Vynull can act as one or more virtual CDJs that play a track, advancing a real beat grid at a set pitch, broadcasting a live playing status, announcing themselves on the link, and controllable at runtime.
Beyond #41 this is a software test rig. Our monitor, stream overlay, MPRIS bridge, and play history could until now only be exercised against the physical rig; a virtual playing deck drives all of them on a loopback or isolated network with no hardware in the room.
This phase is status-only and off by default. There is no audio and no per-beat (0x28) packet yet. Nothing changes in normal operation unless you pass the new
--simulateflag. Full scope, decisions, and phasing live indocs/design/cdj-emulator.md.How it works
proto.MarshalStatusCDJPlayingoverlays the dynamic fields of a playing CDJ status (play state, loaded track, pitch, BPM, beat-in-track, beat-in-bar, sync/master/on-air flags) onto the existing known-good idle template. It sets exactly the fieldsParseCDJStatusreads back, so a built status round-trips through our own parser.device.SimDeckis the engine: a clock over a beat grid that advances a playhead at pitch and reports aCDJPlayState. No audio, just time over the grid: load/play/pause/cue/seek/pitch, beat-in-track and beat-in-bar derivation, end-of-track, effective BPM. The time source is injectable for tests.device.SimManagerholds up to 4 of those decks keyed by Pro DJ Link player number (1-4).--simulateruns Vynull as a rekordbox metadata source (device 17), with the virtual decks as CDJ players that play tracks linked from it: the status broadcast loop emits one dynamic status per deck, each under its own player number, and each deck also announces itself with a 0x06 keep-alive on port 50000.--simulatestarts with a single deck as Player 1. Modelling Vynull as the rekordbox source (rather than the decks as the source) is what lets metadata clients resolve each deck's track: they fetch it from the source device's dbserver, which is us.Local visibility and presence
listenStatusdrops packets from our own IP, so without help the emulator would be invisible to this instance's own UI. In--simulatethe broadcast loop parses the very bytes it sends and feeds them to the monitor, so the virtual decks appear in/api/players, the TUI, the now-playing card, MPRIS, and history on the same host. Each deck's track resolves against our own database (its source device number is ours, soexternalSourcestays false), so titles and artwork are correct.For other software and real gear, each deck sends a 0x06 keep-alive per 1.5s tick, which is what beat-link's DeviceFinder and real players use to list devices. All decks share our MAC/IP with distinct device numbers, the same one-IP-many-numbers pattern rekordbox uses for its claimed slots, so no per-deck claim handshake is needed. In
--simulateonly the deck numbers are announced, not our own source device number, so trackers list exactly the virtual players and not a phantom entry for our controller identity (which carries no status).Control surface
Manager-level, under
/api/sim/:statusadd{number?}remove{number}renumber{number, to}Per-deck, under
/api/sim/{number}/:status(GET),load({track_id|file_path}),play/pause/cue/eject,seek({position_ms}),pitch({pitch_pct}),onair({on}),master({on}). On-air and master are off by default (the safe profile); a now-playing overlay follows the deck marked on-air.CLI (
vynull sim ...,--jsonfor pipelines):Transport subcommands target one deck via
--deck N, defaulting to the lowest-numbered deck, so the common one-deck case still readsvynull sim play.Web UI: an EMULATOR tab that reveals itself only when the server runs with
--simulate(it probes/api/sim/status, which 503s otherwise). Each deck is a card with a transport, seek bar, pitch slider, a live readout (state, title, base to effective BPM, bar-beat dots), a player-number dropdown to renumber, a remove button, and ON-AIR / MASTER toggles; an ADD CDJ button and a shared "load into Player N" library search sit alongside. The bar-beat dots animate smoothly between polls (see below); it otherwise polls at 2 Hz while visible, and the decks already feed the monitor, so the now-playing card and overlay move in step.The bar-beat dots are animated in a requestAnimationFrame loop rather than redrawn on the 2 Hz poll, so they step in time with the music instead of stuttering against the poll clock. The loop extrapolates each playing deck's beat from its effective BPM and is seeded with the exact sub-beat phase the deck reports (
beat_fraction, 0..1 through the current beat, computed from the grid), so the next flip lands at the true wall-clock moment; every poll re-anchors it, so it self-corrects without freezing or flashing. This is still an interpolation of the status, not a hardware-exact lock, which arrives with the 0x28 beat packet in the next phase.Try it (isolated network recommended)
The web overlay, the monitor, and a tool like beat-link-trigger then follow the virtual players.
Hardware testing
--simulateoff (the default) there is no deck-facing change, and the status/announce paths are byte-identical to before. The--simulatepath has been validated against prolink-tools (an independent prolink-connect client) on a LAN that also had a real DJM-A9 present: it lists Vynull as a rekordbox source plus the virtual CDJ players, resolves each deck's track metadata (title/artist/genre/key/BPM) from our dbserver, and its Now-Playing overlay follows whichever deck is toggled on-air. Not yet put in front of real CDJ players.The design validates the test-rig way first. The dynamic status sets exactly the fields
ParseCDJStatusreads, verified by a round-trip test, so it drives our own monitor/overlay/MPRIS/history without hardware. Byte-exact fidelity of the status against a real CDJ-2000NXS2, and the 0x28 beat packet (so trackers lock beat phase, not just presence), are deferred to the next phase (gated on a real-deck capture). Until then--simulateships behind a safe profile: not tempo master, not on-air, documented as isolated/test-rig first. A careful live-rig smoke is the next phase, not this PR.Scope and tests
~2200 lines across
proto(dynamic status),device(SimDeck + SimManager + broadcast/keep-alive wiring),api(the/api/simroutes),cli.go, the web UI, and the design note. Covered by: proto status round-trip + pitch tests; SimDeck engine tests (advance-at-pitch, pause-freeze, end, seek-from-end, downbeat offset, eject, sub-beat fraction); SimManager tests (add/remove/renumber, auto-number, full/collision, injected clock); and httptest coverage of the API (per-deck flow + add/remove/renumber/full/collision + error paths).go build/vet/test ./...pass; the web page JS parses clean undernode --check.Checklist
go build ./...,go vet ./..., andgo test ./...passgofmt -l .is cleanGPL-3.0-or-later)--simulatepath is status-only, round-trip-tested, and gated to isolated use pending the next-phase hardware smoke)