AI Recording Workflow Assistant for REAPER DAW. SessionPilot's core promise is your session never gets stuck — it monitors your session health in real time, greets new sessions with a quick brief, guides you through recording workflows in natural language, and remembers your preferences across sessions.
- Natural language recording assistant (text, voice, and quick-action buttons)
- Two assistant modes (details) — a fast, deterministic Default mode, and an AI Engineer mode where an LLM agent inspects live session state and proposes multi-step plans you confirm before anything runs
- Session intake flow — quick onboarding brief at the start of each session (goal + genre)
- "What should I do next?" session advisor — context-aware suggestions based on current session state
- Persistent user profile — remembers your preferred track style, punch-in method, genre, and custom notes across sessions
- Optional Claude LLM fallback (Default mode) for ambiguous or complex messages
- Context-aware fallback suggestions based on live session state
- Real-time health warnings in the sidebar — proactively flags issues on every state update:
- Recording with no armed tracks
- Armed tracks with input monitoring off
- Duplicate inputs on multiple armed tracks
- No input assigned to armed track
- FX-heavy chains while tracking (latency risk)
- No tracks in session
- Severities: error (red), warning (amber), info (blue)
- Vocal session setup (lead, doubles, adlibs, full stack)
- Punch-in and loop-punch preparation with pre-roll
- Comp takes review and selection
- Song structure marking (verse/chorus/bridge markers and regions)
- Batch recording setup (playlist-style multi-song sessions)
- Transport controls (play, stop, record, pause, go-to-bar) from chat, voice, keyboard, or UI
- Keyboard shortcuts (Space, R, Home, End, Esc)
- Volume and pan control via voice commands, chat, or UI sliders
- FX chain management (insert, remove, bypass, and parameter control per track)
- FX parameter control with real-time sliders (EQ freq, compressor threshold, etc.)
- MIDI/instrument track creation with automatic input routing
- Session template system — save and restore full session configurations
- Track organization with folder management, color-coding, and rough mixing
- Monitoring and input level diagnostics
- Marker navigation ("go to chorus", "jump to verse 2")
- Undo/redo with history log
- Export/bounce workflows (mix, stems, or both)
- Natural language MIDI composition — chord progressions, scale runs, and direct note entry
- Music theory engine powered by tonal (chords, scales, voicings)
- Peak meter visualization (per-track sidebar meters and selected track stereo meter)
- Track type detection and display (audio, MIDI, instrument, folder)
- Action preview and confirmation for destructive operations
- Transport-aware action gating (blocks destructive ops while recording)
- Command chaining ("arm track then hit record")
- Action queue for sequential command execution
- Voice command help overlay (press ? or click help button)
- Real-time session state display via WebSocket with file-change-driven updates
SessionPilot runs in one of two modes, switchable at any time from the Mode: Default / Mode: AI Engineer toggle in the UI. Your choice is saved per user in user_profile.json, so it persists across sessions.
Deterministic, regex-first command routing with a narrow LLM fallback for ambiguous phrasing. No round-trip to an LLM for most commands, so it's fast, free, and private. Best when you want quick, predictable, hands-on control of the session.
Routes non-trivial requests to an LLM agent (claude-sonnet-5, with prompt caching to keep cost down) that can inspect live session state — tracks, FX, takes, transport — and propose a multi-step plan for you to review before anything runs. Trivial, unambiguous commands (play, stop, arm, mute, solo, undo/redo, transport navigation, etc.) still execute instantly and deterministically even while in AI mode — they don't need to go through the agent. Reads (inspecting the session) happen automatically as the agent reasons; any action that changes your session always requires your confirmation first.
AI Engineer mode requires ANTHROPIC_API_KEY to be set on an account with API credits. It uses claude-sonnet-5 with prompt caching to keep per-request cost low. If the key is missing or the account has no credits, AI mode degrades gracefully to Default-mode behavior rather than failing.
npm install
npm start
# Open http://localhost:3000
Run the test suite (no extra dependencies — uses Node's built-in test runner):
npm test
+----------------------------------------------------------------------+
| Frontend (Vanilla JS) |
| Chat | Transport | Voice | Quick Actions | Shortcuts | Help Overlay |
+----------------------------------+-----------------------------------+
| REST + WebSocket
+----------------------------------+-----------------------------------+
| Express Backend |
| Chat Orchestrator |
| - Default mode: AI Orchestrator (regex) -> LLM Planner (fallback) |
| - AI mode: Agent Planner (Claude tool-use loop) + Tool Registry |
| Workflow Service (17 workflows) | Music Theory | Template Service |
| Action Routes (60+ bridge methods) |
+----------------------------------+-----------------------------------+
| Bridge Interface
+----------------------------------+-----------------------------------+
| REAPER Bridge Layer |
| MockBridge (dev) | JsonQueueBridge (file IPC) | (future: WS/HTTP) |
+----------------------------------+-----------------------------------+
| JSON file queue
+----------------------------------+-----------------------------------+
| SessionPilotBridge.lua (REAPER ReaScript) |
+----------------------------------------------------------------------+
server/- Express backendbridge/- REAPER bridge abstraction and implementationsservices/- AI orchestrator, chat orchestrator, workflow service, music theory, template service, session health check, intake service, user profile, preference parser, action log, session memory, and the AI-mode agent (agent planner, agent context, tool registry)workflows/- Individual workflow handlersroutes/- REST API endpointsmodels/- Data model factorieswebsocket/- WebSocket server (broadcasts session state + health warnings)
public/- Frontendcomponents/- UI components (sidebar with health warnings, chat, track panel, etc.)modules/- State management, API client, WebSocket client
reaper/- REAPER Lua bridge script
| Variable | Default | Description |
|---|---|---|
USE_REAL_BRIDGE |
0 |
Set to 1 to use the JSON queue bridge with a real REAPER instance |
REAPER_BRIDGE_DIR |
./reaper_bridge |
Path to the bridge directory (commands/results/state.json) |
ANTHROPIC_API_KEY |
(none) | Optional for Default mode (enables Claude LLM fallback and AI session advisor). Required for AI Engineer mode — see Assistant Modes. |
SESSIONPILOT_PLANNER_MODE |
heuristic |
heuristic (default), heuristic-only (no LLM), or off |
PORT |
3000 |
Server port |
The app defaults to MockReaperBridge for easy first-run. To connect to a real REAPER instance:
- Copy
reaper/SessionPilotBridge.luainto your REAPER Scripts folder - Load and run the script in REAPER (Actions > Load ReaScript)
- Start the server with the real bridge:
USE_REAL_BRIDGE=1 npm startExtend ReaperBridge base class and implement all methods. The bridge contract ensures the rest of the app works unchanged.
| Method | Path | Description |
|---|---|---|
| GET | /api/health | Health check with bridge status and system info |
| GET | /api/session | Project summary |
| GET | /api/tracks | All tracks |
| GET | /api/selected-track | Currently selected track |
| GET | /api/connection-status | Bridge connection status |
| GET | /api/markers | Markers and regions |
| GET | /api/workflows | Available workflows |
| POST | /api/chat | Send message to AI assistant |
| POST | /api/actions/execute | Execute action or workflow |
| POST | /api/actions/preview | Preview workflow actions |
| GET | /api/transport | Current transport state (play/stop/record position) |
| GET | /api/tracks/:id/fx/:idx/params | FX parameters for a specific plugin |
| GET | /api/session-templates | List saved session templates |
| GET | /api/session-templates/:id | Get a specific template |
| POST | /api/session-templates | Save current session as template |
| POST | /api/session-templates/:id/load | Load a session template |
| DELETE | /api/session-templates/:id | Delete a session template |
| GET | /api/action-log | Recent action history |
| GET | /api/assistant-mode | Current assistant mode (default or ai) |
| POST | /api/assistant-mode | Switch assistant mode ({ "mode": "default" | "ai" }) |
Connect to ws://localhost:3000 for real-time updates:
session_update- session state push (file-change-driven + interval fallback)initial_state- sent on connectiontransport_update- transport state changesaction_executed- sent when actions completepeak_update- track peak meter data (250ms poll)health_warnings- session health check results (sent on every state cycle)
The bridge layer is designed for future MCP server exposure:
- Bridge read methods -> MCP resources/read tools
- Bridge action methods -> MCP tools
- Workflow preview/execute -> Higher-level MCP tools
Method signatures use JSON-schema-friendly argument objects.
| Workflow | Description |
|---|---|
| setupLeadVocal | Create and arm a lead vocal track |
| setupLeadDoubleAdlib | Full vocal stack with folder |
| preparePunchIn | Set up punch-in with markers and regions |
| organizeSessionTracks | Sort tracks into folders |
| colorCodeVocals | Color-code vocal tracks by role |
| diagnoseMonitoringIssue | Troubleshoot "can't hear myself" |
| diagnoseLowInputIssue | Troubleshoot low input levels |
| setupHeadphoneMix | Set up a cue mix with reverb for vocal monitoring |
| compTakes | Review and manage vocal takes for comping |
| roughMix | Quick rough mix with levels, panning, and bussing |
| markSongStructure | Mark verse/chorus/bridge sections with markers and regions |
| sessionNotes | Add timestamped session notes as markers |
| preflightCheck | Pre-session checklist for recording readiness |
| quickPunchLoop | Loop punch-in with pre-roll and auto-crossfade |
| manageFxChain | Insert, remove, or bypass FX plugins on tracks |
| batchRecording | Playlist-style multi-song recording setup with markers and regions |
| exportBounce | Render/export mix, stems, or both |
These actions execute immediately without a workflow preview:
| Action | Description |
|---|---|
| play, stop, pause, record | Transport controls |
| goToPosition, goToStart, goToEnd | Cursor navigation (supports bar numbers) |
| goToMarker | Navigate to a named marker or region ("chorus", "verse 2") |
| armTrack, disarmTrack | Recording arm toggle |
| toggleMonitoring | Input monitoring on/off |
| muteTrack, soloTrack | Track mute/solo |
| setTrackVolume, setTrackPan | Volume and pan control |
| createTrack, renameTrack, duplicateTrack | Track management |
| createMidiTrack | Create MIDI/instrument track with optional plugin |
| moveTrackToFolder | Move track into a folder track |
| insertMarker, createRegion | Markers and regions |
| insertMidiNotes | Write MIDI notes into a track (chords, scales, melodies) |
| createMidiItem | Create an empty MIDI item on a track |
| getFxParameters, setFxParameter, setFxPreset | FX parameter control |
| getTrackFx, removeFx, toggleFxBypass | FX management |
| getTrackPeaks | Real-time peak meter data |
| undo, redo | Undo/redo history |
| renderProject, renderStems | Rendering/export |
| Key | Action |
|---|---|
| Space | Play / Pause toggle |
| Escape | Stop transport |
| R | Toggle recording |
| Home | Go to start |
| End | Go to end |
| ? | Show help overlay |
Shortcuts are active when the chat input is not focused.
With voice control enabled (Web Speech API), you can say:
- "Hit play" / "Stop" / "Pause" / "Hit record" / "Roll tape"
- "Go to start" / "From the top" / "Go to bar 16"
- "Go to chorus" / "Go to verse 2" / "Jump to marker Bridge"
- "Arm this track" / "Disarm" / "Solo" / "Mute"
- "Volume up" / "Louder" / "Volume 80%" / "Turn down"
- "Pan left" / "Pan right" / "Pan center" / "Pan 50% left"
- "Set me up to record vocals" / "Punch loop bars 8 to 16"
- "Preflight check" / "Rough mix" / "Comp takes"
- "Create a MIDI track" / "Add an instrument track with ReaSynth"
- "Chord progression with C, Am, F, G" / "Chords using C major and E minor"
- "Play a C major scale" / "Write A minor pentatonic scale in octave 3"
- "Write midi notes C4 E4 G4 B4"
- "Set threshold to -20 on the compressor" / "Show me the parameters"
- "Save template as My Session" / "Load template Vocal Session" / "List my templates"
- "What should I do next?" / "Now what?" / "What's the move?"
- "I always track dry" / "My genre is hip-hop" / "Remember that I punch in by region"
- "Undo" / "Redo"
SessionPilot analyzes your session on every state update and surfaces warnings in the sidebar before they derail your workflow:
| Warning | Severity | Condition |
|---|---|---|
| Recording but no tracks armed | Error | Transport rolling, no armed tracks |
| Monitoring off on armed track | Warning | Armed track has input monitoring disabled |
| Same input on multiple armed tracks | Warning | Two armed tracks share an input — double-recording |
| No input assigned on armed track | Info | Armed track has no input source set |
| FX-heavy chain on armed track | Warning | 4+ plugins on a tracking chain (latency risk) |
| Session has no tracks | Info | Empty session |
Warnings appear color-coded in the left sidebar and clear automatically when resolved.
At the start of each new session, SessionPilot asks two quick questions to brief itself on what you're working on:
- What are we working on today? — goal (new song, overdubs, a hook, etc.)
- What's the genre or vibe? — helps the advisor give relevant suggestions
Say skip to bypass any question. If your first message is already a command (e.g. "arm the vocal track"), intake is skipped automatically and you go straight to work.
Intake answers feed into the LLM context so advice and suggestions are tailored to your session from the first message.
Ask "what should I do next?" (or "now what?", "what's the move?") at any point and SessionPilot gives one concrete, context-aware suggestion:
- Recording in progress → encourages you to stay focused
- Tracks armed but not rolling → confirms you're ready and suggests preflight or record
- Takes exist but nothing armed → suggests comping or punching in
- Fresh session → recommends setting up a first track
- No song sections marked → suggests running
markSongStructure
When ANTHROPIC_API_KEY is set, the advisor uses Claude for richer, session-specific guidance. Otherwise a fast heuristic path runs locally.
SessionPilot remembers your working preferences across sessions in user_profile.json:
| Preference | Example phrase |
|---|---|
| Track style | "I always track dry" / "I prefer tracking wet" |
| Punch-in method | "I punch in by region" / "punch-ins by track" |
| Hook workflow | "My hook workflow is comp then tune" |
| Naming convention | "My naming convention is Artist_Title_Take" |
| Preferred octave | "preferred octave 3" |
| Genre | "I make hip-hop" / "my genre is R&B" |
| Freeform notes | "Remember that I always use a click with reverb" |
Preferences are included in the LLM context so suggestions and plans reflect your workflow automatically.
SessionPilot can write MIDI notes directly into REAPER from natural language. The music theory engine uses tonal to resolve chord names, scales, and note names into MIDI data.
"chord progression with C, Am, F, G"
"create a chord progression using C major, E minor, and G dominant 7"
"write chords Dm7, G7, Cmaj7 with half notes"
"chords using F sharp minor and B flat major in octave 3"
Supports major, minor, 7th, maj7, dim, aug, sus, and more. Natural language names like "C major", "E minor", "G dominant 7", "F sharp minor" are normalized automatically.
"play a C major scale"
"write A minor pentatonic scale in octave 3"
"create a D dorian scale"
"E flat blues scale"
Supports major, minor, pentatonic, blues, dorian, mixolydian, lydian, phrygian, locrian, harmonic minor, melodic minor, whole tone, and more.
"write midi notes C4 E4 G4 B4"
"insert midi notes D3 F#3 A3"
Notes are written sequentially as quarter notes. If no MIDI track is selected, one is created automatically.
- Octave: "in octave 3" (default: 4)
- Duration: "whole notes", "half notes", "quarter notes", "2 beats each"
- Velocity: "soft" (60), "loud" (120), "velocity 100"
Save and restore full session configurations (tracks, markers, regions):
"save template as My Vocal Session"
"list my templates"
"load template My Vocal Session"
Templates are stored as JSON files in the templates/ directory.
You can chain multiple commands with "then":
- "Arm track then hit record"
- "Go to bar 8 then hit play"
- "Stop then go to start"
MIT