Agents With Remote Control Mobile Controller

Local-first agent orchestration: control CLI coding agents (Codex, Claude Code, Gemini, OpenCode, DeepSeek, Zen, and other adapter-backed CLIs) from your phone, with approval gates, Git worktree isolation, durable reconnect/replay, human-approved GitHub/Linear sync, Tailscale remote access, and completed Phase 5 controlled MCP / Notion / Figma synchronization.

Status: Phases 1, 2, 3, 3.5, 4, 4.5, and 5 are complete. Phase 6 is active: productization foundation, auth/settings, provider visibility, multi-agent review workflows, and advanced automation on top of the completed controlled-sync foundation.

TL;DR

Run AI coding agents on your PC. Control them from your phone. The agent works in an isolated Git worktree. Risky actions are surfaced as approval cards. Phase 3 made local coding work safe enough to review; Phase 3.5 made long-lived mobile sessions durable enough to reconnect, replay missed events, checkpoint, and restore dormant sessions.

Phase 4 connects that local loop to GitHub and Linear while preserving the human approval gate. Phase 4.5 established Tailscale as the default private remote-access path for phone approvals outside the home LAN. Phase 5 completed Notion strategy-doc sync, Figma/FigJam link metadata, controlled MCP registry/transport/permission/audit layers, controller status surfaces, and the docs/test/PR contract needed to keep agents constrained across context compaction.

Phase 6 starts with a productization prelude: dedicated Notion/Figma provider status UI, auth/tenancy/settings architecture, user/workspace persistence, session-bound WebSocket auth, and settings UI. After that, it expands into multi-agent review, structured critique, conflict surfacing, scheduled tasks, and push notifications.

Current implementation scope

Phase 1 — Local orchestrator

Root-level NestJS REST API.
Prisma + SQLite persistence for Task, AgentSession, and AgentLog.
CodexAdapter that launches codex exec --json --cd <repoPath> - through node-pty and resumes idle turns with codex exec resume --json <thread_id> -.
Phase 6.1 adapter registry support for codex, claude, gemini, and opencode behind the same task/session/log/replay/approval model.
REST endpoints: POST /tasks, GET /tasks, GET /tasks/:id, POST /tasks/:id/stop.
Real Codex PTY smoke path and token-gated integration e2e tier.

Phase 2 — WebSocket gateway + controller UI

Socket.IO WebSocket gateway with CONTROLLER_SECRET bearer auth.
Per-task rooms (task:<id>) and live task/log events.
POST /tasks/:id/input for follow-up instructions.
Next.js App Router controller UI in controller/.
Dashboard, New Task, and Task Detail pages.
Sequence-based log deduplication.

Phase 3 — Local-loop hardening

Per-task Git worktree provisioning.
Task metadata for worktreePath, branchName, baseRef, baseCommit, and approval mode.
Cooperative ARC_ACTION_REQUEST / ARC_APPROVAL protocol.
ApprovalRequest, AuditLog, GitChangeSummary, and TestRunSummary persistence.
Controller cards for pending approvals, diff summaries, and test status.
Three-tier safety model: SAFE, NEEDS_APPROVAL, BLOCKED.

Phase 3.5 — Continuous Agent stabilization

Durable task-scoped event ledger with replay cursors.
Mobile reconnect/resubscribe behavior.
Checkpoint and dormant-session lifecycle.
Explicit distinction between live PTY processes, idle Codex thread resume, and reconstructed DB view.
Feature/provider seams prepared for GitHub, Linear, Notion, Figma, and MCP.
Dependency/package hygiene and docs linting improvements.

Phase 4 — GitHub + Linear sync

GitHub issue and Linear issue linking at task creation.
Provider adapter seams for GitHub and Linear.
SyncEvent idempotency model.
Branch/worktree lifecycle tied to external issue metadata.
Approval-gated commit, push, and draft PR creation.
Linear ↔ GitHub cross-reference sync.
PR merge detection and Linear completion sync.
Mobile sync UI, provider error surfaces, and token-gated provider e2e tests.

Phase 4.5 — Tailscale remote-access baseline

Tailscale remote access baseline for daily phone use outside the home LAN.
ARC_HOST=0.0.0.0 and ARC_ALLOW_PUBLIC_BIND=true only behind a trusted private overlay.
Controller remote config for browser WebSocket access and server-side REST proxying.
Windows/WSL2 networking notes for direct access, mirrored networking, or scoped port proxy.
Mobile smoke coverage for task list, task detail, replay, and approval cards.

See docs/remote-access.md for the default daily remote-access setup.

Phase 5 — Complete controlled sync expansion

MCP registry schema and config loader with declared server/tool capability ceilings.
MCP transport abstractions for stdio, Streamable HTTP, and legacy SSE compatibility.
Permission ladder and non-escalation rules for read-only, append-only, write, and admin-blocked behavior.
Mobile approval routing for write-capable MCP calls.
MCP audit log with sanitized previews plus argument/result hashes.
Notion strategy doc sync through append-only session summaries.
Figma/FigJam URL parsing, read-only metadata, and task/issue link attachments.
Controller surfaces for registry state, pending MCP approvals, audit events, and provider sync status.
Repo-local official source snapshot, test matrix, and PR contract for implementation-agent token efficiency.

Phase 5 implementation docs:

Phase 5 provider config uses placeholder-safe env names:

ARC_NOTION_TOKEN=""
ARC_NOTION_PROJECT_PAGE_ID=""
ARC_NOTION_SYNC_SECTION_BLOCK_ID=""
ARC_FIGMA_TOKEN=""
ARC_FIGMA_ALLOWED_FILE_KEYS=""

Phase 6 — Active: productization + multi-agent review

Phase 6.0: provider status UI, auth/tenancy ADR, user/workspace/settings persistence, first-admin bootstrap, session-bound WebSocket auth, settings UI, and auth/security docs.
Phase 6.1: Adapter parity (Claude, Gemini, OpenCode) lands first; role-aware scheduler, reviewer prompts, structured critique, and diff/conflict surfacing follow in separate tickets.
Phase 6.2: scheduled task queue, web push notifications, and future PR lifecycle intelligence.

Phase 6 implementation docs:

Still deferred:

Auto-merge, auto-deploy, or unattended production actions.
Figma writes, Notion page replacement/deletion, destructive/admin MCP tools.
Cloud deployment as a required baseline.

Why this exists

CLI coding agents are powerful but tethered to terminal sessions and keyboard presence. The moment you walk away — gym, errands, dinner — the agent stalls on the next approval prompt.

This project gives agents a remote command surface so the human-in-the-loop part can happen from your phone, while keeping the safety model strict by default.

It is not a mobile IDE. It is not a VS Code chat extension. It is a thin orchestrator + mobile/web controller around existing CLI agents.

Desired UX

Start or monitor an AI coding task from your phone.
The local orchestrator runs a CLI agent in WSL2.
The agent works inside an isolated repo worktree.
When the agent needs input, approval, or review, it pings your phone.
Reply with free text, structured actions, or approve/deny tool use.
Inspect diff summaries, run configured local tests, and restore/replay long-lived sessions.
Link a task to GitHub/Linear, approve commit/push/PR actions, and sync project state without leaving the phone.
Use Tailscale so approvals keep working from cellular or non-home WiFi.
Review controlled Notion/Figma/MCP sync actions with explicit context and auditability.
In Phase 6, coordinate authenticated, customizable, multi-agent review workflows on top of the same approval and audit spine.

Architecture (high level)

flowchart LR
  subgraph Mobile["Phone / Web Controller"]
    UI["Controller UI"]
    Auth["Auth + Settings"]
  end
  subgraph Host["Local Host (Windows + WSL2)"]
    Orch["Local Orchestrator (NestJS)"]
    WSL["WSL2 Runtime<br/>Codex · Claude · Gemini · OpenCode"]
    WT["Repo Worktrees"]
    DB[("SQLite MVP DB")]
  end
  subgraph Sync["Phase 4 Sync"]
    GH["GitHub"]
    LN["Linear"]
  end
  subgraph Controlled["Phase 5 Controlled Sync"]
    NT["Notion append-only summaries"]
    FG["Figma/FigJam read-only links"]
    MCP["MCP registry + permissions + audit"]
  end
  subgraph Review["Phase 6"]
    MA["Multi-agent review"]
    SCH["Scheduled tasks + notifications"]
  end
  UI <-->|"WebSocket"| Orch
  UI -->|"REST"| Orch
  Auth --> UI
  Orch --> WSL
  WSL --> WT
  Orch --> DB
  Orch --> GH
  Orch --> LN
  Orch --> NT
  Orch --> FG
  Orch --> MCP
  Orch --> MA
  Orch --> SCH

Full architecture, lifecycle, approval-gate state machine, ERD, and alternatives considered: docs/diagrams.md and docs/ARCHITECTURE.md. FigJam companion diagrams are mirrored in docs/figma-companion-diagrams.md.

Phased plan

Phase	Focus	Linear	GitHub
1	Local orchestrator + single-agent CLI runner	TSH-77	#2
2	Mobile/web controller + live session UI	TSH-78	#3
3	Worktree isolation + approval gates + diffs + tests	TSH-79	#4
3.5	Continuous-agent stabilization, reconnect, checkpointing, package hygiene	TSH-83	PRs #18, #21, #22, #23
4	GitHub + Linear sync (issue → branch → commit → PR)	TSH-80	#5
4.5	Tailscale private remote-access baseline	TSH-111	#43
5	Notion + Figma + controlled MCP expansion	TSH-81	#6
6	Productization + multi-agent review + advanced automation	TSH-82	#7

Phase 5 completed ticket map

Linear	Focus
TSH-112	Complete: MCP registry schema and config loader
TSH-113	Complete: MCP transport abstractions for stdio, Streamable HTTP, and legacy SSE
TSH-114	Complete: MCP permission ladder and non-escalation rules
TSH-115	Complete: mobile approval cards for write-capable MCP calls
TSH-116	Complete: MCP audit log model with argument/result hashing
TSH-117	Complete: Notion adapter for project-doc reads and append-only session summaries
TSH-118	Complete: Figma/FigJam adapter for read-only metadata and task link attachments
TSH-119	Complete: controller surfaces for MCP registry, tool-call audit, and provider sync status
TSH-120	Complete: Phase 5 provider/MCP test matrix, docs bundle, and PR contract

Phase 6 planned ticket map

Linear	Focus
TSH-121	Provider status UI for Notion/Figma
TSH-122	Auth, tenancy, and productization ADR
TSH-123	User/workspace/settings persistence
TSH-124	Auth foundation and first-admin bootstrap
TSH-125	Session-bound WebSocket auth
TSH-126	Settings UI
TSH-127	Auth/security docs and PR contract updates
TSH-128	Adapter parity and configurable agent profiles
TSH-129	Role-aware task scheduler
TSH-130	Reviewer prompts and structured critique
TSH-131	Diff comparison and conflict surfacing
TSH-132	Scheduled task queue
TSH-133	Web push notifications

Tech stack

Backend (orchestrator)

Node.js + TypeScript
NestJS
Prisma + SQLite (MVP)
REST endpoints for one-shot commands
Socket.IO for live updates
node-pty for wrapping CLI agents
Git worktree operations and cooperative approval gates
Provider seams for GitHub/Linear in Phase 4
Controlled MCP, Notion, and Figma/FigJam seams in Phase 5

Frontend (controller)

Next.js App Router, mobile-first, runs on port 3001
Tailwind CSS
socket.io-client
virtualized log rendering
local/private-overlay auth via CONTROLLER_SECRET, to be replaced or explicitly scoped by Phase 6 auth work

Runtime

Windows host
WSL2 for agent execution
Git worktrees for task isolation
Tailscale private overlay for default daily phone access outside the home LAN

Safety model

Three-tier classification on every requested action:

Tier	Examples	Behavior
SAFE	Read repo, inspect git, run tests, summarize, plan, declared MCP read-only calls	Auto-allow, log only
NEEDS APPROVAL	Edit files, install, migrate, branch, commit, push, open PR, external sync, MCP append/write tools	Ping phone, wait for human
BLOCKED BY DEFAULT	Read `.env`/secrets, force push, prod deploy, modify auth, exfiltrate repo, destructive/admin MCP tools, modify global system config, run unknown shell scripts	Refuse outright, log event

Every approval and denial is recorded in an audit log. Full taxonomy and rationale: docs/SAFETY.md.

Getting started

Prerequisites:

Node.js 22+
pnpm
Local Codex CLI authentication already configured
A Linux-side repo path for ARC_REPO_PATH

Install dependencies and generate Prisma:

pnpm install
pnpm prisma:generate

Create local config and initialize SQLite:

cp .env.example .env
pnpm prisma:migrate

Run the orchestrator:

pnpm start:dev

In a separate terminal, run the controller UI:

cd controller
pnpm install
pnpm dev          # http://localhost:3001

The controller proxies REST calls through its Next.js server so CONTROLLER_SECRET can stay server-side for HTTP actions. WebSocket auth still uses NEXT_PUBLIC_CONTROLLER_SECRET because the browser connects directly to the orchestrator socket. Phase 6 is expected to replace production dependence on this browser-exposed secret with session-bound WebSocket authorization.

Accessing from your phone

Same network: set ARC_HOST=0.0.0.0 and ARC_ALLOW_PUBLIC_BIND=true in .env, update controller/.env.local with your LAN IP, and open http://<LAN-IP>:3001 on your phone.

Outside your network: use the Tailscale setup in docs/remote-access.md. This is the default daily setup for remote approvals and controlled external sync.

Phase PR gates

Every implementation PR should include exact commands run and results:

pnpm install
pnpm audit --audit-level=low
pnpm typecheck
pnpm test
pnpm test:e2e
pnpm lint:md
pnpm --filter controller typecheck
pnpm --filter controller build

See .github/PULL_REQUEST_TEMPLATE.md for the ongoing PR contract.

Project links

GitHub: https://github.com/mjshuff23/agents-with-remote-control-mobile-controller
Linear project: https://linear.app/michaelshuff/project/agents-with-remote-control-mobile-controller-181d4f51202c
Notion strategy doc: https://www.notion.so/35bc2ea5f18f8186b134efa7759a19e6
Phase 4 handoff: docs/phase-4-implementation.md
Remote access: docs/remote-access.md
Phase 5 blueprint: docs/phase-5-implementation.md
Controlled MCP sync: docs/mcp-controlled-sync.md
Phase 5 official source snapshot: docs/phase-5-official-sources.md
Phase 6 blueprint: docs/phase-6-implementation.md
Auth/productization ADR: docs/auth-productization-adr.md
Auth/session security: docs/security-auth-session.md
Provider UI status: docs/provider-ui-status.md
Phase 6 official source snapshot: docs/phase-6-official-sources.md
PR contract: .github/PULL_REQUEST_TEMPLATE.md

License

Apache 2.0

Name		Name	Last commit message	Last commit date
Latest commit History 79 Commits
.github		.github
.husky		.husky
__mocks__		__mocks__
controller		controller
docs		docs
prisma		prisma
scripts		scripts
src		src
test		test
.env.example		.env.example
.gitignore		.gitignore
.markdownlint.json		.markdownlint.json
.markdownlintrc		.markdownlintrc
.nvmrc		.nvmrc
AGENTS.md		AGENTS.md
LICENSE		LICENSE
PLAN.md		PLAN.md
README.md		README.md
arc.config.json		arc.config.json
arc.mcp.example.json		arc.mcp.example.json
nest-cli.json		nest-cli.json
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
prisma.config.ts		prisma.config.ts
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Agents With Remote Control Mobile Controller

TL;DR

Current implementation scope

Why this exists

Desired UX

Architecture (high level)

Phased plan

Phase 5 completed ticket map

Phase 6 planned ticket map

Tech stack

Safety model

Getting started

Accessing from your phone

Phase PR gates

Project links

License

About

Uh oh!

Releases 5

Packages

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Agents With Remote Control Mobile Controller

TL;DR

Current implementation scope

Why this exists

Desired UX

Architecture (high level)

Phased plan

Phase 5 completed ticket map

Phase 6 planned ticket map

Tech stack

Safety model

Getting started

Accessing from your phone

Phase PR gates

Project links

License

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 5

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages