Streamfog MCP

📖 Installation Guide — quick start, manual setup, and troubleshooting 📖 Docs — Configuration · Tools · Troubleshooting · Development 🤖 llms-full.txt — full LLM-readable documentation

AI-driven AR lens orchestrator for live OBS streams. Control Streamfog face filters, AR effects, and Vtuber avatars through MCP tools via the local Streamer.bot WebSocket bridge. Your AI assistant becomes a stream producer.

What is Streamfog?

Streamfog is a Windows desktop app (https://streamfog.com) that streamers use to put AR effects on their face during live broadcasts — face filters ("beauty smooth", "cyber helmet"), animated lenses, background effects, and Vtuber-style avatars. It takes your webcam feed, applies the effect in real time, and renders the result into OBS Studio through a browser source.

Two things matter for automation:

1. Streamfog has no public API, CLI, or plugin SDK. You cannot script it directly, and it cannot tell you which lens is active. It is effectively a black box that only listens to Streamer.bot.
2. Streamer.bot is the remote control. Streamer.bot is a free automation tool for streamers. Streamfog registers itself with Streamer.bot as an integration, and Streamer.bot exposes a local WebSocket server that accepts DoAction commands by name (e.g. fire the action SetLens_BeautySmooth).

This project is the bridge between the two: it gives an AI assistant (or the dashboard, or any script) a clean tool interface — streamfog_set_lens, streamfog_clear_effects, streamfog_toggle_avatar, streamfog_status, streamfog_list_lenses — and translates each call into a Streamer.bot DoAction dispatch. Without this server you would have to open Streamer.bot and click actions manually; with it, your agent becomes a stream producer.

⚠️ Fire-and-forget: Streamer.bot does not report whether an action succeeded, and Streamfog cannot be queried. A tool returning "success" means dispatched, not applied — confirm visually on the stream.

You might use this if…	You want your AI to switch AR lenses, toggle Vtuber avatars, or clear effects during live OBS broadcasts — controlled by Twitch chat events, channel points, or agentic automation.
What it connects to	Streamfog desktop app → Streamer.bot WebSocket → this MCP server
Ports	Backend 10994, Dashboard 10995
Start	just bootstrap then start.ps1

How to use this server (3 steps)

1. Install the two external apps (one-time): Streamfog + Streamer.bot, enable Streamfog's integration inside Streamer.bot, and enable Streamer.bot's WebSocket server on port 8080. Full walkthrough: docs/ONBOARDING.md.
2. Map your lenses: create lenses.json linking human-readable lens names to the exact Streamer.bot action names that apply them (see Lens Map).
3. Start and drive it: start.ps1 boots the backend + dashboard; then either connect your MCP client (Claude Desktop / Cursor / opencode) over stdio or http://127.0.0.1:10994/mcp, click lenses on the dashboard, or call the REST API from scripts. All three surface the same bridge.

Architecture

Plain English: your AI talks to this server, this server talks to Streamer.bot, Streamer.bot tells Streamfog what to render, and OBS shows the result. Every hop is local.

┌─────────────┐     MCP            ┌──────────────────┐     WebSocket      ┌──────────────┐
│  LLM Agent  │ ───────────────→  │  streamfog-mcp   │ ────────────────→ │ Streamer.bot  │
│  (Claude,   │ ←───────────────  │  :10994 (FastMCP) │ ←──────────────── │ :8080         │
│   Gemini)   │   stdio / /mcp    │  :10995 (React)   │   DoAction JSON   │               │
└─────────────┘                  └──────────────────┘                    └──────┬────────┘
                                                                                │ Native Hook
                                                                         ┌──────▼────────┐
                                                                         │  Streamfog    │
                                                                         │  Desktop App  │
                                                                         └──────┬────────┘
                                                                                │ Browser Source
                                                                         ┌──────▼────────┐
                                                                         │  OBS Studio   │
                                                                         └───────────────┘

• MCP tools (for AI agents): served over stdio or the streamable HTTP endpoint at http://127.0.0.1:10994/mcp.
• REST API (for scripts + dashboard): /api/* on the same port, backed by the same bridge instance.
• Dashboard (for humans): React SPA at :10995 — lens grid, status KPIs, chat, logs.

The MCP endpoint is served at http://127.0.0.1:10994/mcp (streamable HTTP) in dual mode, alongside the REST API and dashboard.

Quick Start

uv sync
# Edit lenses.json with your Streamer.bot action names
# Set STREAMFOG_MCP_STREAMERBOT_TOKEN in .env if using auth
.\start.ps1

MCP-only via stdio (for Cursor, Claude Desktop):

uv run -m streamfog_mcp --stdio

Prerequisites

1. Streamfog installed and running
2. Streamer.bot installed and running
3. Streamfog → Streamer.bot integration enabled in Streamfog's Integrations panel
4. Streamer.bot WebSocket server enabled (Settings → WebSocket Server)
5. Actions created in Streamer.bot (e.g. SetLens_BeautySmooth, ClearEffects, ToggleAvatar)
6. lenses.json populated with your action→lens mappings

See docs/ONBOARDING.md for the step-by-step walkthrough.

Configuration

Variable	Default	Description
STREAMFOG_MCP_STREAMERBOT_HOST	127.0.0.1	Streamer.bot WebSocket host
STREAMFOG_MCP_STREAMERBOT_PORT	8080	Streamer.bot WebSocket port
STREAMFOG_MCP_STREAMERBOT_TOKEN	—	Streamer.bot auth token
STREAMFOG_MCP_LENS_MAP_PATH	lenses.json	Path to lens→action mapping file
STREAMFOG_MCP_PORT	10994	Backend port
STREAMFOG_MCP_LOG_LEVEL	INFO	Logging level

Lens Map (lenses.json)

{
  "beauty_smooth": "SetLens_BeautySmooth",
  "cyber_helmet": "SetLens_CyberHelmet",
  "vtuber_avatar": "SetLens_VTuberAvatar"
}

Keys are human-readable lens identifiers used in MCP tool calls. Values are the corresponding Streamer.bot action names. Keys starting with _ are ignored; unknown identifiers fall back to the action SetLens_{identifier}.

MCP Tools (5)

Lens Control

Tool	Description
streamfog_set_lens	Activate a specific AR lens or face filter
streamfog_clear_effects	Strip all effects, return camera to baseline
streamfog_toggle_avatar	Toggle Vtuber-style avatar on/off

Discovery — READ_ONLY

Tool	Description
streamfog_list_lenses	List all configured lenses from lenses.json
streamfog_status	Bridge connection health + lens count

REST API

Endpoint	Method	Description
/api/health	GET	Liveness probe (status, version, uptime, tool count, bridge)
/api/v1/status	GET	Server + bridge health
/api/v1/lenses	GET	List all lenses
/api/v1/lenses/set	POST	Activate a lens ({"lens_identifier": "beauty_smooth"})
/api/v1/lenses/reload	POST	Reload lens map from disk
/api/v1/effects/clear	POST	Clear all effects
/api/v1/avatar/toggle	POST	Toggle avatar
/api/tools	GET	Dynamic MCP tool list with schemas
/api/capabilities	GET	Capability discovery
/api/skills	GET	Skill list
/api/skills/{name}	GET	Skill content (markdown)
/api/llm/discover	GET	Local LLM provider probe (Ollama/LM Studio/vLLM) + GPU
/api/logs	GET	Ring-buffer log window
/api/v1/diagnostics	GET	Tool list, system info, errors

Web Dashboard

Dark SOTA dashboard at :10995 (React + Vite + Tailwind + Zustand):

• Dashboard — hero + KPI cards (server, tools, bridge, uptime), lens grid, quick actions, exponential-backoff health
• Tools — dynamic tool list with schemas from /api/tools
• Chat — skill-first local LLM chat (Ollama/LM Studio/vLLM), 4 personalities, localStorage history, export
• Settings — backend health + LLM provider detection
• Help — architecture, env reference, troubleshooting
• Logs — live ring-buffer modal (Ctrl+L)

Keyboard: Ctrl+Scroll zoom, Ctrl+0 reset, Ctrl+L logs.

Project Structure

streamfog-mcp/
├── src/streamfog_mcp/
│   ├── _mcp.py              FastMCP singleton + resources
│   ├── server.py            FastAPI gateway (REST + /mcp mount)
│   ├── __main__.py          CLI entry (--stdio / --serve)
│   ├── config.py            Pydantic settings (STREAMFOG_MCP_ prefix)
│   ├── errors.py            _error_response() with logger.exception
│   ├── logging_ring.py      Ring buffer for /api/logs
│   ├── skills/streamfog/    SKILL.md (chat preprompt)
│   ├── tools/core_tools.py  5 @mcp.tool() decorators
│   └── services/streamerbot.py  Streamer.bot WebSocket client
├── webapp/                  Vite + React 19 + Tailwind + Zustand
├── native/                  Tauri 2.0 desktop wrapper
├── lenses.json              Lens → action mapping
├── pyproject.toml / uv.lock
├── start.ps1 / start.bat
├── justfile
└── tests/                   13 tests (units + endpoint + e2e)

Verification

just lint          # ruff
just test          # pytest (coverage-gated)
uv run pyright src/  # types
cd webapp && bun run check && bun run biome:ci
cd webapp && bunx playwright test   # e2e

Known Limitations

• Streamfog does not expose a native CLI or local API — all control goes through Streamer.bot
• Lens activation is fire-and-forget (Streamer.bot does not report success/failure for actions)
• No lens preview or thumbnail retrieval (Streamfog desktop is a black box)
• Lumia/Crowd Control bridge path is documented but not yet implemented as an alternative transport