A real-time voice AI agent with a distinctive web front-end. Speak to it in the browser and it answers back, across six domains: astrology, Bollywood, cricket, devotion, entertainment, and finance.
- Voice pipeline —
livekit-agents ~1.5: speech-to-text → LLM → text-to-speech, with local VAD and turn detection. - LLM — a Databricks Foundation Model API endpoint over the AI Gateway Responses API.
- STT / TTS — the Sarvam plugin (Indian-language voice stack).
- Tools — six Lakebase-backed hot-path tools (latest-N + BM25 search) over seeded demo content.
- Tracing — every session exports OpenTelemetry spans to Databricks managed MLflow, stored in Unity Catalog Delta tables.
- Front-end — Rely IQ, a self-contained page (
web/) that connects the browser directly to the agent over WebRTC.
LiveKit Cloud is the only meeting point: the browser sends audio to it over WebRTC, and the agent receives jobs from it over an outbound WebSocket. The web server only serves the page and mints access tokens; media never touches it.
| Path | What |
|---|---|
src/agent.py |
The agent: voice pipeline, dispatch, UC tracing |
src/hotpath_tools.py |
Six Lakebase-backed LiveKit tools |
infra/ |
Lakebase schema setup + simulated demo content |
web/ |
Rely IQ front-end (page, token server, build) |
scripts/ |
Standalone health / trace checks |
internals/life-of-a-call.svg |
Animated sequence diagram of a call |
ARCHITECTURE.md |
Full architecture + deployment notes |
CLAUDE.md |
Repo guide (also read by Claude Code) |
Requires Python 3.12 and uv. A LiveKit Cloud
project, a Sarvam API key, and a Databricks workspace (with a Lakebase project
for the tools) are needed for a full run.
cp .env.example .env.local # then fill in real values
uv sync
uv run python src/agent.py download-files # VAD + turn-detector models
uv run python src/agent.py console # talk in your terminal (no LiveKit needed)Connect it to LiveKit Cloud and hot-reload while developing:
uv run python src/agent.py devcd web
node build.mjs # src/index.template.html + fonts → public/index.html
node server.mjs # http://localhost:8787Local dev falls back to the repo's .env.local for LiveKit credentials. See
web/README.md for deploying to a Node host, and ARCHITECTURE.md for the
single-container Databricks Apps deployment (frontend + agent worker together).
All config comes from .env.local at the project root (git-ignored); see
.env.example for the full list. Workspace-specific values (host, catalog, IDs,
app name) live in WORKSPACE.md, git-ignored — copy it from
WORKSPACE.template.md. Never commit real secrets.