You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Friday is not a chatbot. It is a fully autonomous desktop AI agent that lives on your Windows machine, watches your screen, listens to your voice, understands your goals, and takes action — [...]
Think of it as having a real-world version of Tony Stark's FRIDAY running on your PC. It monitors what you're doing, proactively comments and helps, controls your entire desktop via voice, manages[...]
What makes Friday different from other agents: she learns who you are. Import your conversation history from Claude, ChatGPT, or Gemini and Friday extracts your preferences, communication styl[...]
Friday is open source, Windows-native, self-hosted, and built to eventually compete with commercial agents like Devin, Cline, and Claude Code.
Feature Overview
🎙️ Voice & Audio
Feature
Status
Details
Gemini Live native audio
✅ Working
Primary voice interface via friday_live.py
Groq Whisper STT
✅ Working
Fallback speech-to-text
Porcupine wake word
✅ Working
"Hey Friday" detection via voice_wake.py
Multi-TTS engine support
🔧 In Progress
ElevenLabs, Google TTS, pyttsx3
Typing effect output
✅ Working
Terminal typing effect for responses
Audio crackling fix
✅ Fixed
Jitter buffer increased to 12 chunks (~2.4s), smoother drain, no discontinuity
👁️ Vision & Screen Awareness
Feature
Status
Details
Live API native video feed
✅ Working
Automatic ~1 FPS, 720p streaming — model sees screen without calling any tool
Active window detection
✅ Working
screen_watcher.py via pywinctl
Gemini Vision analysis
✅ Working
Screen-to-description pipeline
Vision-grounded clicking
✅ Working
vision_click() — finds elements by description
Background monitor with proactivity
✅ Working
Unsolicited observations via Gemini proactivity
Error detection on screen
🔧 In Progress
Detects visible Python/JS errors
Visual search
✅ Working
"Friday find the word X on screen"
🖥️ Desktop Automation & Control
Feature
Status
Details
Mouse & keyboard control
✅ Working
pyautogui-based via friday/tools.py
App launching & closing
✅ Working
open_app(), close_app() — system discovery chain
Spotify control
✅ Working (Web API + keyboard fallback)
Full Spotify API — search, play, queue, volume
Netflix/streaming control
🔧 In Progress
Vision-based navigation
File system access
✅ Working
With run_cmd + system access
Roblox launcher
✅ Working
Web search place ID → roblox:// URI
Microsoft Store launcher
✅ Working
ms-windows-store:// URI
Windows Clock (alarms, timers, stopwatch, reminders, focus)
The installer checks Python, installs dependencies (skips already-installed), creates .env, adds friday to PATH, and verifies all modules.
After install, edit .env with your API keys, then run:
friday
That's it. FRIDAY starts the dashboard, API, sidecar heartbeat, memory checks, and live voice engine with structured terminal output.
Usage
friday # Full supervisor: dashboard + voice engine
friday dashboard start # Start dashboard services (background)
friday dashboard stop # Stop dashboard services
friday dashboard status # Dashboard health + URLs + latency
friday dashboard open # Open dashboard in browser
friday status # System health: dashboard, sidecar, memory
friday doctor # Diagnostics: modules, ports, configs
friday doctor --verbose # Full diagnostic report
friday fix # Auto-repair common issues
friday memory-tree read people # Read from memory tree
friday sidecar status # Sidecar network status
Environment Variables
Variable
Required
Purpose
GOOGLE_API_KEY
Yes
Gemini Live (brain + vision)
PICOVOICE_ACCESS_KEY
Yes
Wake word detection
SPOTIFY_CLIENT_ID
Yes
Spotify control
SPOTIFY_CLIENT_SECRET
Yes
Spotify auth
GROQ_API_KEY
No
Fast inference
ANTHROPIC_API_KEY
No
Claude (coming)
OPENAI_API_KEY
No
GPT (coming)
Create .env from .env.example:
copy .env.example .env
Add to Windows Startup
FRIDAY can auto-start with Windows:
python -m friday.cli config
# Or ask: "Friday, add yourself to Windows startup"
Voice Commands
Friday responds to natural language. No rigid syntax required.
Screen & Vision
"Friday, what do you see on my screen?"
"Friday, is there an error on my screen?"
"Friday, find the word msconfig on my screen"
"Friday, click the play button"
Apps & Desktop
"Friday, open Spotify"
"Friday, play Naruto OST on Spotify"
"Friday, open VS Code"
"Friday, close Chrome"
"Friday, play Bloxfruits on Roblox"
Browser & History
"Friday, open the Jarvis repo I was loOKing at — the one by vierisid"
"Friday, what was I reading about LangGraph earlier?"
"Friday, open my IITM course"
Messaging & Communication
"Friday, message Mangesh on Instagram — tell him I'll be back soon"
"Friday, send an email to dev@example.com with a job offer letter from NexSemble"
"Friday, read my latest emails"
Goals & Productivity
"Friday, I have an IITM 8-week course ending May 31st at iitm.course.url — track it"
"Friday, what are my goals today?"
"Friday, how long have I been on YouTube today?"
"Friday, what's my screen time?"
Smart Home
"Friday, tell Alexa to switch off the lights"
"Friday, switch off my bedroom lights"
LLM & System
"Friday, switch to Claude"
"Friday, what LLM are you using?"
"Friday, do a deep research on NVIDIA's new GPU architecture and generate a report"
"Friday, generate a Python Flask server file"
Memory & Learning
"Friday, I've imported my Claude chat history — learn from it"
"Friday, what do you know about me?"
"Friday, update your understanding of my preferences"
"Friday, forget what I told you about X"
Module Reference
File
Purpose
friday.py
Root entry point. Gemini Live session launch with 5x auto-restart
friday/live.py
Main engine. 150+ tool declarations, TOOL_MAP, audio playback, system prompt, Phase 14-16 integration
CV context wired into situational_awareness() — LLM sees camera without user knowing
Native Windows app (PyQt6 or Tauri)
Settings panel with key vault
Plugin system
Packaged installer (.exe)
Contributing
Friday is an open source project built solely by Arnav Phulari, shared publicly through NexSemble — a peer-learning and collaborative tech community in Pune, India[...]
Pull requests are welcome. For major changes, open an issue first.
# Fork and clone
git clone https://github.com/your-username/friday.git
# Create feature branch
git checkout -b feature/your-feature-name
# Make changes, then push
git push origin feature/your-feature-name
# Open a Pull Request on GitHub
Guidelines
Keep Windows-native compatibility (no Docker, no mandatory WSL2)
Never hardcode localhost — use OLLAMA_BASE_URL from env
Every new tool must be registered in friday_live.py's execute_tool()
Test on Windows 10 and 11
Known Issues
Issue
Status
Workaround
Gemini GOAWAY timeout on long idle
Fixed
keepalive_task() pings every 45s
Gemini 1008 GoAway on long tool sequences
Mitigated
Tool-ceiling removed; user accepts risk
Chrome SQLite locked when browser open
Fixed
Copy to temp before reading
pyautogui coordinates break on resolution change
In Progress
Use vision_click() instead
OpenCLI browser open <url> hangs on Windows
Fixed
Now opens via webbrowser + bind instead
Spotify "No active device" error
In Progress
Open Spotify app first, then retry
See_screen Gemini API 429 (rate limit)
Workaround
Falls back to Gemini 1.5 Flash automatically
SDK bug: send_realtime_input raises ValueError with >1 arg
Friday is solely built by Arnav (@hackers-reality)
Co-leader of NexSemble — a peer-learning and collaborative tech community based in Pune, Maharashtra, India.
"This is not a chatbot . This is Friday — the real one."
Community — Join NexSemble
Friday is built in public and shared through NexSemble — a community of builders, hackers, and learners who ship real projects together. Come talk Friday, AI agents, and whatever you're bui[...]
