Aven — Talk to your computer

A native macOS voice assistant for Apple Silicon. It opens a live WebSocket to OpenAI's GPT-Realtime 2, captures your microphone while you hold a key, and lets the model control your Mac through tool calls. Every tool call is printed to the terminal as it fires. Ships with one tool — open_app — and is built to make adding more trivial.

How it works

Mic (AVAudioEngine) ──24kHz PCM16──▶ RealtimeClient ──WebSocket──▶ GPT-Realtime 2
                                          │                              │
   NotchView (SwiftUI overlay) ◀── AppState ◀──── function_call ────────┘
                                          │
                                     ToolRegistry ── runs open_app, returns result

• Push-to-talk: hold ⌥ Option to talk; release to send. The mic only streams while the key is held — nothing leaves your Mac otherwise.
• Silent: the model replies with actions, not speech. You get a notch animation and a terminal log.
• Notch overlay: a small pill pinned to the top-center of the screen that changes color/animation per state (idle, listening, thinking, tool-firing, error).

Setup

1. Put your key in .env.local:

   OPENAI_API_KEY=sk-...
   

2. Build the app bundle:

   ./build.sh
   

3. Run it:

   ./Aven.app/Contents/MacOS/Aven     # logs tool calls in this terminal
   

   or open ./Aven.app to run it as a background agent.

On first launch a one-time setup window walks you through the three permissions Aven needs — Microphone, Screen Recording, and Accessibility — then never asks again. You can reopen it anytime from the menu bar (Permissions & Setup…). The global ⌥ push-to-talk hotkey watches only modifier keys, so it needs no Input Monitoring grant.

Say something like "open Safari" — you'll see:

-> open_app({"name":"Safari"})

Adding a tool

Open Sources/Aven/ToolRegistry.swift and add one entry in init():

t["set_volume"] = Tool(
    schema: [
        "type": "function",
        "name": "set_volume",
        "description": "Set the system output volume from 0 to 100.",
        "parameters": [
            "type": "object",
            "properties": ["level": ["type": "number", "description": "0-100"]],
            "required": ["level"]
        ]
    ],
    handler: { args in
        let level = (args["level"] as? NSNumber)?.intValue ?? 50
        // ... run osascript / API ...
        return #"{"ok": true}"#
    }
)

Dispatch, logging, the notch flash, and the result round-trip are all generic — you only write the schema and the closure.

Files

File	Role
RealtimeClient.swift	WebSocket lifecycle + GPT-Realtime 2 event protocol
AudioEngine.swift	Mic capture, resample to 24kHz mono PCM16, base64 chunks
ToolRegistry.swift	The tools (add yours here)
AppState.swift	Wires socket + audio + tools + UI together
NotchView.swift	The SwiftUI notch overlay
main.swift	App entry, notch window, push-to-talk hotkey, .env.local loading