[Phase 2] Local Flow processor (Wispr-style polish + structured output) #2

Summary

Add a local LLM "Flow" processor that takes raw transcript output and produces polished Markdown — fixing punctuation, removing filler, structuring as summary + action items, etc. Wispr Flow-inspired.

Depends loosely on #1 (local Whisper) but doesn't strictly require it — Flow can polish Deepgram output too. Phase 2 of the offline-mode roadmap.

Why

Raw transcripts are unpolished (uh, um, repetitions, broken syntax, run-on sentences). The current AnthropicAI.summarize() and actionItems() use Claude over the wire. A local equivalent:

• Removes the Anthropic dependency for users who want to be 100% local
• Faster for short content (no API round-trip)
• Privacy: full pipeline runs on-device
• Lets users with good local hardware (DGX, Mac Studio M3 Max+) use their own gear

Approach

Pluggable processor protocol

protocol FlowProcessor {
    func cleanup(_ paragraphs: [Paragraph]) async throws -> String
    func summarize(_ paragraphs: [Paragraph]) async throws -> SummaryResult
    func extractActionItems(_ paragraphs: [Paragraph]) async throws -> ActionItemsResult
    func fullReport(_ paragraphs: [Paragraph]) async throws -> FlowReport
}

Two implementations:

• AnthropicFlowProcessor (refactor of current AnthropicAI)
• LocalFlowProcessor (new)

Local backend options (pick at compile time AND runtime)

• Ollama — already supported via the "Custom Endpoint" setting. Just point at http://localhost:11434/v1.
• MLX-LM direct integration — smaller binary, no Ollama dependency. Uses mlx-swift-examples.

UI changes (Detail view)

• New "Polished" tab between Transcript and Summary
• Side-by-side: raw transcript on left, polished markdown on right
• "Re-run flow" button per tab to regenerate with different model
• Status indicator: which provider was used (cloud Claude vs local MLX vs local Ollama)

Recommended models for Flow on Apple Silicon

Structured output

Flow produces a single JSON document with:

• cleaned_transcript: string (markdown)
• summary: { overview: string, key_topics: string[], decisions: string[], notable_quotes: [{speaker, text}] }
• action_items: { team: [...], by_person: {...} }
• key_decisions: [...]

This replaces the current two-call (summarize + actions) flow with a single richer pass.

Acceptance Criteria

• FlowProcessor protocol; existing AnthropicAI conforms
• LocalFlowProcessor implementation (Ollama route + MLX-LM route)
• "Polished" tab in Detail view
• Side-by-side diff/preview of raw vs polished
• Settings: choose Flow provider (Cloud / Local Ollama / Local MLX) per-action
• System prompts: cleanup, summarize, action-items, full report (see [Feature] Local Wispr-Flow prompt templates + editing UI #5 for prompt templates)
• Falls back gracefully if local LLM unavailable (timeout, OOM, missing model)
• Performance: < 30 s for a 10-min transcript on M2 with Phi-3.5

Out of scope

• Streaming token output (batch-only for v1)
• Custom prompt editing UI (defaults only for v1; see [Feature] Local Wispr-Flow prompt templates + editing UI #5 for editable templates)
• Cross-transcript queries (that's the "Ask XooMe AI" RAG feature, separate)

References

• https://wispr.flow
• https://github.com/ml-explore/mlx-swift-examples
• https://ollama.com