feat: observational memory — programmatic extraction + working memory block

[platypusrex]

Problem

When context compaction triggers, the summarizer produces a narrative summary that loses critical structured details — file paths, error messages, commands run, decisions made. In testing, a 142-message conversation produced a 474-char summary. The developer loses hours of context and has to re-explain everything.

Solution: Observational Memory (Phase 0)

Add a new memory layer that continuously extracts structured facts from tool calls and injects them as a "working memory block" in the system prompt. This complements (not replaces) the narrative summary from compaction.

What gets extracted (programmatically, zero LLM cost)

Tool	Observation	Importance
edit_file / write_file	File path, what changed	7
read_file	File path (low — reading isn't action)	3
run_command	Command, exit code, error if failed	4/8
glob / grep	Query, result count, top matches	3
todo_write	Task state changes	5
task (subagent)	Delegation summary	5

How it works

1. After every tool call, programmatic extractors produce Observation[] from the tool name, args, and result. Pure functions, <1ms.

2. Observations stored in SQLite — new observations table (migration v5). Session-scoped. Columns: id, session_id, type, content, metadata (JSON), timestamp, turn_index, importance, source.

3. Working memory block injected into system prompt before each LLM call. Structured sections (Current Task, Modified Files, Recent Errors, Decisions), capped at ~800 tokens. Built from observations by priority and recency.

4. Survives compaction. Observations are in a separate table, not in message history. After compaction, the working memory block is rebuilt from the full observation set — this is why observations preserve more context than summarization alone.

Hook point

onToolResult callback in use-agent-submit.ts already receives every tool result. Extract observations here and write to the store.

Working memory injection

buildInitialMessages() in src/agent/index.ts appends the working memory block to the system prompt.

Working memory format

## Working Memory

### Current Task
Implementing observational memory extraction for the CLI agent.
Step 3 of 5: Building retrieval logic.

### Modified Files
- src/memory/observations.ts (created this session)
- src/memory/extraction.ts (modified: added programmatic extractors)
- src/agent.ts (modified: integrated memory system)

### Recent Errors
- TypeScript: TS2345 in src/session/convert.ts line 45 (resolved)

### Key Decisions
- Chat history is never altered by compaction
- Using structured summarizer prompt with 6 sections

New files

• src/memory/types.ts — Observation type definitions
• src/memory/store.ts — ObservationStore (SQLite-backed, session-scoped)
• src/memory/extractors.ts — Programmatic extractors per tool
• src/memory/working-memory.ts — Builds the working memory text block

Modified files

• src/session/migrations.ts — Migration v5: observations table
• src/agent/index.ts — Accept + inject working memory into system prompt
• src/tui/hooks/use-agent-submit.ts — Extract observations after tool calls
• src/tui/index.tsx — Wire observation store

Out of scope (future phases)

• LLM-based extraction for goals, decisions, reasoning (Phase 1)
• Observation staleness detection and supersedes mechanism (Phase 1)
• Cross-session persistence for project facts/preferences (Phase 2)
• Semantic retrieval with embeddings (Phase 3)

Research

See docs/research/observational-memory.md for the full research report covering MemGPT, Generative Agents, Mem0, and coding-specific implementations (aider, Cline, opencode, continue.dev).

Related

• Follows from fix: redesign compaction — never alter chat history, fix token accuracy, use subagent for summarization #61 (compaction redesign)
• Depends on compaction redesign being merged (uses the summary + working memory hybrid pattern)