💡 Infinite Conversations via Client-Side Compaction

[don-petry]

Summary

Leverage the Claude SDK'"'"'s new client-side compaction feature to enable effectively infinite conversation sessions. When the context window fills up during long workflows, the SDK automatically summarizes earlier conversation turns while preserving key decisions and context. This directly enables TalkTerm'"'"'s cross-session memory vision (FR46-47) without custom summarization logic, allowing the avatar to maintain rich understanding of a user'"'"'s work across weeks and months of interaction.

Market Signal

The Claude API platform launched client-side compaction in both Python and TypeScript SDKs in early 2026, with server-side compaction API in beta on Opus 4.6. This is a direct response to the industry'"'"'s "context window problem" — agents that lose context mid-workflow break user trust. Competitors like ChatGPT Atlas handle this with conversation branching, while Devin uses project-level memory stores. The compaction approach is unique in preserving conversational continuity rather than forcing users to start new sessions, which aligns perfectly with TalkTerm'"'"'s relationship-first design philosophy.

User Signal

FR46 requires persisting "conversation context, user decisions, project vocabulary, and workflow outcomes across sessions." FR47 requires the avatar to "reference past work naturally in conversation." The architecture calls for "lightweight memory index in SQLite (key decisions, project vocabulary, user preferences) for fast avatar greeting personalization." Currently, without compaction, long BMAD workflows (brainstorming can generate 24+ ideas) risk hitting context limits. The PRD'"'"'s User Journey 1 shows Sarah using TalkTerm daily — compaction is what makes multi-day continuity possible.

Technical Opportunity

The Claude Agent SDK'"'"'s tool_runner now supports automatic client-side compaction — it is a configuration option, not a custom implementation. The architecture already stores session IDs in SQLite for resume (FR31), and the compaction summaries can be stored alongside them. The AgentBackend port in the Agent bounded context can expose compaction configuration as part of session options. The get_context_usage() method enables the UI to show users how much context remains, which pairs naturally with the effort modes proposal for cost transparency.

Assessment

Dimension	Score	Rationale
Feasibility	high	SDK provides compaction out of the box; configuration-level change, not a custom build
Impact	high	Enables the core "relationship layer" vision — avatar that remembers and grows with the user
Urgency	high	Without compaction, long workflows and multi-session continuity are fundamentally limited by context window

Suggested Next Step

Spike on enabling client-side compaction in the Claude Agent SDK TypeScript integration. Test with a multi-session BMAD brainstorming workflow to validate that compacted context preserves decision history and project vocabulary. Design the SQLite schema for storing compaction summaries alongside session metadata.

---

Proposed by the BMAD Analyst (Mary) on 2026-04-05.

[github-actions[bot]]

Research Enhancement — Infinite Conversations via Client-Side Compaction

Critical Update: Client-Side Compaction is Deprecated

The original proposal assumed the Claude SDK's client-side compaction_control parameter. This parameter is now deprecated in the Python, TypeScript, and Ruby SDKs and will be removed in a future version. The SDKs emit a deprecation warning when it is enabled.

Anthropic's recommended replacement is server-side compaction via the Compaction API (beta: compact-2026-01-12). This is a significant architectural shift — compaction moves from the SDK's agent loop to the Messages API itself.

Sharpened Problem Framing

The core problem remains valid and urgent: without compaction, long agentic workflows (BMAD brainstorming can generate 24+ ideas across many turns) hit context window limits, breaking the "relationship layer" that makes TalkTerm's avatar meaningful. But the implementation path has changed — the solution is now server-side API configuration, not SDK-level configuration.

Server-Side Compaction: How It Works

1. Include context_management: { edits: [{ type: "compact_20260112" }] } in Messages API calls
2. API monitors token usage; when trigger threshold is reached (default: 150K tokens, min: 50K), it generates a summary
3. A compaction block replaces older conversation history
4. Subsequent requests automatically drop content before the compaction block
5. Requires beta header: anthropic-beta: compact-2026-01-12

Supported models: Opus 4.6-4.8, Sonnet 4.6-5, Fable 5, Mythos 5

Prior Art & Competitive Approaches

Product	Context Management	Key Lesson
Claude Code	Auto-compact (uses compaction internally)	Proves the pattern works for long agentic sessions
ChatGPT Atlas	Conversation branching	Forces users to manage context manually — worse UX
Devin	Project-level memory stores	Separates "memory" from "context" — heavier but more structured
Cursor / Cline	Context window + manual summarization	No automatic solution — users manually restart sessions

Options Table

Option	Pros	Cons	Effort
A. Server-side compaction via Messages API	Anthropic-recommended; better token usage calculation; less integration complexity	Requires beta header; additional sampling step costs tokens; TalkTerm must pass the parameter through the Agent SDK to the underlying API	Med
B. Custom summarization (build your own)	Full control over what is preserved	Significant engineering; inferior quality vs Claude's own summarization; maintenance burden	High
C. Hybrid: server-side compaction + SQLite memory index	Compaction handles in-session context; SQLite stores cross-session decisions/vocabulary per FR46-47	More complex architecture; two persistence paths	Med-High

Recommendation: Option C — server-side compaction solves the in-session context window problem, while the SQLite memory index (already planned in the architecture) handles cross-session memory. These are complementary, not alternatives. The AgentBackend port can expose compaction configuration as part of session options, and the trigger threshold can be linked to effort modes (Discussion #55) — Turbo mode might use a higher trigger to preserve more context.

Risks & Unknowns

• Beta API stability: The compaction API is still in beta (compact-2026-01-12). The beta header and parameter shape may change before GA. Design the integration to isolate the beta-specific code.
• Cost of compaction: Each compaction event is an additional sampling step that consumes tokens and counts against rate limits. For long sessions with many compaction cycles, this adds up. The usage.iterations array now separates compaction costs — surface this in the session cost display (ties to Discussion 💡 Effort Modes: Economy / Power / Turbo Session Tiers #55).
• Summary quality for domain-specific content: Compaction summarization may lose domain-specific terminology or project vocabulary that FR46 requires persisting. The instructions parameter allows custom summarization prompts — TalkTerm should use this to instruct Claude to preserve project-specific context.
• Agent SDK passthrough: The Claude Agent SDK manages the conversation loop internally. Server-side compaction needs to be passed through the SDK's Messages API calls. Verify that the TypeScript Agent SDK exposes the context_management parameter or allows raw API options passthrough.
• pause_after_compaction option: The API supports pausing after compaction for manual summary adjustment. This could power a UX where the avatar says "I've summarized our earlier conversation — here's what I'm keeping in mind" before continuing.

Phased Validation Path

1. Phase 1: Spike on passing server-side compaction parameters through the Claude Agent SDK TypeScript integration. Verify the SDK supports context_management passthrough to the Messages API. Test with a long BMAD brainstorming workflow to validate that compacted context preserves key decisions.
2. Phase 2: Implement pause_after_compaction: true with avatar narration — "Let me organize what we've discussed so far" — turning a technical necessity into a natural conversation beat.
3. Phase 3: Wire compaction trigger threshold to effort modes. Build the SQLite cross-session memory index alongside compaction for the full FR46-47 vision.

Success Metrics

• Session length: Sessions can exceed 200K tokens without context degradation (baseline: hard fail at context limit)
• Context fidelity: Key decisions from compacted sessions are retrievable with >90% accuracy (measured via automated Q&A test)
• User perception: <5% of users report the avatar "forgetting" something discussed earlier in a session
• Cost efficiency: Compaction overhead adds ≤15% to total session token cost