Skip to content

feat: summarizing conversation manager #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions docs/user-guide/concepts/agents/context-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,108 @@ Key features of the `SlidingWindowConversationManager`:
- **Dangling Message Cleanup**: Removes incomplete message sequences to maintain valid conversation state.
- **Overflow Trimming**: In the case of a context window overflow, it will trim the oldest messages from history until the request fits in the models context window.
- **Configurable Tool Result Truncation**: Enable / disable truncation of tool results when the message exceeds context window limits. When `should_truncate_results=True` (default), large results are truncated with a placeholder message. When `False`, full results are preserved but more historical messages may be removed.

#### SummarizingConversationManager

The [`SummarizingConversationManager`](../../../api-reference/agent.md#strands.agent.conversation_manager.summarizing_conversation_manager.SummarizingConversationManager) implements intelligent conversation context management by summarizing older messages instead of simply discarding them. This approach preserves important information while staying within context limits.

Configuration parameters:

- **`summary_ratio`** (float, default: 0.3): Percentage of messages to summarize when reducing context (clamped between 0.1 and 0.8)
- **`preserve_recent_messages`** (int, default: 10): Minimum number of recent messages to always keep
- **`summarization_agent`** (Agent, optional): Custom agent for generating summaries. If not provided, uses the main agent instance. Cannot be used together with `summarization_system_prompt`.
- **`summarization_system_prompt`** (str, optional): Custom system prompt for summarization. If not provided, uses a default prompt that creates structured bullet-point summaries focusing on key topics, tools used, and technical information in third-person format. Cannot be used together with `summarization_agent`.

**Basic Usage:**

By default, the `SummarizingConversationManager` leverages the same model and configuration as your main agent to perform summarization.

```python
from strands import Agent
from strands.agent.conversation_manager import SummarizingConversationManager

agent = Agent(
conversation_manager=SummarizingConversationManager()
)
```

You can also customize the behavior by adjusting parameters like summary ratio and number of preserved messages:

```python
from strands import Agent
from strands.agent.conversation_manager import SummarizingConversationManager

# Create the summarizing conversation manager with default settings
conversation_manager = SummarizingConversationManager(
summary_ratio=0.3, # Summarize 30% of messages when context reduction is needed
preserve_recent_messages=10, # Always keep 10 most recent messages
)

agent = Agent(
conversation_manager=conversation_manager
)
```

**Custom System Prompt for Domain-Specific Summarization:**

You can customize the summarization behavior by providing a custom system prompt that tailors the summarization to your domain or use case.

```python
from strands import Agent
from strands.agent.conversation_manager import SummarizingConversationManager

# Custom system prompt for technical conversations
custom_system_prompt = """
You are summarizing a technical conversation. Create a concise bullet-point summary that:
- Focuses on code changes, architectural decisions, and technical solutions
- Preserves specific function names, file paths, and configuration details
- Omits conversational elements and focuses on actionable information
- Uses technical terminology appropriate for software development

Format as bullet points without conversational language.
"""

conversation_manager = SummarizingConversationManager(
summarization_system_prompt=custom_system_prompt
)

agent = Agent(
conversation_manager=conversation_manager
)
```

**Advanced Configuration with Custom Summarization Agent:**

For advanced use cases, you can provide a custom `summarization_agent` to handle the summarization process. This enables using a different model (such as a faster or a more cost-effective one), incorporating tools during summarization, or implementing specialized summarization logic tailored to your domain. The custom agent can leverage its own system prompt, tools, and model configuration to generate summaries that best preserve the essential context for your specific use case.

```python
from strands import Agent
from strands.agent.conversation_manager import SummarizingConversationManager
from strands.models import AnthropicModel

# Create a cheaper, faster model for summarization tasks
summarization_model = AnthropicModel(
model_id="claude-haiku-4-20250514", # More cost-effective for summarization
max_tokens=1000,
params={"temperature": 0.1} # Low temperature for consistent summaries
)
custom_summarization_agent = Agent(model=summarization_model)

conversation_manager = SummarizingConversationManager(
summary_ratio=0.4,
preserve_recent_messages=8,
summarization_agent=custom_summarization_agent
)

agent = Agent(
conversation_manager=conversation_manager
)
```

Key features of the `SummarizingConversationManager`:

- **Context Window Management**: Automatically reduces context when token limits are exceeded
- **Intelligent Summarization**: Uses structured bullet-point summaries to capture key information
- **Tool Pair Preservation**: Ensures tool use and result message pairs aren't broken during summarization
- **Flexible Configuration**: Customize summarization behavior through various parameters
- **Fallback Safety**: Handles summarization failures gracefully