Description
Problem Statement
The addition of a ConversationManager that can summarize and replace the conversation thus far. Currently, when faced with a ContextWindowOverflowError, the Agent can be configured with a SlidingWindowConversationManager that trims off the oldest messages in the conversation until it fits into the models context window. This leads to the deletion of context from the conversation.
Proposed Solution
When trying to reduce the context of the conversation history, instead of removing context all-together, instead the Agent can use an LLM to summarize the conversation thus-far, and then use the summarization to replace the history.
- Create a new
CompactionConversationManager
class - Implement the
reduce_context
function that will call an agent with the compaction prompt and the current messages array. It will overwrite the messages history with the summarized version. - Have this
ConversationManager
use the parentAgent
model_provider by default, but allow for an optionalsystem_prompt
override. - The new
ConversationManager
can also take in anAgent
at initialization time that would be used for summarization instead of the default parent Agent. This allows for the specification of a summarization agent that can use tools as part of the summarization process.
Considerations:
- This will need to update
reduce_context
abstract method to pass in anagent: Agent
parameter - Tools should be able to be used by the summarization agent override
- If the messages array is too large to summarize, break it in half by character count, and summarize each part
Use Case
- If the messages array overflows during conversation, the conversation is summarized and can continue as normal.
Alternatives Solutions
No response
Additional Context
No response