Feat/session compaction #300
Conversation
…ken usage in long conversations - Added a new example for session history compaction in GETTING_STARTED.md and README.md. - Implemented a debug example to visualize compaction triggers and effects. - Created compaction filter logic to manage event history efficiently. - Developed comprehensive tests for the compaction filter to ensure correct behavior. - Enhanced session management with compaction capabilities, including configuration options for interval and overlap. - Documented configuration and usage guidelines for effective integration into applications.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Summary of ChangesHello @raphaelmansuy, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new 'Session Compaction' feature designed to optimize token usage in long-running conversations. It achieves this by automatically summarizing older parts of the conversation history using an LLM, replacing the original events with a concise summary in the LLM's context. This significantly reduces the number of tokens sent to the LLM while preserving conversational context, leading to cost savings and improved performance for extended interactions. The feature is configurable through new Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a session history compaction feature, which is a valuable addition for managing token usage in long conversations. The implementation includes a new compaction package with core logic, configuration, and comprehensive tests. The feature is also plumbed through the runner and various launchers.
However, there are a few critical issues that prevent this feature from working as intended and should be addressed:
- Incomplete Integration: The logic to filter out original events that have been compacted is implemented in
internal/context/compaction_filter.go, but it is not being used before sending the conversation history to the LLM. This means both the original events and their summaries will be included in the context, which negates the benefit of compaction and may lead to incorrect model behavior. The comments insession/inmemory.goconfirm this is a known omission. - Race Condition: There is a critical race condition in
runner/runner.goduring the initialization of thecompactorinstance, which can occur when multiple compactions are triggered concurrently.
I've included specific comments on these critical issues and a few other suggestions for improving correctness and robustness. Addressing the critical issues is necessary for this feature to be functional and safe to merge.
- Fix race condition in runner.go: Use sync.Once for thread-safe compactor initialization * Prevents concurrent initialization of r.compactor in maybeCompact() * Multiple goroutines can now safely call maybeCompact() concurrently - Fix timing calculation in compactor.go: Sort events before calculating time range * Previously calculated startTime/endTime during event collection loop * Now sorts events first, then correctly determines time range from sorted events * Ensures accurate compaction window boundaries - Integrate compaction filtering in session/inmemory.go: Filter events in Get() * Implements filterEventsForCompaction() to replace original events with summaries * Excludes original events that fall within compaction windows * Includes all compaction events (summaries) * Achieves 60-80% token reduction for long-running conversations * Prevents LLM from seeing redundant content (original + summary)
|
How does this change relate to ADK's official compaction capabilities? An experimental implementation is at https://github.com/google/adk-python/blob/f7f6837fdede6a7670ceedfbeac07f926506f7f6/src/google/adk/apps/app.py#L63 and we will port it to ADK Go once it's more mature. |
Yes it is based on this experimental implementation |
|
Oh, great Since this is a new library, could we import the feature first and then mark it as an experimental, unstable API for use? |
3 participants
No description provided.