Skip to content

[Feature Request] Add context window management and conversation summarization support #1173

@jeanlopezxyz

Description

@jeanlopezxyz

Summary

When using kagent with models that have limited context windows (e.g., gpt-4o-mini with 128k tokens), agents with many MCP tools can exceed the context limit, causing "Session terminated" errors.

Currently, kagent does not expose ADK's context management features through the Agent CRD.

Problem

With an agent configured with ~40+ MCP tools, the context quickly exceeds limits:

  • Each tool definition adds tokens to the context
  • Conversation history accumulates
  • No way to truncate, summarize, or manage context programmatically

Error observed:

context_length_exceeded: This model's maximum context length is 128000 tokens. 
However, your messages resulted in 155XXX tokens.

Proposed Solution

Expose ADK's context management capabilities through the Agent CRD:

Option 1: Context Compression (Compaction)

ADK has Context Compression that summarizes older events:

apiVersion: kagent.dev/v1alpha2
kind: Agent
spec:
  declarative:
    contextCompression:
      enabled: true
      compactionInterval: 5  # Compress every 5 events
      overlapSize: 1
      summarizer:
        type: llm  # Use LlmEventSummarizer
        model: gpt-4o-mini

Option 2: Context Caching

ADK has Context Caching for optimizing repeated requests:

spec:
  declarative:
    contextCache:
      enabled: true
      minTokens: 1000
      ttlSeconds: 3600
      cacheIntervals: 10

Option 3: Context Window Compression Config

From the ADK API, there's ContextWindowCompressionConfig:

spec:
  declarative:
    contextWindowCompression:
      enabled: true
      maxLength: 100000  # Max context length before compression

Alternatives Considered

  1. Reduce tools: Works but limits agent capabilities
  2. Use larger context models: More expensive (gpt-4o vs gpt-4o-mini)
  3. Memory/RAG: Currently not supported in ADK (v0.6 release notes confirm this)

Additional Context

Environment

  • kagent version: v0.7.6
  • Kubernetes: OpenShift 4.17
  • Model: gpt-4o-mini (128k context)
  • Tools: 16 MCP tools (reduced from 42 to mitigate issue)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions