Skip to content

feat(chat): context overflow protection — auto-summarize conversation when approaching model context limit #9043

Closed

Description

@mrveiss

Summary

Long conversations silently hit the model's context window limit, causing the model to lose early context, hallucinate, or fail entirely. Users have no warning and no automatic protection. This is especially acute for LLC agents running multi-hour tasks.

What Users Want

  • Detect when conversation is approaching the context limit (>80% full — builds on feat(chat): context window usage indicator — show token consumption during conversation #8990)
  • Auto-summarize the earliest portion of the conversation history and replace it with a compact summary
  • 'Rolling window' mode: always keep the last N tokens verbatim; summarize everything older
  • User preference: warn only / auto-summarize / hard stop
  • For LLC agents: checkpoint before summarization so the original history is preserved
  • Summary injected as a system message so the model knows context was compressed

AutoBot Fit

autobot-backend/chat_history/ manages session history. New ContextCompressor service: monitors token count, triggers summarization via the LLM gateway when threshold is crossed. Summarization prompt: 'Compress the following conversation segment preserving all decisions, facts, and action items.' LLC heartbeat runs: compress between task steps to stay within context.

Acceptance Criteria

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions