Skip to content

feat(chat): tiered L0-L3 context wake-up to replace unconditional prompt injection #5066

Closed

Description

@mrveiss

Problem

AutoBot currently loads context into agent sessions via unconditional snippet injection (e.g. `CAUSAL_REASONING_SNIPPET` — see #4811) and full chat-history recall. This wastes tokens and degrades signal-to-noise: every agent gets the same preamble regardless of what the session is about.

There is no token-budgeted, tiered wake-up strategy. The agent either has too little context (cold start) or too much (session with accumulated history).

Proposed change

Introduce a 4-tier wake-up stack in `autobot-backend/chat_history/layers.py`:

  • L0 Identity (~100 tok, always loaded) — who this agent is, role, owner. From a small config file / identity table.
  • L1 Essential Story (~500-800 tok, always loaded) — auto-generated from highest-weight / most-recent memory-graph entities and KB facts for the current user/session.
  • L2 On-Demand (~200-500 tok each) — loaded when a topic/entity is mentioned in the user message; scoped pull from memory graph and KB.
  • L3 Deep Search (unlimited) — existing hybrid search + rerank path, invoked only for explicit retrieval-heavy queries.

Target: wake-up cost ≤ 900 tokens for L0+L1, leaving ≥95% of the context window for actual conversation.

Acceptance criteria

Related

Activity

  1. mrveiss commented on Apr 28, 2026

    @mrveiss
    OwnerAuthor

    ✅ Closed — merged to Dev_new_gui.

    Merge commit: 52ac25c feat(chat): tiered L0-L3 context wake-up to replace unconditional prompt injection (#5066) (#6436)

    Acceptance criteria met:

    • ✅ Layer0/Layer1/Layer2/Layer3 with render() + token_estimate() — chat_history/layers.py
    • ✅ Wired into chat_workflow/llm_handler.py _prepare_llm_request_params
    • ✅ L0+L1 combined ≤ 900 tokens — enforced in code and asserted in tests
    • ✅ L2 fires on entity detection (capitalized words + memory graph lookup)
    • ✅ Feature flag TIERED_CONTEXT_ENABLED=true/false (default false) — safe A/B rollout
    • ✅ 17 tests; L1 delegates to existing EssentialStoryGenerator

    Discovery issues filed: none found

  2. added 4 commits that reference this issue on Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions