feat(gateway): auto-compact long sessions and retry once on context overflow - #424
feat(gateway): auto-compact long sessions and retry once on context overflow#424veenyi wants to merge 1 commit into
Conversation
…verflow
IM/dashboard sessions grow unbounded; once the conversation exceeds the
model context window the LLM returns 400 ("Requested token count exceeds
the model's maximum context length") and messages are silently dropped —
users must manually delete the session to restore the channel. deepagents'
SummarizationMiddleware (default 85% trigger) does not fire reliably in
this stack, so add a belt-and-braces layer in the message processor:
- Before each turn, check thread context usage (aget_context_usage); when
it exceeds 75% of the model window, auto-compact via acompact_conversation
(keeps recent messages + summary, content is not lost).
- If a turn still fails with a context-overflow error, compact and retry
once so long-running QQ/IM sessions keep working without manual resets.
Applies to all channels (QQ group/DM, dashboard, CLI).
Windows CI failure — pre-existing, not from this PRThe failing check is: That test (introduced in #420) simulates a missing
Could the maintainers take a look at making |
沟通语言说明后续沟通请用中文(团队是腾讯中国团队)。以下是 Windows CI 失败的中文说明: Windows CI 失败是本 PR 之前就存在的问题,与本 PR 无关: 该测试由 #420 引入,模拟 getuid 缺失场景,但 Windows 的 frozen os 模块根本没有 getuid,测试本身报错。它只涉及 browser 模块,与本 PR 改的 processor.py(网关会话处理)无关。
建议维护者将该测试在 Windows 上跳过(如 |
问题
会话(IM 群/私聊、工作台、CLI)无限增长。一旦对话超过模型上下文窗口,LLM 返回
400 "Requested token count exceeds the model's maximum context length",消息被静默丢弃——通道直接哑火,直到用户手动删除会话才恢复。deepagents 自带的 SummarizationMiddleware(默认 85% 触发)在该栈中未能可靠触发(实测会话涨到 518K token、模型窗口 512K 仍未压缩)。修复
在消息处理器(
octop/infra/gateway/process/processor.py)增加双层兜底:agent.aget_context_usage(thread_id)检查会话上下文占用,超过模型窗口 75% 时自动agent.acompact_conversation(thread_id)(保留最近消息 + 摘要,内容不丢失)。_is_context_overflow匹配 "Requested token count exceeds" / "maximum context length" / "input prompt token len"),自动压缩并重试一次——即使会话已超限也能自动恢复,无需用户手动操作。说明
GlobalProcessor.__call__,因此对所有通道生效(QQ 群/私聊、工作台、CLI)。aget_context_usage可能返回过期的内存快照(重试路径覆盖该情况);所有异常只记录日志、绝不向上抛出。