Repository navigation
【No.8】Agentic 多协议一致性测试 - #344
Closed
AllureCurtain wants to merge 2 commits into
Closed
AllureCurtain wants to merge 2 commits into
AllureCurtain wants to merge 2 commits into
Conversation
# 🐛 Bug Fix
## Canonicalize image blocks in check_messages
- Add `_canonical_image_block`: every legal image spelling across the three
ingress protocols normalizes to `{"type": "image_url", "image_url": {"url": ...}}`
- Drop request-only keys (e.g. OpenAI `detail`) from the canonical state so two
spellings of one image cannot produce two state hashes
- Reject a missing, empty or non-string url, reporting the exact field path
## Require tool call ids
- `tool_calls[].id` must now be a non-empty string; a missing id used to pass
validation and let a malformed tool call into the canonical session state
# ✅ Tests ## Add golden tests for the three agentic ingress protocols - Add `tests/agentic/test_protocol_golden.py` (114 cases) pinning the canonical `messages` / `tools` / `chat_template_kwargs` triple and session hash for Chat Completions, OpenAI Responses and Anthropic Messages - 10 semantic scenarios authored natively in all three protocols, asserted in three layers: per-protocol golden, cross-protocol agreement, one hash - 9 protocol-specific rules; four tool-argument encodings collapse to one canonical JSON string - 25 illegal inputs assert exact message, `param` and cause type - 7 tolerated anomalies and 2 dataset-shaped turns pinned, so later tightening is deliberate - Add `tests/agentic/__init__.py` matching the existing per-directory layout
AllureCurtain
requested review from
NINGBENZHE and
Yangruipis
as code owners
September 20, 2026 06:44
Contributor
Nyanpasu 审查看板审查状态: ✅ 已通过 审查版本: 已完成完整变更审查,未发现需要修改的问题。真实规范化函数的隔离验证 114 项通过;标准 pytest 因缺少 Ray 无法收集,暂无 CI 检查,未运行模型服务或多节点 GPU 集成测试。 没有未解决的审查问题。
Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.
|
rai-studio-bot
approved these changes
Sep 20, 2026
rai-studio-bot
left a comment
Contributor
There was a problem hiding this comment.
已审查 39bbf746 的完整变更及共享校验的调用路径,未发现需要修改的问题,同意合入。
从当前源码提取真实规范化函数进行隔离验证,114 项测试通过;标准 pytest 因缺少 Ray 在收集阶段停止,因此不视为完整模块测试通过。当前未发现 CI 检查,未运行模型服务或多节点 GPU 集成测试(本环境缺少相应依赖与硬件)。
Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.
Author
|
@SigureMo 任务已完成,请求 review。 |
Member
|
本赛题已经由 #335 合入并锁定,感谢参与~ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
messages/tools/chat_template_kwargstriple.paramand cause type with full field paths such asmessages[2].tool_calls[0].id.relax/agentic/session/state.pywhere malformed input used to pass validation silently: canonicalize every legal image spelling to one block form (dropping request-only keys such as OpenAI'sdetail), and require a non-emptytool_calls[].id.Type of Change
Test plan
pytest tests/agentic/test_protocol_golden.py -q— 114 items, CPU only; no network, model service or GPU involved.pre-commit run --fileson all touched files — every hook passes (ruff, docformatter, gitleaks, whitespace/EOF checks).0651812).Conventions
tests/per-directory structure (tests/agentic/__init__.py), and golden values are authored inline, consistent with the existing canonicalization tests intests/test_agentic_rollout.py.check_messages/normalize_tools/normalize_template_kwargsas the task requires; no new dependencies.Related to #321