feat(config): add GLM-5.2 FP4 GB200 SGLang AgentX MTP / 添加 GLM-5.2 FP4 GB200 SGLang AgentX MTP 配置 - #2642
feat(config): add GLM-5.2 FP4 GB200 SGLang AgentX MTP / 添加 GLM-5.2 FP4 GB200 SGLang AgentX MTP 配置#2642RohitNagraj wants to merge 5 commits into
Conversation
添加 GLM-5.2 FP4 GB200 Dynamo-SGLang AgentX MTP 配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| prefill: | ||
| enable-flashinfer-allreduce-fusion: true | ||
| data-parallel-size: 8 | ||
| expert-parallel-size: 8 | ||
| hicache-size: 100 | ||
| max-prefill-tokens: 8192 | ||
| tensor-parallel-size: 8 |
There was a problem hiding this comment.
🟡 In zip_override_mtp_agentx_dep16.backend.sglang_config.prefill (lines 251-257), tensor-parallel-size: 8, data-parallel-size: 8, expert-parallel-size: 8, and max-prefill-tokens: 8192 merely restate the exact values already set in base.backend.sglang_config.prefill, adding no behavior. Only enable-flashinfer-allreduce-fusion and hicache-size actually change for this override; trimming the four redundant keys (as the sibling zip_override_mtp_agentx_tp4 correctly does) would avoid the false impression that DEP16 retunes prefill parallelism and the risk of silent divergence if base is edited later.
Extended reasoning...
zip_override_mtp_agentx_dep16 is a zip-override block that deep-merges over base, so any key it does not restate is inherited unchanged from the base recipe. In its prefill sub-block (lines 251-257), it explicitly sets tensor-parallel-size: 8, data-parallel-size: 8, expert-parallel-size: 8, and max-prefill-tokens: 8192 — all four are byte-identical to the values already declared in base.backend.sglang_config.prefill (lines 68-70 and 77). The only keys in this override block that actually change behavior relative to base are enable-flashinfer-allreduce-fusion (false -> true) and hicache-size (135 -> 100).
That the merge is a deep-merge inheriting unset keys from base — rather than a full block replacement — is proven within the same file: the sibling zip_override_mtp_agentx_tp4 prefill block sets only enable-flashinfer-allreduce-fusion and max-prefill-tokens, deliberately omitting tensor-parallel-size, data-parallel-size, and expert-parallel-size and relying on inheritance from base for those. If DEP16s restatement were required for correctness, TP4 would be broken by the same logic, but TP4 is the pattern the file otherwise follows consistently.
Because this is inert restatement rather than a functional change, it does not affect the rendered job at runtime — srtctl will resolve identical parallelism values whether or not these four keys are repeated. The impact is purely a maintenance/readability hazard: a reader skimming the DEP16 override sees tensor-parallel-size, data-parallel-size, expert-parallel-size, and max-prefill-tokens under prefill and reasonably infers DEP16 retunes prefill topology, when in fact only allreduce fusion and hicache sizing change. Worse, if a future PR changes base.backend.sglang_config.prefill.tensor-parallel-size (e.g., to retune prefill for a new checkpoint), this shadow copy in DEP16 will silently keep the old value of 8 instead of inheriting the update, since the override now wins over base for that key.
Step-by-step proof:
- Read
base.backend.sglang_config.prefill:tensor-parallel-size: 8,data-parallel-size: 8,expert-parallel-size: 8,max-prefill-tokens: 8192. - Read
zip_override_mtp_agentx_dep16.backend.sglang_config.prefill(lines 251-257): the same four keys with the same four values, plusenable-flashinfer-allreduce-fusion: trueandhicache-size: 100. - Compare with
zip_override_mtp_agentx_tp4.backend.sglang_config.prefill, which sets onlyenable-flashinfer-allreduce-fusion: trueandmax-prefill-tokens: 16384— omitting tp/dp/ep entirely and relying on base inheritance, confirming the deep-merge model. - Therefore, dropping the four redundant keys from DEP16 prefill leaves the effective/rendered configuration byte-for-byte identical (all four values still resolve to
8/8192via inheritance from base), while removing the redundant keys eliminates the false impression of a prefill-parallelism retune and the future silent-divergence risk.
The fix is simple: delete tensor-parallel-size: 8, data-parallel-size: 8, expert-parallel-size: 8, and max-prefill-tokens: 8192 from the zip_override_mtp_agentx_dep16 prefill block, leaving only enable-flashinfer-allreduce-fusion: true and hicache-size: 100, matching the minimal-delta convention already used by zip_override_mtp_agentx_tp4. This is a pure code-quality/consistency cleanup with no behavioral effect, so it does not block merging.
| - config-keys: | ||
| - glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp | ||
| scenario-type: | ||
| - agentic-coding | ||
| description: | ||
| - "Add GLM-5.2 NVFP4 GB200 AgentX with Dynamo-SGLang disaggregated serving across five recipe variants." | ||
| - "Use NIXL KV transfer, prefill HiCache DRAM offload, native MTP with the committed thinking-on golden acceptance length, and lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
There was a problem hiding this comment.
🟡 perf-changelog.yaml ends the new entry with a literal placeholder pull/XXX instead of the real PR number; every other entry records its actual PR link. Please replace it with https://github.com/SemiAnalysisAI/InferenceX/pull/2642 before merge, since perf-changelog.yaml is append-only and this would otherwise bake an unresolvable reference into history permanently.
Extended reasoning...
What the bug is
The new perf-changelog.yaml entry added by this PR (for glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp) ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXXXX is a literal, unfilled placeholder rather than an actual PR number. Every neighboring entry in the same file records a real PR number for pr-link, e.g. .../pull/2636, .../pull/2621, .../pull/2620, .../pull/2619 (the three entries immediately preceding this one at lines ~6050, 6059, 6068, 6077).
Why it matters here specifically
perf-changelog.yaml is called out in the project conventions (and in this PR's own checklist) as append-only and byte-sensitive — historical entries are never edited after being merged, only appended to. That means whatever value ships in this entry becomes a permanent part of the file's history. A pull/XXX placeholder is not a transient typo that gets caught later; once merged it is an unresolvable dangling reference that anyone auditing this changelog entry (or tooling that parses pr-link to cross-reference a PR) will hit indefinitely.
Why the existing code/process does not catch it
The PR author's own checklist claims local validation parsed the changelog YAML and ran various dry-run/consistency checks, but none of those checks assert that pr-link matches a real, resolvable PR URL — they only check YAML syntax and config/recipe parity. There is nothing enforcing that the placeholder gets swapped for the real number before merge.
The fix
Since this PR is #2642 (known at review time), the fix is a one-line change:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2642Step-by-step proof
- Open
perf-changelog.yamland note the file convention: each entry's trailingpr-linkfield records the exact PR number that introduced it (confirmed by the three entries directly above this one: 2620, 2621, 2619/2636 merge). - Look at the new entry added in this diff (lines 6079-6086): its
pr-linkreads.../pull/XXX. - Check the PR metadata: this change is submitted as PR feat(config): add GLM-5.2 FP4 GB200 SGLang AgentX MTP / 添加 GLM-5.2 FP4 GB200 SGLang AgentX MTP 配置 #2642.
- Compare:
XXX != 2642, so the field is factually wrong/incomplete relative to every other entry's convention. - Because the file is append-only, once merged this entry can never be edited in place to correct the number — the placeholder is permanent unless fixed now, before merge.
This is a nit — it does not affect benchmark behavior, recipe correctness, or CI, but it is a trivial one-line fix that avoids a permanent, unresolvable placeholder in an append-only audit trail.
保留 GLM-5.2 TP8 配方的前端设置。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32099186956 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32099186956 |
更新 GLM-5.2 GB200 AgentX 配方集合，并同步当前主分支以解决性能变更日志冲突。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32099186956 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32207758126 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32207758126 |
1 participant
Description
Add a GLM-5.2 NVFP4 AgentX configuration for GB200 using Dynamo-SGLang disaggregated serving.
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642./mnt/numa1/models/GLM-5.2-NVFP4while preserving the existing path used by other GLM-5.2 GB200 recipes.benchmark, partitionbatch, andsa-sharedcontainer storage.Local validation:
srtctl dry-runfor all five recipe selectors against the pinned v1.0.50 SRT contract.git diff --check.RECIPE_NOT_APPLICABLE— disaggregated/multi-node submission.中文说明
为 GB200 添加使用 Dynamo-SGLang 解耦部署的 GLM-5.2 NVFP4 AgentX 配置。
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642。/mnt/numa1/models/GLM-5.2-NVFP4，同时保留其他 GLM-5.2 GB200 配方现有的模型路径。benchmark账户、batch分区和sa-shared容器存储。本地验证：
srtctl dry-run。git diff --check。RECIPE_NOT_APPLICABLE— 解耦/多节点提交。Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.