feat(minimaxm3): refresh B200 AgentX Pareto points with EAGLE3-GQA / 使用 EAGLE3-GQA 刷新 MiniMax-M3 B200 AgentX 帕累托点 - #2611
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31832925193 |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31833045923 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟣
benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh:129-132— This PR fixes CPU_OFFLOAD_BYTES in the B200 script to use decimal1000^3instead of binary1024^3(matching howTOTAL_CPU_DRAM_GBis computed in decimal GB byagentic_dram_offload_gb()), but its B300 sibling atbenchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh:46still uses1024*1024*1024, over-allocatingcpu_bytes_to_useby ~7.4% versus the intended decimal-GB budget. This is a pre-existing issue outside this PR's diff, but worth a follow-up fix since the B200 script explicitly aims to stay directly comparable to its B300 twin.Extended reasoning...
The bug:
benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh:46computesCPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1024 * 1024 * 1024)), using a binary (GiB) multiplier.TOTAL_CPU_DRAM_GBis produced upstream byagentic_dram_offload_gb()inutils/matrix_logic/generate_sweep_configs.py, which returnsint(proportional_bytes / BYTES_PER_GB)withBYTES_PER_GB = 1_000_000_000(line 30) — i.e. a decimal-GB count. Multiplying a decimal-GB integer by1024^3instead of1000^3silently inflates the resulting byte count by a factor of1024^3 / 1000^3 ≈ 1.0737, roughly a 7.4% over-allocation.\n\nThis exact PR's fix, applied inconsistently: this PR changes the identical line in the B200 twin script (minimaxm3_fp4_b200_mtp.sh:131) from1024*1024*1024to1000*1000*1000specifically to correct this unit mismatch. That confirms the authors/maintainers consider1000^3the correct multiplier forTOTAL_CPU_DRAM_GB. However, the PR only touches the B200 script; the B300 script that the B200 header explicitly calls out as a twin ("Every engine flag is the B300 script unchanged so the two SKU curves stay directly comparable") retains the old, now-inconsistent binary multiplier.\n\nStep-by-step proof: The B300 master-config comment (configs/nvidia-master.yaml:7407) documents a "1,024 GB engine-level CPU KV budget" intent for this offload arm. With the intended decimal semantics,1024 GBdecimal should become1024 * 1_000_000_000 = 1,024,000,000,000bytes (1.024 TB). But becauseminimaxm3_fp4_b300_mtp.sh:46multiplies by1024^3instead, it actually computes1024 * 1024^3 = 1,099,511,627,776bytes (~1.0995 TB) — about 75.5 GB (~7.4%) more host memory requested than the policy intends. The B300vllm-simpleoffload arm atdram-utilization: 0.683runs this exact code path today, so the miscalculation is live, not theoretical.\n\nWhy existing code doesn't catch it: there is no assertion or validation anywhere in the vLLM launch path (OFFLOAD_CONFIG/cpu_bytes_to_use) that cross-checks the requestedcpu_bytes_to_useagainst the intended decimal-GB policy value, so the wrong multiplier passes through silently as a plausible-looking but incorrect byte count.\n\nImpact and fix: at the current 0.683 DRAM utilization this is unlikely to push the host over its physical DRAM ceiling and cause an OOM, so it does not appear to break the B300 sweep today — but it undermines the explicit "directly comparable" contract between the two SKU scripts (the B300 run would now request ~7.4% more CPU KV cache capacity than the B200 run for what should be the same nominal budget), and could contribute to an OOM ifdram-utilizationis later raised. The fix is a one-line change identical to what this PR already did for B200: replace1024 * 1024 * 1024with1000 * 1000 * 1000onminimaxm3_fp4_b300_mtp.sh:46. Since this PR does not modify the B300 file at all, this is a pre-existing bug unrelated to this PR's own changes, but flagged here because the PR draws direct attention to the same bug in the sibling script it is explicitly designed to mirror. -
🟡
configs/nvidia-master.yaml:7461-7473— The header comment block at lines 7429-7447 was not updated alongside this PR's search-space rewrite, so it now flatly contradicts the config: it claims TP2 'does not fit on B200' and 'no conc-list would have made the arm run,' but lines 7471-7473 add a working TP2/C24 dram-offload arm. It also still describes TP4 no-offload as 'samples 5-15' (now [1,5,10,15,20], and doesn't mention the new TP8/C1 arm) and TP4+SimpleCPU as running '20...to 40' (now [30] only). Please rewrite the comment to describe the new search-space.Extended reasoning...
The descriptive comment block at
configs/nvidia-master.yaml:7429-7447is untouched by this diff, but the PR rewrites the exact search-space that block documents, leaving several concrete factual contradictions between the prose and the config directly below it.Contradiction 1 — TP2 feasibility. Line 7429 states unconditionally "TP4-only. The B300 sibling also runs a TP2 arm, but TP2 does not fit on B200," and lines 7434-7436 elaborate that this is "structural, not a concurrency cliff... so no conc-list would have made the arm run." The PR adds exactly the arm the comment says cannot exist:
{ tp: 2, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [24] }at line 7473, under a newdram-utilization: 1.0scenario block. The PR's own new inline comment at 7464-7465 even explains how this works ("TP2 uses 1.0 so 2 of 8 GPUs receive the 750 GB policy maximum") — the SimpleCPU KV offload changes the memory math that made TP2 infeasible without offload, but the older block was never reconciled with this.Contradiction 2 — TP4 no-offload concurrency description. Line 7440 says "B200 samples 5-15 densely... and retains 20 past it," implying a list like
[5, 8, 10, 12, 15, 20](matching the pre-PR config). The new list at line 7469 is[1, 5, 10, 15, 20]— it drops the dense 8/12 samples and adds a new low point at 1. The comment also never mentions the brand-new{ tp: 8, ..., conc-list: [1] }arm at line 7468 at all.Contradiction 3 — SimpleCPU offload range. Lines 7442-7444 say the TP4+SimpleCPU arm "picks up at 20 rather than B300's 30... and stops at 40," describing a sweep across the 20-40 range. Line 7470 narrows this arm to
conc-list: [30]— a single point, not a range, and one that no longer even reflects the "20 rather than 30" framing the comment uses to contrast with B300.Why this isn't caught elsewhere: the diff only touches the config lines starting at 7461 (previously 7455) and adds a short, narrowly-scoped inline comment at 7464-7465 about the
dram-utilizationvalues. It does not touch or invalidate the broader block above it, so nothing in the review flow forces a reconciliation. A future reader (or another PR author extending this recipe) skimming the header comment would incorrectly conclude TP2 is structurally impossible on B200 and that the offload arm still sweeps 20-40, when neither is true after this PR.Proof walkthrough: (1) Read lines 7429-7436 — claim: "TP2 does not fit on B200... no conc-list would have made the arm run." (2) Read line 7473 — fact: a TP2 arm with
conc-list: [24]is now live in the config. (3) These two statements about the same recipe cannot both be true simultaneously; the comment is stale. Same pattern applies to the TP4 conc-list description (comment: "5-15... retains 20" vs. actual[1,5,10,15,20]) and the SimpleCPU range description (comment: "20...to 40" vs. actual[30]).Fix: update the comment block (roughly lines 7429-7447) to state that TP2 is now viable when paired with SimpleCPU DRAM offload at
dram-utilization: 1.0(referencing the new inline comment's explanation), and update the described conc-lists for the TP4 no-offload and TP4+offload arms, plus mention the new TP8/C1 arm, so the prose matches the search-space it sits above.This is a documentation-only issue — the config itself is internally consistent and functionally correct, so it doesn't block merging, but a maintainer should fix it before the stale comment misleads someone extending this recipe further.
| # 0.683 maps the B200 TP4 SimpleCPU point to a 1,024 GB engine-level CPU | ||
| # KV budget. TP2 uses 1.0 so 2 of 8 GPUs receive the 750 GB policy maximum. |
There was a problem hiding this comment.
🟡 The new comment (line 7465) and perf-changelog.yaml entry both say the TP2/C24 offload arm gets a "750 GB" policy maximum, but agentic_dram_offload_gb() actually computes 749 GB (int-truncated) for tp=2, dram-utilization=1.0, gpus-per-node=8 with the DRAM cap applied — so TOTAL_CPU_DRAM_GB the benchmark script receives is 749, not 750. This is a 1 GB documentation overstatement with no runtime effect; worth a one-line fix for consistency (the sibling TP4 comment correctly documents its truncated value, 1,024 GB).
Extended reasoning...
agentic_dram_offload_gb() in utils/matrix_logic/generate_sweep_configs.py computes the CPU DRAM offload budget as proportional_bytes = Decimal(available_mib) * BYTES_PER_MIB * utilization * gpu_count / gpus_per_node, then returns int(proportional_bytes / BYTES_PER_GB) with BYTES_PER_GB = 1_000_000_000 — i.e. it truncates to whole GB using decimal (not binary) gigabytes.
For the new TP2/C24 arm added in this PR (dram-utilization: 1.0, tp: 2 → gpu_count=2, gpus_per_node=8, cluster b200-dgxc), available_mib clamps to MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB = 2_861_022 (the file's own comment states b200/b300 report more DRAM than this cap). Plugging in:
proportional_bytes = 2_861_022 * 1_048_576 * 1.0 * 2 / 8 = 749_999_751_168
int(749_999_751_168 / 1_000_000_000) = 749
So the TOTAL_CPU_DRAM_GB environment variable that benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh actually receives for this arm is 749, not 750 — and after this PR's own fix to use decimal bytes (CPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000))), the script will request exactly 749 * 1000^3 bytes of CPU KV cache, not the 750 GB the comment and changelog claim.
Both new lines describe this arm as hitting the "750 GB policy maximum":
configs/nvidia-master.yaml:7465: "TP2 uses 1.0 so 2 of 8 GPUs receive the 750 GB policy maximum."perf-changelog.yaml: "...TP2 C24 with the proportional-policy 750 GB SimpleCPU KV-offload cap."
Neither is derived from the generator's actual output — 3 TB / 4 = 750 GB is the nominal fraction, but the real cap (2_861_022 MiB = 2_999_999_004_672 bytes) is fractionally below 3 TB, so integer truncation lands one GB short, at 749. Notably, this is a self-inconsistency within the same comment block: the sibling TP4 line ("0.683 maps the B200 TP4 SimpleCPU point to a 1,024 GB engine-level CPU KV budget") correctly documents the truncated value — int(1_024_499_660_095 / 1_000_000_000) = 1024 matches exactly — so the author clearly intended to document the generator's true output, and 750 vs. 749 for TP2 is simply an arithmetic slip, not an intentional rounding convention.
Impact is purely cosmetic: the generator still computes and passes the correct 749 GB value to the benchmark script at runtime, so nothing breaks and no config drifts. It's worth a one-line fix (change "750 GB" to "749 GB" in both the yaml comment and the changelog description) for accuracy and to keep the changelog trustworthy as a record of what was actually run, but it does not block merging.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31833401868 |
|
/reuse-sweep-run |
Ankur-singh
changed the title
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:Scope. This PR refreshes the Validation and eval evidence. Run 31833401868 ran on the exact PR head Speculative decoding — acceptance length and the eval guard. The submission runs EAGLE3 with the GQA draft head Single-node recipe publication — satisfied by an already-published recipe. This config is Model and scenario scope. MODELS.md lists MiniMax-M3 as active for Agentic coding, with Single-turn 1k1k and 8k1k recorded as deprecated (removed 2026-08-04, #2493). This submission is agentic-coding only. MODELS.md records the engine expectation for this model as the native/upstream vLLM engine with No engine or serving-stack patching. The diff is two YAML files. No Append-only item — not applicable. This submission does not set Recorded exception — the config comment above this entry is now stale. The commented block immediately above Recorded exception — creator-preflight findings accepted as out of scope for this PR. The creator preflight reports Noted for the author, outside this PR's diff. Mergeability. The PR is currently Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
/stage-results 31833401868 |
|
@cquil11 staged run 31833401868: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-14~r31833401868 This run remains available across future |
Resolve the nvidia-master and append-only performance changelog conflicts while preserving the PR's B200 Pareto sweep changes.\n\n中文：将 main 合并到 MiniMax-M3 B200 刷新分支；解决 nvidia-master 与仅追加性能变更日志的冲突，同时保留本 PR 的 B200 帕累托扫描调整。
5 participants
Description
Refresh the MiniMax-M3 NVFP4 B200 AgentX submission to fifteen EAGLE3-GQA Pareto
points and bump the pinned vLLM nightly.
dram-utilization: 1.0block andsweep concurrency 15/20/25/30/32/34/36/38/40, replacing the former C20/C30/C40 arm
that ran under
dram-utilization: 0.683.vllm/vllm-openai:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9.The offload arm's move from
dram-utilization: 0.683to1.0raises its engine-levelCPU KV budget at TP4 from 1,024 GB to 1,499 GB — the full proportional-policy share of
the 3 TB AgentX ceiling for 4 of 8 GPUs (
utils/matrix_logic/generate_sweep_configs.py,agentic_dram_offload_gb).Acceptance length stays pinned to the committed golden curve: 2.78 from
golden_al_distribution/minimaxm3_eagle3_gqa.yamlthinking_on[3], matchingNUM_SPEC_TOKENS=3and theInferact/MiniMax-M3-EAGLE3-GQAdraft head.Throughput jobs run synthetic acceptance; eval jobs run real verification
(
EVAL_ONLY=trueomitsrejection_sample_method).Scope note: this PR changes
configs/nvidia-master.yamlandperf-changelog.yamlonly. No benchmark script is modified.
中文说明
将 MiniMax-M3 NVFP4 B200 AgentX 提交刷新为十五个 EAGLE3-GQA 帕累托点，并升级所固定的
vLLM nightly 镜像。
dram-utilization: 1.0区块，并将并发扫描扩展为15/20/25/30/32/34/36/38/40，取代原先在
dram-utilization: 0.683下的 C20/C30/C40 分支。vllm/vllm-openai:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9。卸载分支从
dram-utilization: 0.683调整为1.0，使其在 TP4 下的引擎级 CPU KV 预算从1,024 GB 提高到 1,499 GB，即 3 TB AgentX 上限中 8 卡取 4 卡的完整按比例份额
（见
utils/matrix_logic/generate_sweep_configs.py的agentic_dram_offload_gb）。接受长度仍固定为已提交的黄金曲线值：取自
golden_al_distribution/minimaxm3_eagle3_gqa.yaml的thinking_on[3]= 2.78，与
NUM_SPEC_TOKENS=3及Inferact/MiniMax-M3-EAGLE3-GQA草稿头一致。吞吐作业使用合成接受率，评测作业使用真实验证（
EVAL_ONLY=true时不设置rejection_sample_method）。范围说明：本 PR 仅修改
configs/nvidia-master.yaml与perf-changelog.yaml，未改动任何基准测试脚本。
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamlentry added forminimaxm3-fp4-b200-vllm-agentic-mtpminimaxm3_eagle3_gqa.yamlthinking_on[3]= 2.78)models/MiniMaxAI/MiniMax-M3.yaml(b200: verified)