add B300 vLLM AgentX single-node MiniMax-M3 FP4 EAGLE3-GQA MTP / 新增 B300 vLLM AgentX 单节点 MiniMax-M3 FP4 EAGLE3-GQA MTP by hshrivastava-droid · Pull Request #2328 · SemiAnalysisAI/InferenceX

hshrivastava-droid · 2026-07-24T19:01:17Z

Day-zero single-node agentic-coding recipe for MiniMax-M3 (NVFP4, MoE) on B300 with vLLM and EAGLE3-GQA speculative decoding. GPU-resident KV only — no DRAM offload arm.

Serve flags

FlashInfer TRT-LLM attention (use_trtllm_attention=true) with FP8 indexer KV
EAGLE3 drafter on FLASH_ATTN
--enable-prefix-caching, --block-size 128, --gpu-memory-utilization 0.9
--max-cudagraph-capture-size 512, --max-num-batched-tokens 16384, --stream-interval 20
--reasoning-parser minimax_m3, --default-chat-template-kwargs '{"thinking_mode":"enabled"}'
--all2all-backend flashinfer_nvlink_one_sided

Search space

Five configs, GPU-resident KV only:

TP	conc
8	1
4	1, 2, 16
2	2

dram-utilization: 0.80 is set in the matrix but only applies to DRAM-offload arms; all points here have kv-offloading: none.

Eval logic

Added EVAL_ONLY branch: when EVAL_ONLY=true the script calls run_eval --port $PORT instead of the benchmark replay path, so lm-eval can be run against a live server without re-running the full agentic sweep.

中文说明

MiniMax-M3（NVFP4，MoE）在 B300 单节点上的首日 agentic-coding 配方，使用 vLLM 搭配 EAGLE3-GQA 投机解码，仅含 GPU 驻留 KV 配置（无 DRAM 卸载分支）。

镜像：vllm/vllm-openai:nightly-387189c42997b27e2c04b5d97ef8190ffa2bf909

草稿模型：Inferact/MiniMax-M3-EAGLE3-GQA，投机 token 数 3，合成拒绝采样，thinking-on 合成接受长度 2.83（来自 MiniMax-M3 EAGLE3 AL 分布，action 28061204145）。

共 5 个配置：TP8 并发 1；TP4 并发 1、2、16；TP2 并发 2。新增 EVAL_ONLY 分支，支持对运行中的服务端单独执行 lm-eval 评估。

github-actions · 2026-07-24T19:01:26Z

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

github-actions · 2026-07-24T19:01:27Z

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

github-actions · 2026-07-24T19:01:27Z

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

github-actions · 2026-07-24T19:41:40Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30120171478
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30120171478

github-actions · 2026-07-24T21:05:06Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30121331788
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30121331788

github-actions · 2026-07-27T23:28:02Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30309054681
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30309054681

github-actions · 2026-07-28T03:55:30Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30326099097
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30326099097

github-actions · 2026-07-28T05:59:39Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30326099097
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30326099097

github-actions · 2026-07-28T18:06:43Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30375457395
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30375457395

github-actions · 2026-07-28T19:51:19Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30375457395
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30375457395

…AL for eval Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

… minimaxm3-fp4-b300-vllm-agentic-mtp 条目 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-07-28T21:33:22Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30394805725
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30394805725

github-actions · 2026-07-30T19:07:02Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30566516423
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30566516423

github-actions · 2026-07-30T23:26:47Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30566516423
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30566516423

github-actions · 2026-07-31T10:27:52Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30566516423
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30566516423

中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf PR #31，并同步当前 main。

github-actions · 2026-07-31T19:24:12Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30655420064
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30655420064

github-actions · 2026-07-31T19:36:53Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30655420064
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30655420064

…r profiling handoff) 中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf ed05782（全局锚定性能分析切换点） Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-07-31T23:44:40Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30666322654
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30666322654

github-actions · 2026-08-01T01:22:58Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30673796617
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30673796617

xinli-sw · 2026-08-01T02:37:29Z

/stage-results 30673796617

github-actions · 2026-08-01T02:37:42Z

@xinli-sw staged run 30673796617: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-31~r30673796617

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30673796617 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-31~r30673796617

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

xinli-sw · 2026-08-01T02:37:53Z

/stage-results 30514549263

github-actions · 2026-08-01T02:38:04Z

@xinli-sw staged run 30514549263: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30514549263

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30514549263 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30514549263

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

xinli-sw · 2026-08-01T02:47:34Z

/stage-results 30506325008

github-actions · 2026-08-01T02:47:51Z

@xinli-sw staged run 30506325008: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30506325008

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30506325008 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-07-30~r30506325008

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

…hdogs active across barriers) 中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf abf55f9（跨 barrier 保持空闲看门狗活跃） Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-08-02T04:39:41Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30730035219
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30730035219

xinli-sw · 2026-08-02T04:40:42Z

/stage-results 30730035219

github-actions · 2026-08-02T04:41:02Z

@xinli-sw staged run 30730035219: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30730035219

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30730035219 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30730035219

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

…DRAM arms TP4 GPU-resident: [1,2,5,10,15,20]; TP2 GPU-resident: [1,2,5,10,15,20]; TP4 DRAM offload: [20,30,40,50,60] 中文：扩展 MiniMax-M3 B300 MTP AgentX 扫描空间：TP4 GPU 驻留 [1,2,5,10,15,20]；TP2 GPU 驻留 [1,2,5,10,15,20]；TP4 DRAM 卸载 [20,30,40,50,60] Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-08-02T08:11:54Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30733110281
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30733110281

xinli-sw · 2026-08-02T13:44:52Z

/stage-results 30733110281

github-actions · 2026-08-02T13:45:04Z

@xinli-sw staged run 30733110281: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30733110281

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30733110281 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30733110281

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

…offload TP2: [1,2,5] (drop c10,c15,c20); TP4 DRAM: [30,40,50,60,80,90,100] (drop c20, add c80/90/100) 中文：精简扫描空间——TP2 缩减至 [1,2,5]，TP4 DRAM 卸载调整为 [30,40,50,60,80,90,100] Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-08-02T17:05:04Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30751989616
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30751989616

xinli-sw · 2026-08-02T17:38:11Z

/stage-results 30751989616

github-actions · 2026-08-02T17:38:23Z

@xinli-sw staged run 30751989616: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30751989616

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@xinli-sw 已将运行 30751989616 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-02~r30751989616

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

hshrivastava-droid requested a review from a team July 24, 2026 19:01

hshrivastava-droid requested review from Ankur-singh and kedarpotdar-nv as code owners July 24, 2026 19:01

github-project-automation Bot added this to InferenceMAX Board Jul 24, 2026

hshrivastava-droid added a commit that referenced this pull request Jul 24, 2026

Update perf-changelog pr-link for #2328

347d2f5

hshrivastava-droid added the full-sweep-enabled label Jul 24, 2026

claude Bot reviewed Jul 24, 2026

View reviewed changes

Comment thread benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh

hshrivastava-droid added full-sweep-fail-fast and removed full-sweep-enabled labels Jul 24, 2026

hshrivastava-droid force-pushed the minimaxm3-fp4-b300-vllm-agentic-mtp branch from df796e1 to a71fd2b Compare July 27, 2026 21:57

faradawn mentioned this pull request Jul 28, 2026

MiniMax-M3 spec decoding Blackwell vllm-project/recipes#694

Open

xinli-sw changed the title ~~add b300 vllm agentX single-node minimaxm3 fp4 EAGLE-GQA MTP~~ add B300 vLLM AgentX single-node MiniMax-M3 FP4 EAGLE3-GQA MTP / 新增 B300 vLLM AgentX 单节点 MiniMax-M3 FP4 EAGLE3-GQA MTP Jul 28, 2026

hshrivastava-droid and others added 3 commits July 28, 2026 16:00

add b300 vllm agentX single-node minimaxm3 fp4 EAGLE-GQA MTP

4a91df9

update eval logic

6ad6ee9

feat: add tool-call-parser + enable-auto-tool-choice; drop synthetic …

3e84e61

…AL for eval Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

xinli-sw force-pushed the minimaxm3-fp4-b300-vllm-agentic-mtp branch from f52ea00 to 3e84e61 Compare July 28, 2026 20:00

xinli-sw and others added 2 commits July 28, 2026 16:02

perf-changelog: add trailing newline / 末尾添加空行

3f57bd6

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

perf-changelog: append minimaxm3-fp4-b300-vllm-agentic-mtp entry / 追加…

31ec006

… minimaxm3-fp4-b300-vllm-agentic-mtp 条目 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

update config

df08354

SemiAnalysisAI deleted a comment from xinli-sw Jul 30, 2026

chore(agentx): pin MiniMax-M3 sweep to AIPerf PR 31

5e785d0

中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf PR #31，并同步当前 main。

chore(agentx): pin MiniMax-M3 sweep to AIPerf ed05782 (globally ancho…

01dc27e

…r profiling handoff) 中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf ed05782（全局锚定性能分析切换点） Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

xinli-sw added full-sweep-enabled and removed full-sweep-fail-fast labels Jul 31, 2026

chore(agentx): pin MiniMax-M3 sweep to AIPerf abf55f9 (keep idle watc…

2af3fbc

…hdogs active across barriers) 中文：将 MiniMax-M3 AgentX 扫描固定到 AIPerf abf55f9（跨 barrier 保持空闲看门狗活跃） Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Uh oh!

Conversation

hshrivastava-droid commented Jul 24, 2026 • edited by xinli-sw Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contents

Serve flags

Search space

Eval logic

中文说明

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 27, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

xinli-sw commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xinli-sw commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xinli-sw commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 2, 2026

Uh oh!

xinli-sw commented Aug 2, 2026

Uh oh!

github-actions Bot commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 2, 2026

Uh oh!

xinli-sw commented Aug 2, 2026

Uh oh!

github-actions Bot commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 2, 2026

Uh oh!

xinli-sw commented Aug 2, 2026

Uh oh!

hshrivastava-droid commented Jul 24, 2026 •

edited by xinli-sw

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 2, 2026 •

edited

Loading

github-actions Bot commented Aug 2, 2026 •

edited

Loading

github-actions Bot commented Aug 2, 2026 •

edited

Loading