fix(golden-al): validate curves by draft model / fix(golden-al):按草稿模型校验黄金 AL 曲线 - #2394
fix(golden-al): validate curves by draft model / fix(golden-al):按草稿模型校验黄金 AL 曲线#2394xinli-sw wants to merge 1 commit into
Conversation
| @@ -1,4 +1,4 @@ | |||
| # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28061204145 | |||
| # # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049 | |||
There was a problem hiding this comment.
🟡 This PR bumps the MiniMax-M3 EAGLE3 golden AL source run to 29784780049, but golden_al_distribution/README.md line 114 still links that row to the old run 28061204145, leaving the README and YAML in disagreement about which CI run produced the committed values. Please update the README table's link/run-id to match (the other three rows are kept in sync with their YAML source-run lines, so this is now the sole stale entry).
Extended reasoning...
What the bug is
golden_al_distribution/minimaxm3_eagle3.yaml line 1 records the source GitHub Actions run that produced the committed AL numbers. This PR bumps that run from 28061204145 to 29784780049. However, golden_al_distribution/README.md maintains a "Current golden curves" table (line 114) that separately links each model row to its source run for traceability/audit purposes. That table entry was not updated in this PR and still reads:
| MiniMax-M3 | EAGLE3 | [`minimaxm3_eagle3.yaml`](minimaxm3_eagle3.yaml) | [28061204145](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28061204145) |
Why existing code/process does not prevent it
The README table is manually maintained, not auto-generated from the YAML source-run comment — there is no CI check or generation script that keeps the two in sync. Nothing in the current review/merge process flags a mismatch between a YAML's first-line source run and the README table's "Source run" column.
Proof this is a real, isolated regression
Cross-checking the other three rows in the same table confirms the table is normally kept byte-for-byte in sync with each YAML's first line:
kimik2.5_eagle3.yamlfirst line run =28122195822, README =28122195822✅dsv4_mtp.yamlfirst line run =27180633016, README =27180633016✅qwen3.5_mtp.yamlfirst line run =27317114007, README =27317114007✅minimaxm3_eagle3.yamlfirst line run (after this PR) =29784780049, README =28061204145❌
So this row is now the sole diverging entry, directly caused by this PR changing the YAML without a matching README update.
Impact
This is documentation-only drift — the benchmarking/comparison code reads the YAML directly, not the README table, so no runtime behavior, build, or test is affected. The practical impact is that a reviewer or future engineer following the README's link to "verify the source run" for MiniMax-M3 EAGLE3 will land on the superseded workflow run (28061204145) instead of the run that actually produced the currently-committed values (29784780049), undermining the auditability the README table exists to provide. Note README_zh.md likely carries the same stale table and should be checked too.
Suggested fix
Update README.md (and README_zh.md if present) line 114 to link to 29784780049:
| MiniMax-M3 | EAGLE3 | [`minimaxm3_eagle3.yaml`](minimaxm3_eagle3.yaml) | [29784780049](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049) |
This is a one-line fix and does not need to block merging, since no functional path is affected — but it is worth including in this PR (or a fast follow-up) to keep the traceability table accurate, especially since a prior identical golden-AL update (commit 873a954) also missed syncing this README table, suggesting this is a recurring oversight in this workflow.
| @@ -1,4 +1,4 @@ | |||
| # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28061204145 | |||
| # # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049 | |||
There was a problem hiding this comment.
🟡 Line 1 of golden_al_distribution/minimaxm3_eagle3.yaml now reads # # Source GitHub Actions run: ... with a doubled # prefix, while every other file in golden_al_distribution/ uses a single #. It's a cosmetic copy-paste artifact that doesn't affect YAML parsing or the AL values, but it should be fixed to keep the auditable source-run pointer consistent with the established convention.
Extended reasoning...
This PR updates the golden AL reference values for minimax-m3 after a re-run of the SPEED-Bench workflow against a new EAGLE3 head (Inferact/MiniMax-M3-EAGLE3-GQA), and as part of that update it rewrites the audit-pointer comment on line 1 to point at the new GitHub Actions run URL (.../runs/29784780049). In doing so, the line was changed from:
# Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049to:
# # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049with an extra # prepended before Source. This is almost certainly a mechanical artifact — the automation/script that regenerates this header line appears to have re-prepended a # comment marker onto a line that was already commented, rather than replacing the whole line.
Why this wasn't caught elsewhere: the Source GitHub Actions run line is prepended by a step separate from the main header-generation logic in benchmarks/single_node/speedbench/minimaxm3_fp4_b300_vllm.sh (which emits the Acceptance Length (AL) line onward). Nothing validates that this specific line retains exactly one #, so the doubling passed through unnoticed. I confirmed by checking head -1 across all golden_al_distribution/*.yaml files (dsv4_mtp.yaml, kimik2.5_eagle3.yaml, kimik3_dspark.yaml, kimik3_dspark_probabilistic*.yaml, qwen3.5_mtp.yaml) that every other file uses a single # prefix — minimaxm3_eagle3.yaml is the sole outlier introduced by this PR.
Impact: none functionally. This is a YAML comment; it has zero effect on parsing, and no tooling in the repo consumes this line programmatically (it's purely a human-readable audit pointer, documented in the golden_al_distribution README as the way to trace which CI run produced the pasted values). So the golden AL values themselves are correct and usable as-is.
Proof/repro:
head -1 golden_al_distribution/minimaxm3_eagle3.yamlonmain(pre-PR) →# Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28061204145- Same command post-PR →
# # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049 head -1 golden_al_distribution/*.yamlfor any of the other 5 files → single#prefix in each case, confirming this file alone deviates.
Fix: simply drop the extra # so the line reads # Source GitHub Actions run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29784780049, matching the convention used by every other file in the directory. Since it's purely cosmetic and doesn't affect any consumed value, this doesn't need to block the merge, but it's a quick one-line fix worth making before or shortly after merge to keep the audit trail clean.
Add canonical draft-model metadata and deterministic AL validation. Update the bilingual guidance and CODEOWNER verifier policy. 中文:添加规范化草稿模型元数据和确定性的 AL 校验,并同步更新中英文指南与 CODEOWNER 验证策略。
d8488a0 to
ce4e6f0
Compare
|
replaced by #2413 |
Summary
Inferact/MiniMax-M3-EAGLE3andInferact/MiniMax-M3-EAGLE3-GQA.Validation
uv run --with pytest --with pyyaml python -m pytest utils/test_validate_golden_al.py -v— 6 passed.中文说明
Inferact/MiniMax-M3-EAGLE3与Inferact/MiniMax-M3-EAGLE3-GQA分别保留独立的 MiniMax-M3 黄金 AL 曲线。验证
uv run --with pytest --with pyyaml python -m pytest utils/test_validate_golden_al.py -v— 6 项测试通过。