Repository navigation
[Do not merge] Qwen-3.8 Flash Next runtime on v0.17.0 - #2801
Draft
Wang, Zhenze (zz002) wants to merge 2 commits into
Draft
Wang, Zhenze (zz002) wants to merge 2 commits into
Wang, Zhenze (zz002) wants to merge 2 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
1 of 3 tasks
Generator-path runtime from microsoft#2714 (head b4fdda5), ported onto v0.17.0 without the main commits it builds on: - Engine (src/engine) and CUDA (src/ep/cuda) changes are left out; they depend on the main-only engine and GPU state-replay refactors. - StateUpdateReplayDesc keeps the v0.17.0 raw-pointer layout; the Indexer and Snapshot replay fields used only by the engine are omitted, and RecurrentState fills the descriptor with raw pointers. - Conflicting hunks in embeddings (graph-capture buffers), scheduled_requests, CUDA/WebGPU interfaces and examples keep v0.17.0.
From microsoft#2782: fall back to a growing (concat) indexer cache when the graph has no past_sequence_length input.
Wang, Zhenze (zz002)
added a commit
to ROCm/hip-ep
that referenced
this pull request
Oct 28, 2026
## Summary Replace the 42-PR OGA patch list used for Qwen3.8-Flash-Next with the existing patches plus two: microsoft/onnxruntime-genai#2792 (onnxruntime-extensions pin with `Qwen3_5Tokenizer`) and microsoft/onnxruntime-genai#2801 (Qwen3.8 runtime from #2714 plus the IndexerCache fix #2782, ported onto v0.17.0). CI vehicle stacked on #1335; not for merge. ## Related issue or design Stacked on #1335. ## Why The previous list applied #2713/#2714 together with 35 OGA main PRs they depend on, which moved a large part of OGA past v0.17.0. #2801 ports only the Generator-path runtime onto v0.17.0, so the OGA used by CI stays close to the release. ## What - `windows-deps.yml`, `linux-build.yml`: `OGA_PR_PATCHES` = `2685 2691 2705 2712 2792 2801`. Linux now also gets the extensions pin. ## Test plan - [x] Local: v0.17.0 + `pull/<n>.diff` of 2685 2691 2705 2712 2792 2801 applied with the CI `git apply --3way` command; Windows DML Release build; Qwen3.8-Flash-Next image description correct, 1024-token prefill / decode on par with the previous chain build. - [ ] Windows Build and Linux Build green. - [ ] Jenkins daily_hip_ep_genai on this PR: Qwen3.8 VLM output correct. ## Notes for reviewers The OGA cache key only contains PR numbers; updating #2801 does not invalidate it.
3 of 4 tasks
Wang, Zhenze (zz002)
added a commit
to ROCm/hip-ep
that referenced
this pull request
Oct 28, 2026
## Summary Replace the 42-PR OGA patch list used for Qwen3.8-Flash-Next with the existing patches plus two: microsoft/onnxruntime-genai#2792 (onnxruntime-extensions pin with `Qwen3_5Tokenizer`) and microsoft/onnxruntime-genai#2801 (Qwen3.8 runtime from #2714 plus the IndexerCache fix #2782, ported onto v0.17.0). CI vehicle stacked on #1335; not for merge. ## Related issue or design Stacked on #1335. ## Why The previous list applied #2713/#2714 together with 35 OGA main PRs they depend on, which moved a large part of OGA past v0.17.0. #2801 ports only the Generator-path runtime onto v0.17.0, so the OGA used by CI stays close to the release. ## What - `windows-deps.yml`, `linux-build.yml`: `OGA_PR_PATCHES` = `2685 2691 2705 2712 2792 2801`. Linux now also gets the extensions pin. ## Test plan - [x] Local: v0.17.0 + `pull/<n>.diff` of 2685 2691 2705 2712 2792 2801 applied with the CI `git apply --3way` command; Windows DML Release build; Qwen3.8-Flash-Next image description correct, 1024-token prefill / decode on par with the previous chain build. - [ ] Windows Build and Linux Build green. - [ ] Jenkins daily_hip_ep_genai on this PR: Qwen3.8 VLM output correct. ## Notes for reviewers The OGA cache key only contains PR numbers; updating #2801 does not invalidate it.
Wang, Zhenze (zz002)
added a commit
to ROCm/hip-ep
that referenced
this pull request
Oct 29, 2026
… (#1335) ## Summary Build OGA v0.17.0 with the Qwen3.8-Flash-Next runtime: microsoft/onnxruntime-genai#2801 (the Generator-path runtime of #2714 plus the IndexerCache fix #2782, ported onto v0.17.0) and microsoft/onnxruntime-genai#2792 (onnxruntime-extensions pin with `Qwen3_5Tokenizer`). ## Related issue or design microsoft/onnxruntime-genai#2714, microsoft/onnxruntime-genai#2782, microsoft/onnxruntime-genai#2792, microsoft/onnxruntime-genai#2801 ## Why The pinned OGA rejects the Qwen3.8 `genai_config.json` (`Unknown value "rotary_position_ids"`). #2714 adds that runtime but is based on OGA main after v0.17.0, so its diff does not apply to the tag alone. #2801 carries only the Generator-path part of #2714 (no Engine or CUDA changes) on top of v0.17.0, so no other main PRs are needed. ## What - `OGA_PR_PATCHES` (Windows and Linux): `2685 2691 2705 2712 2792 2801`. - #2792 replaces 2763: it pins onnxruntime-extensions main plus microsoft/onnxruntime-extensions#1136 (the pre-tokenizer fix 2763 carried), which also brings microsoft/onnxruntime-extensions#1142 (`Qwen3_5Tokenizer`). 2763 now merges OGA main into its branch, so its diff no longer applies on v0.17.0. Linux gets the pin too. - #2801 is applied after #2685, which rewrites the same extensions pin line, so the pin stays in its own PR. - Exclude `*.md` from the applied diffs; the build does not read Markdown. - Windows: retry the `.diff` fetches (`-MaximumRetryCount 6`), as Linux already does with curl. ## Test plan - [x] The list applies cleanly on v0.17.0 with the CI `git apply --3way` command (locally) - [x] build-windows / build-linux green with this list (#1351) - [x] Qwen3.8-Flash-Next with the Windows `gpu-test-package`: image description correct; 1024-token prefill / decode on par with the earlier OGA build from #2714 plus its main dependencies - [ ] GPU tests show no regression on the existing OGA models ## Notes for reviewers The OGA cache key only contains PR numbers, so pushing new commits to #2792 or #2801 does not invalidate it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge. Snapshot for downstream testing of Qwen-3.8 Flash Next on the 0.17.0 release; based on
rel-0.17.0sopull/<n>.diffapplies directly onv0.17.0.Contents
b4fdda54), Generator path only, ported onto v0.17.0 without the main commits it builds on:src/engine) and CUDA (src/ep/cuda) changes are left out; they depend on main-only engine and GPU state-replay refactors.StateUpdateReplayDesckeeps the v0.17.0 raw-pointer layout (the Indexer/Snapshot replay fields are engine-only), andRecurrentStatefills it with raw pointers.scheduled_requests, the CUDA/WebGPU interfaces and examples keep the v0.17.0 version.past_sequence_lengthinput.#2713 (model builder) is not included. The onnxruntime-extensions pin with
Qwen3_5Tokenizersupport is #2792 and is applied separately, after #2685.Validation
Applied after #2685, #2691, #2705, #2712 and #2792 on
v0.17.0withgit apply --3way; Windows DML Release build. Qwen-3.8 Flash Next (text + vision): image description correct, prefill / decode throughput on par with the same model built from #2714 plus its main prerequisites. CUDA and WebGPU are not built or tested.