[Akshay Sonawane (apsonawane)]

Main integration update

Merged repository main at 4b34a64af8b763c3bb2d8781b45a26f0c3841118. Published head: f0665f7571eee9fdf8a789c291e4c0d8569441c9. Existing PR bases remain unchanged.

• The exact remote head contains main and all preceding updated stack layers.
• Used the qwen-flash conda environment; ran lintrunner -a before each new commit, followed by clean lint checks. Final branch-vs-main lint passes.
• Published with normal fast-forward pushes, without amendments or force pushes.
• No new benchmark/profiling/evaluation scripts or co-author trailers.
• GenAI validation: 801 builder tests; 26 optimizer tests plus 81 subtests; nine C++ syntax checks; ABI-10 layout and Python AST/CLI checks pass.
• No complete native rebuild or GPU/model requalification has run after integration.

This update supersedes earlier notes saying main has not been integrated, but does not remove the draft/build/runtime qualification requirements.

Summary

Add package constructors and guarded generic-contrib graph rewrites, with 26 model-independent optimizer unit tests and integration documentation.

Stack and main target

Depends on #2755.

This is layer 2/2 of the microsoft/onnxruntime-genai stack. The current review base is asonawane/ported-qwen-graph-runtime so this PR shows only its incremental group. The final destination is upstream main: land/reconcile prerequisites first, then rebase and retarget this layer to main. Keep this PR in draft until that base-specific validation succeeds. Do not merge the entire stack through an unvalidated intermediate base.

Cross-repository dependency: GenAI package/runtime integration requires the companion ORT contrib operators.

Validation and blockers

• python -m unittest discover -s tools/qwen38_graph -p test_optimize_ported_package.py: 26 tests pass.
• Tool source matches the existing qualified workspace; no benchmark or profiling scripts are included.
• Package construction/integration requires the runtime and companion ORT operator stack.

Scope

• Preserve the original dirty worktrees and existing PR branches.
• No benchmark runners, profiling utilities, evaluation drivers, model weights or local task metadata in the new commits.
• No Copilot co-author trailers in the new commit messages.
• Broad compatibility with other models is not established by local Qwen qualification.

Stack navigation

• Add Qwen CUDA graph execution through Generator
• Add generic operator package rewrites and integration tests

Companion repository stack:

• Add exact normalization and rotary CUDA operator modes
• Add fused hyper-connection and tensor CUDA operators
• Extend GDN and causal convolution fused state handling
• Extend sparse attention caches and optimize packed indexing
• Optimize quantized MoE and block-scaled FP8 CUDA projections
• Add fused MTP MoE and speculative state fold CUDA operators
• Document generic CUDA operators and complete build compatibility

[Akshay Sonawane (apsonawane)]

Closing this draft to rebuild the changes from current main rather than the divergent feature branch. Existing branches are preserved; replacement PRs will be linked here.

[Akshay Sonawane (apsonawane)]

Clean replacement opened directly against main: #2759. The old branch is preserved. Both replacements are independent main-based PRs; end-to-end package execution still needs the companion runtime and ORT operators.