[AIMIGRAPHX-1163] Add lower_device_ops pass by eddieliao · Pull Request #5030 · ROCm/AMDMIGraphX

eddieliao · 2026-07-01T23:58:36Z

Motivation

Adds a pass to lower hip::copy and hip::fill instructions before compile_ops pass.

Technical Details

If MIGRAPHX_GPU_LOWER_HIP_OPS is set, a lower_hip_ops pass will run before compile_ops to replace any hip::copy and hip::fill instructions with a precompiled op.

Example IR below after compiling using reducesum and MIGRAPHX_SPLIT_REDUCE_SIZE=1.

w/o lowering:

@1 = hip::hip_allocate_memory[shape=int8_type, {288}, {1},id=main:scratch] -> int8_type, {288}, {1}
@2 = load[offset=0,end=288](@1) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
@3 = hip::fill[value=0](@2) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
x = @param:x -> float_type, {3, 4, 5, 6}, {120, 30, 6, 1}
@5 = gpu::code_object[code_object=5952,symbol_name=reduce_sum_kernel,global=2880,local=64,](x,@3) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
main:#output_0 = @param:main:#output_0 -> float_type, {3, 4, 6}, {24, 6, 1}
@7 = squeeze[axes={2}](@5) -> float_type, {3, 4, 6}, {24, 6, 1}
@8 = hip::copy(@7,main:#output_0) -> float_type, {3, 4, 6}, {24, 6, 1}
@9 = @return(@8)

w/ lowering:

@0 = check_context::migraphx::gpu::context -> float_type, {}, {}
@1 = hip::hip_allocate_memory[shape=int8_type, {288}, {1},id=main:scratch] -> int8_type, {288}, {1}
@2 = load[offset=0,end=288](@1) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
@3 = gpu::code_object[code_object=5360,symbol_name=hip_fill_kernel,global=72,local=1024,](@2) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
x = @param:x -> float_type, {3, 4, 5, 6}, {120, 30, 6, 1}
@5 = gpu::code_object[code_object=5952,symbol_name=reduce_sum_kernel,global=2880,local=64,](x,@3) -> float_type, {3, 4, 1, 6}, {24, 6, 6, 1}
main:#output_0 = @param:main:#output_0 -> float_type, {3, 4, 6}, {24, 6, 1}
@7 = squeeze[axes={2}](@5) -> float_type, {3, 4, 6}, {24, 6, 1}
@8 = gpu::code_object[code_object=5680,symbol_name=hip_copy_kernel,global=36,local=1024,](@7,main:#output_0) -> float_type, {3, 4, 6}, {24, 6, 1}
@9 = @return(@8)

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

- Added: New functionality.
- Changed: Changes to existing functionality.
- Removed: Functionality or support that has been removed. (Compared to a previous release)
- Optimized: Component performance that has been optimized or improved.
- Resolved Issues: Known issues from a previous version that have been resolved.
- Not Applicable: This PR is not to be included in the changelog.

Copilot

Pull request overview

Adds an optional GPU pipeline pass (gpu::lower_hip_ops) to lower hip::fill / hip::copy into gpu::precompile_op (and thus HIP code objects) before compile_ops, gated by MIGRAPHX_GPU_LOWER_HIP_OPS. This is intended to avoid leaving hip::* memory ops in the IR at the point where compilation happens.

Changes:

Introduces gpu::lower_hip_ops pass and wires it into the GPU target pipeline behind MIGRAPHX_GPU_LOWER_HIP_OPS.
Extends GPU JIT fill compilation with a dedicated hip::fill compiler producing hip_fill_kernel.
Adds GPU tests (including tuple + dynamic-shape behavior) and documents the new environment variable.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/targets/gpu/lower_hip_ops.cpp`	Implements matcher-based lowering of `hip::fill`/`hip::copy` into `gpu::precompile_op`, including tuple handling.
`src/targets/gpu/include/migraphx/gpu/lower_hip_ops.hpp`	Declares the new `gpu::lower_hip_ops` pass.
`src/targets/gpu/target.cpp`	Adds env-var gating and inserts the pass before `compile_ops` in the GPU pipeline.
`src/targets/gpu/jit/fill.cpp`	Factors common fill-kernel compilation and adds a `hip::fill` JIT compiler path.
`src/targets/gpu/CMakeLists.txt`	Adds `lower_hip_ops.cpp` to the GPU target build.
`test/gpu/lower_hip_ops.cpp`	Adds unit + end-to-end tests for lowering behavior (static, dynamic, tuple) and kernel execution.
`docs/reference/MIGraphX-dev-env-vars.rst`	Documents `MIGRAPHX_GPU_LOWER_HIP_OPS`.

codecov · 2026-07-02T02:01:43Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #5030   +/-   ##
========================================
  Coverage    92.89%   92.89%           
========================================
  Files          603      603           
  Lines        32448    32448           
========================================
  Hits         30140    30140           
  Misses        2308     2308

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

gh-app-migraphx-bot-pr-write · 2026-07-02T03:40:24Z

Test	Batch	New Rate (3c7162)	Old Rate (55ca93)*	Diff	Status
torchvision-resnet50	64	1,327.62	3,169.20	-58.11%	🔴
torchvision-resnet50_fp16	64	2,759.95	6,721.88	-58.94%	🔴
torchvision-densenet121	32	787.62	2,712.99	-70.97%	🔴
torchvision-densenet121_fp16	32	3,794.53	4,578.90	-17.13%	🔴
torchvision-inceptionv3	32	1,596.85	1,805.02	-11.53%	🔴
torchvision-inceptionv3_fp16	32	2,530.26	2,855.31	-11.38%	🔴
cadene-inceptionv4	16	269.04	824.13	-67.35%	🔴
cadene-resnext64x4	16	167.08	785.46	-78.73%	🔴
slim-mobilenet	64	5,954.65	8,444.46	-29.48%	🔴
slim-nasnetalarge	64	200.71	229.50	-12.55%	🔴
slim-resnet50v2	64	1,507.55	3,185.35	-52.67%	🔴
bert-mrpc-onnx	8	368.68	1,173.56	-68.58%	🔴
bert-mrpc-tf	1	463.09	484.20	-4.36%	✅
pytorch-examples-wlang-gru	1	479.55	472.27	1.54%	✅
pytorch-examples-wlang-lstm	1	382.94	386.76	-0.99%	✅
torchvision-resnet50_1	1	674.74	754.58	-10.58%	🔴
cadene-dpn92_1	1	446.53	444.58	0.44%	✅
cadene-resnext101_1	1	348.83	366.32	-4.77%	✅
onnx-taau-downsample	1	397.31	402.99	-1.41%	✅
dlrm-criteoterabyte	1	26.74	32.58	-17.94%	🔴
dlrm-criteoterabyte_fp16	1	45.87	52.65	-12.87%	🔴
agentmodel	1	9,648.13	7,784.58	23.94%	🔆
unet_fp16	2	45.99	57.48	-19.99%	🔴
resnet50v1_fp16	1	875.14	932.89	-6.19%	🔴
resnet50v1_int8	1	742.93	934.74	-20.52%	🔴
bert_base_cased_fp16	64	935.64	1,105.09	-15.33%	🔴
bert_large_uncased_fp16	32	341.10	347.51	-1.84%	✅
bert_large_fp16	1	205.13	205.61	-0.23%	✅
distilgpt2_fp16	16	2,092.03	2,104.89	-0.61%	✅
yolov5s	1	556.44	559.63	-0.57%	✅
tinyllama	1	45.42	46.13	-1.54%	✅
vicuna-fastchat	1	44.03	44.16	-0.28%	✅
whisper-tiny-encoder	1	411.04	415.45	-1.06%	✅
whisper-tiny-decoder	1	410.27	412.20	-0.47%	✅
llama2_7b	1	6.76	21.00	-67.79%	🔴
qwen1.5-7b	1	23.23	23.73	-2.10%	✅
phi3-3.8b	1	26.61	26.83	-0.83%	✅
llama3-8b	1	19.21	21.87	-12.18%	🔴
whisper-large-encoder	1	10.00	10.23	-2.30%	✅
whisper-large-decoder	1	109.92	104.89	4.79%	✅
mistral-7b	1	23.60	23.89	-1.24%	✅
FLUX.1-schnell	1	759.30	760.54	-0.16%	✅

Regressions detected 🔴

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

gh-app-migraphx-bot-pr-write · 2026-07-02T03:40:25Z

Test	Status	Result
bert-mrpc-onnx	✅	PASSED: MIGraphX meets tolerance
bert-mrpc-tf	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-gru	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm	✅	PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte	✅	PASSED: MIGraphX meets tolerance
agentmodel	✅	PASSED: MIGraphX meets tolerance
unet	✅	PASSED: MIGraphX meets tolerance
resnet50v1	✅	PASSED: MIGraphX meets tolerance
bert_base_cased_fp16	✅	PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
bert_large	✅	PASSED: MIGraphX meets tolerance
yolov5s	✅	PASSED: MIGraphX meets tolerance
tinyllama	✅	PASSED: MIGraphX meets tolerance
vicuna-fastchat	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-decoder	✅	PASSED: MIGraphX meets tolerance
distilgpt2_fp16	✅	PASSED: MIGraphX meets tolerance
llama2_7b	✅	PASSED: MIGraphX meets tolerance
qwen1.5-7b	✅	PASSED: MIGraphX meets tolerance
phi3-3.8b	✅	PASSED: MIGraphX meets tolerance
llama3-8b	✅	PASSED: MIGraphX meets tolerance
whisper-large-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-large-decoder	✅	PASSED: MIGraphX meets tolerance
mistral-7b	✅	PASSED: MIGraphX meets tolerance
FLUX.1-schnell	✅	PASSED: MIGraphX meets tolerance

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

TedThemistokleous

Some initial comments

TedThemistokleous

Thanks for answering the questions. No other issues/concerns with this. Approved

pfultz2

The find_contiguous_copy needs to also be removed from fuse_ops since this pass should handle it.

Add lower_hip_ops pass

f9f99e9

eddieliao self-assigned this Jul 1, 2026

eddieliao added enhancement New feature or request Matchers Updates or adds a change to compile time Matchers Changelog: Added New functionality. labels Jul 1, 2026

eddieliao requested a review from Copilot July 2, 2026 00:04

Copilot started reviewing on behalf of eddieliao July 2, 2026 00:05 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread src/targets/gpu/lower_device_ops.cpp Outdated

Comment thread test/gpu/lower_hip_ops.cpp Outdated

eddieliao and others added 2 commits July 6, 2026 16:34

Fix tidy, address copilot comments

4eed669

Merge branch 'develop' into lower_fill_copy

740af86

eddieliao marked this pull request as ready for review July 6, 2026 17:37

eddieliao requested review from a team and causten as code owners July 6, 2026 17:37

Fix tidy

bebfa74

eddieliao requested review from pfultz2 and shivadbhavsar July 6, 2026 18:10

eddieliao and others added 2 commits July 6, 2026 11:14

Merge branch 'develop' into lower_fill_copy

8281292

Update CHANGELOG.md

64fdfab

rainayera requested a review from Copilot July 7, 2026 20:10

eddieliao requested a review from TedThemistokleous July 7, 2026 20:10

eddieliao added the high priority A PR with high priority for review and merging. label Jul 7, 2026

Copilot started reviewing on behalf of rainayera July 7, 2026 20:12 View session

Copilot AI reviewed Jul 7, 2026

View reviewed changes

Comment thread CHANGELOG.md Outdated

Comment thread src/targets/gpu/lower_device_ops.cpp Outdated

TedThemistokleous reviewed Jul 7, 2026

View reviewed changes

Comment thread src/targets/gpu/lower_device_ops.cpp Outdated

Comment thread src/targets/gpu/lower_hip_ops.cpp Outdated

Move assert before empty check

9f179f7

eddieliao requested a review from TedThemistokleous July 8, 2026 00:09

TedThemistokleous approved these changes Jul 8, 2026

View reviewed changes

Merge branch 'develop' into lower_fill_copy

fa119fa

pfultz2 requested changes Jul 8, 2026

View reviewed changes

Comment thread src/targets/gpu/target.cpp Outdated

Comment thread src/targets/gpu/lower_hip_ops.cpp Outdated

Comment thread src/targets/gpu/jit/fill.cpp Outdated

Comment thread src/targets/gpu/include/migraphx/gpu/lower_hip_ops.hpp Outdated

Address reviews

77d08cf

eddieliao requested a review from pfultz2 July 8, 2026 22:55

eddieliao changed the title ~~[AIMIGRAPHX-1163] Add lower_hip_ops pass~~ [AIMIGRAPHX-1163] Add lower_device_ops pass Jul 9, 2026

Merge branch 'develop' into lower_fill_copy

9cba7b2

eddieliao mentioned this pull request Jul 15, 2026

[AIMIGRAPHX-1164] Replace device ops inserted during compile_ops #5066

Merged

pfultz2 reviewed Jul 15, 2026

View reviewed changes

Comment thread src/targets/gpu/lower_device_ops.cpp Outdated

pfultz2 requested changes Jul 15, 2026

View reviewed changes

Comment thread src/targets/gpu/lower_device_ops.cpp Outdated

pfultz2 reviewed Jul 15, 2026

View reviewed changes

Comment thread test/gpu/lower_device_ops.cpp Outdated

Remove sub-shape handler

ee727ee

eddieliao requested a review from pfultz2 July 15, 2026 15:51

Support tuple inputs for fill kernel

3c7162f

pfultz2 approved these changes Jul 16, 2026

View reviewed changes

causten merged commit 6ab7aea into develop Jul 16, 2026
39 checks passed

causten deleted the lower_fill_copy branch July 16, 2026 18:46

Uh oh!

Conversation

eddieliao commented Jul 1, 2026

Motivation

Technical Details

Changelog Category

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

TedThemistokleous left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TedThemistokleous left a comment

Choose a reason for hiding this comment

Uh oh!

pfultz2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Jul 2, 2026 •

edited

Loading

gh-app-migraphx-bot-pr-write Bot commented Jul 2, 2026 •

edited

Loading

gh-app-migraphx-bot-pr-write Bot commented Jul 2, 2026 •

edited

Loading