[Harshal Dharpure (hdharpure9922)]

This change fixes an ORT fusion issue where Add + LayerNormalization could be fused into SkipLayerNormalization even when the Add output was still being used elsewhere or was exposed as a graph output. In those cases, the fusion was not safe and could lead to backend load failures, including with OpenVINO.

The fix adds a guard to ensure fusion only happens when the residual Add output can be safely removed. If the output is still needed, the original graph is preserved.

Context

This issue was reported in intel#748 with GPT-like ONNX models failing to load on OpenVINOExecutionProvider when SkipLayerNormalization was present. The root cause was an overly aggressive ORT fusion that generated an invalid SkipLayerNormalization pattern while the residual tensor was still live.

What changed
Skip fusion when the residual Add output has multiple consumers.
Skip fusion when the residual Add output is a graph output.
Added regression tests covering both scenarios.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Harshal Dharpure (hdharpure9922)]

@microsoft-github-policy-service agree company="Intel"

[Copilot]

🟡 Changes recommended

Four biased test cases have incorrect Add counts, and the multiple-consumer condition lacks focused regression coverage.

2 open issues

• Update skipped-fusion fixture Add count expectations · New
• Add regression test for multiple consumers blocking fusion · New

What changed in this PR

This PR makes the Python transformer optimizer preserve Add + LayerNormalization when the residual output is still needed, avoiding multi-output SkipLayerNormalization patterns associated with the reported OpenVINO load failure.

Changes:

• Skip fusion when the residual Add has multiple consumers or produces a graph output.
• Update six existing graph-output tests to expect the unfused pattern.

File	Description
onnxruntime/​test/​python/​transformers/​test_skip_layer_norm_fusion.py	Updates graph-output cases to expect retained Add and LayerNormalization nodes.
onnxruntime/​python/​tools/​transformers/​fusion_skiplayernorm.py	Adds an early return for live residual outputs and removes multi-output fusion handling.

_{Review effort: Balanced}