[0xShug0]

MOSS SoundEffect now uses a shared device denoiser runtime with session-owned graphs and buffers. Graph-scoped normalization/affine, bias/activation and gated-residual fusions reduce warmed denoiser time by 7.8% on CUDA and 9.1% on Vulkan compared with the same runtime using ordinary kernels.

Changes

• Add DeviceDenoiserRuntime and DeviceFlowSamplerRuntime for cached execution and device-resident CFG/Euler sampling. Each denoiser runtime owns one multi-graph PreparedGraph, which manages execution plans and shared scratch.
• Keep one active MOSS denoiser configuration: reuse it while CFG batch is unchanged, and release/rebuild it when switching between batch 1 and 2. Conditioning remains independently cached.
• Extend AdaptiveLayerNormModule with prepared modulation parameters and LayerNorm-affine modulation, reusing the existing normalization and tensor modules.
• Add explicitly registered, per-graph CUDA/Vulkan pointwise fusions. Existing attention and GEMM implementations are retained. Unregistered graphs use ordinary dispatch; backends without the extension use the ordinary graph.
• Enable the three fusions by default for MOSS SoundEffect. Set the session option moss_soundeffect.fused_epilogues=false to disable them while retaining the device runtime.
• Preserve native BF16 projection operands in the MOSS Vulkan graph. Also advertise existing Vulkan BF16-to-F32 and F16/BF16 copy conversions. This copy-support expansion is backend-wide and does not depend on fusion registration.

Performance

RTX 5090, fully GPU-resident BF16 package, eight threads, 50 Euler steps, CFG 4, sigma shift 5 and a full 30-second denoising canvas. Each configuration used one unrelated warmup followed by three measured 10-second outputs, covering English rain/thunder and Chinese forest/birds prompts with fixed seeds and a repeated request. Denoiser time excludes text encoding and VAE decoding; RTF covers the complete request. Peak VRAM is the observed process peak during warmed requests.

Backend	Configuration	Denoiser ms	RTF	Peak VRAM MiB
CUDA	Main	3,313	0.3391	8,267
CUDA	PR, fusions off	3,317	0.3395	8,267
CUDA	PR default	3,059	0.3137	8,267
Vulkan	Main	9,584	0.9670	8,428
Vulkan	PR, fusions off	6,063	0.6141	8,320
Vulkan	PR default	5,511	0.5589	8,272

Vulkan's main-to-PR gain also includes native BF16 projection handling. The fusions-off/default comparison isolates the three pointwise fusions.

Validation

• Debug server and device denoiser runtime test targets build successfully.
• CUDA measured WAVs are byte-identical to main with the fusions both enabled and disabled.
• Vulkan measured waveform cosine is at least 0.99973 across main/PR and fusion on/off comparisons; the maximum absolute sample difference is 0.01444.
• Repeated requests, changed conditioning and CFG 4→1→4→1→4 reuse pass on CUDA and Vulkan. Returning to a configuration reproduces the same WAV byte-for-byte; measured outputs also match the pre-refactor implementation, including CFG 1.
• Graph preparation occurs only initially and when the CFG batch changes. Vulkan peak VRAM remains unchanged across switching; CUDA settles 2 MiB above its initial CFG-4 peak, with no further growth observed.
• Model-spec validation passes. Metal execution was reviewed for ordinary-path selection but was not run on hardware.