[0xShug0]

Summary

Add configurable DitModule and session-owned DitRuntime, and migrate CosyVoice3, MOSS SoundEffect, Stable Audio 3 Small/Medium, and Dots SOAR/MeanFlow to the shared implementation. Architecture configuration is separate from checkpoint tensor-name mappings.

The shared sampling layer provides CFG/APG guidance and Euler/PingPong updates, while the existing host sampler gains midpoint/RK4 support for Dots. Graph-side sampling removes per-step prediction readback and host CFG from the migrated GGML paths. Conditioning, modulation banks, prefix caches and graph lifetimes are runtime-owned. Stable Audio APG uses the equivalent direct velocity equation.

Existing framework callers retain their defaults. FFN precision policies, packed projections and releasable graph buffers require explicit selection. There are no GGML/kernel, server, WebUI or model-spec changes. Stable Audio Foundation and CosyVoice3's native RKNN path remain unchanged.

Performance

CUDA on RTX 5090, eight CPU threads, matching packages and requests for main versus this branch. Lower RTF is faster; peak VRAM is process-level. Warmup requests are excluded where present.

Model / path	Main RTF	PR RTF	Main peak VRAM (MiB)	PR peak VRAM (MiB)
CosyVoice3	0.06073	0.05949	2,623	2,623
Dots SOAR	0.26629	0.22217	6,723	6,285
Dots MeanFlow	0.13081	0.11104	4,875	4,735
Stable Audio Small	0.01298	0.01231	1,939	1,925
Stable Audio Medium	0.01576	0.01464	3,847	3,847
MOSS, CFG 4	0.6115–0.6428	0.6218–0.6242	8,269	8,247
MOSS, CFG off	0.3421–0.3514	0.3454	8,269	8,247

MOSS uses the full 30-second denoising canvas and 100 sampling steps. Its PR timings fall within the measured main variation; no speedup is claimed for that path.

Stable Audio Small also runs on CPU: CFG-off request time is 3,100 → 2,919 ms, and CFG 4/APG is 5,026 → 4,805 ms. Peak RAM for the paired-case comparison is 1,696 → 1,695 MiB.

Parity

Model / path	Result versus main
CosyVoice3	WER unchanged on checked requests
Dots SOAR	WER unchanged on checked requests
Dots MeanFlow	WER unchanged on checked requests
Stable Audio Medium, CUDA	Byte-identical
Stable Audio Small, CPU, CFG off	Byte-identical
Stable Audio Small, CPU, APG	Waveform cosine 0.98236
MOSS, CUDA, CFG 4	Waveform cosine 0.99996783; maximum sample error 0.00119
MOSS, CUDA, CFG off	Waveform cosine 0.99956924; maximum sample error 0.00308

Arithmetic is normalized to shared framework expressions rather than preserving every original floating-point operation order. WER checks cover Mandarin/Cantonese CosyVoice3 requests, English/Mandarin SOAR requests and English MeanFlow requests. Unchanged WER establishes transcription agreement for those cases, not complete voice or perceptual equivalence.

Validation

• Debug build and framework tests for DiT execution, solver stages, APG equivalence, FFN binding and graph-buffer lifecycle.
• Repeated requests, changed conditioning, CFG switching and Dots cache release/reuse; repeated outputs are deterministic.
• CUDA end-to-end checks across all four families; Vulkan end-to-end checks across all four families; Stable Audio Small CPU generation with CFG off and APG.
• Metal and HIP execution have not been validated for this refactor.