[0xShug0]

PreparedGraph owns session graph metadata, persistent input buffers, execution plans and backend capture cleanup. It supports independent graphs sharing a GraphWorkspace, full/body entry points sharing one allocation, and single-graph GGML scheduler execution.

Migrate Drama3, Nemotron Transcribe and Ming3 Audio to this lifecycle. Backend placement and expert-staging callbacks remain model-local. Model math, precision, options and generation settings are unchanged.

The GGML scheduler also now propagates a failed allocation reservation instead of continuing with a null buffer. This changes allocation-failure handling only; kernels and successful dispatch remain unchanged.

Validation used Debug builds on RTX 5090, eight threads, sequential warmed A→B→B→A comparisons: 20 configurations, 80 sessions and 416 requests. The baseline uses the original model implementations with the initial PreparedGraph framework API present but unused; the candidate migrates those implementations. These are pre-migration comparisons, not pristine-main benchmarks.

Resident and disk-expert offload paths were tested on CUDA and Vulkan. Drama3 covers default and cuda_perf, streaming and offline generation, different seeds and repeated requests. ASR covers two different long recordings, short requests after long recordings, and repeats; Ming3 speech responses also cover voice switching.

RTF before → after, with observed peak VRAM in MiB in parentheses. A single peak value means before and after matched.

Model / configuration	CUDA	Vulkan
Drama3 default, resident	0.17250 → 0.17190 (23,385)	0.23232 → 0.23291 (23,417)
Drama3 cuda_perf, resident	0.14665 → 0.14651 (23,513)	0.21514 → 0.21523 (23,444)
Drama3 default, offload	0.42492 → 0.42354 (6,649)	0.54886 → 0.54760 (6,552)
Drama3 cuda_perf, offload	0.39894 → 0.39819 (6,729)	0.54504 → 0.54310 (6,580)
Nemotron Transcribe, resident	0.01626 → 0.01624 (22,255)	0.02175 → 0.02173 (21,690)
Nemotron Transcribe, offload	0.05943 → 0.05963 (6,539)	0.05504 → 0.05531 (6,034)
Ming3 Audio ASR, resident	0.01920 → 0.01925 (10,885)	0.03628 → 0.03634 (10,571)
Ming3 Audio ASR, offload	0.03678 → 0.03682 (4,361)	0.04923 → 0.05018 (3,812)
Ming3 Audio speech response, resident	0.18526 → 0.18392 (12,557)	0.30949 → 0.30897 (12,409 → 12,408)
Ming3 Audio speech response, offload	0.24200 → 0.24197 (4,627)	0.56153 → 0.54688 (4,244)

All corresponding WAVs, streamed PCM and non-timing responses match exactly, including ASR transcripts, timestamps, speaker labels and confidence values. Repeated requests and long→short reuse showed no retained-memory increase. No peak-VRAM increase was observed.

Timing changes range from −2.61% to +1.95%. The Ming3 Vulkan offload ASR aggregate includes one slower first long request; its repeat and the second candidate block match baseline timing, so no persistent slowdown was observed.

PreparedGraph unit tests pass on CPU, CUDA and Vulkan, covering shared allocation layouts, changing inputs, workspace growth, entry points, scheduler splits, reset/release and resource ordering. Real allocation-failure and recovery tests pass on CUDA and Vulkan without leaked buffers.