feat(model-runtime): expose incremental LLM result accumulation #354

Delivery status — 2026-11-02

Implementation is complete through merged #355; this issue remains closed.
This dependency is tracked for the Dify 1.19.0 Agent replay rollout in langgenius/dify#41092 / #44502.
Refresh the host candidate and record the selected production Graphon dependency before rollout.

Problem

Host agent loops need to display streamed model output while retaining the complete assistant message for subsequent tool rounds and persisted history.
Graphon already implements the required aggregation in the private _LLMChunkAccumulator, but its public normalize_non_stream_runtime_result() helper consumes the entire iterator before returning.
Hosts currently have to use a private class, buffer the stream, or duplicate Graphon's content, tool-call, and provider-state aggregation rules.

This is a follow-up to #260 and #261, which added provider-state preservation.
The remaining requirement is a public incremental interface to that existing implementation.
The Dify FC, CoT, and Agent v2 consumers are being integrated in langgenius/dify#44502.

Expected behavior

Expose the existing accumulator as LLMResultAccumulator so a host can call consume(chunk) while forwarding each chunk, then call to_result(model=..., prompt_messages=...) to obtain the complete result.
Each invocation owns its accumulator.
The shared aggregation must retain content order, tool-call arguments, content-block state, and the last non-None assistant opaque_body snapshot.

Acceptance criteria

• Reuse the existing aggregation implementation and keep non-stream normalization behavior intact.
• Document the public entry point and its per-invocation lifetime.
• Verify that mixed content, an empty signed content block, tool calls, and false-valued provider state survive JSON serialization through the public API.

Related work

• Provider round-trip state (opaque_body) is dropped on assistant prompt messages and LLM results #260 and feat: preserve opaque_body provider state on prompt messages and LLM results #261: provider-state preservation, completed.
• fix(agent): preserve complete model messages dify#44502: downstream agent message replay integration.