Track remaining OpenAI-compatible reasoning compatibility work #366

Current delivery status — 2026-11-02

Target: Dify 1.19.0, with SDK publication tracked in #413.
The latest public SDK remains 0.10.2 on both PyPI and GitHub.
#382, #390, #404, #410, and #411 are merged; #412 is a draft follow-up with passing checks.
The package and replay checks below retain their original 2026-10-28 verification date.
Official plugin candidates langgenius/dify-official-plugins#4034 and langgenius/dify-official-plugins#4156 must be reconciled against the final published SDK before delivering the newer reasoning and response behavior.

Background

vLLM renamed Chat Completions reasoning output from reasoning_content to reasoning, then removed the deprecated field (vLLM #27752, #33402).
SDK #295 supports both fields, and #367 preserves reasoning across explicit empty deltas.
Both fixes shipped in SDK 0.10.1 and remain in 0.10.2.

Merged SDK #382 subsequently extended the heartbeat rule: absent, null, and empty reasoning fields keep an active block open until real content, tool/function output, or stream completion closes it.
It also preserves the final finish reason and usage and assembles interleaved tool arguments by provider index.
This supersedes the earlier null/missing-field boundary behavior described in this issue.
SDK #404 adds the shared SSE framing fixes.
These later changes are merged but are not in the public SDK 0.10.2 release.

Verified delivery state — 2026-10-28

The actual Marketplace archives below were downloaded and checked with the official plugin CLI.
Both their uv.lock and exported requirements.txt contain SDK 0.10.2, and their LLM source matches official-plugins 888886e6a.

The original OpenAI-compatible 0.10.2 lock refresh and package publication are complete.
#3740 was closed without merging and is no longer the pending delivery step.
Stepfun's lock refresh landed through #3739.
Non-stream reasoning was handled separately in #3695.

Validation

The two downloaded packages' existing suites passed in separate environments installed from their frozen locks: OpenAI API Compatible 120 tests and Stepfun 93 tests.

Additional acceptance replayed 15 cases per provider through public invoke, real loopback HTTP, and the package's request conversion and response parser.
The historical inputs were the SSE excerpt in langgenius/dify-plugin-sdks#277, with an explicit terminal/usage sequence added because the report is an excerpt, and the complete terminal sequence in vLLM #38499.
Derived controls covered both reasoning field names, empty/null/missing heartbeats, mixed reasoning and answer text, reasoning-only completion, non-stream responses, zero usage, and interleaved parallel tool arguments.
Stepfun's fixed provider destination was redirected to loopback; its parsing and usage implementation ran unchanged.

• Both published packages pass the original reported reasoning sequences.
• Published OpenAI API Compatible passes 14/15 content/tool/usage checks, but its SDK 0.10.2 still joins interleaved parallel arguments incorrectly: the first call remains incomplete and the second receives both continuations.
  Its four null/missing heartbeat controls also retain the old two-block behavior rather than the single-block behavior introduced in fix(model): preserve reasoning streams and final results #382.
• Published Stepfun passes 15/15 checks, and its custom handler already preserves all three heartbeat forms in one block.
• Running the same unchanged provider sources with an installed build of SDK main 3652576f passes all 30 checks, including the updated heartbeat boundaries and parallel tool calls.
  This is an SDK upgrade validation, not a claim that either published archive already contains SDK 0.11.0.

The resulting serialized chunks also passed 30 checks through Graphon a315092e's result accumulator and LLM node.
The 26 streaming cases then passed both Dify Chatflow and Workflow reasoning handlers, for 52 handler checks.
Dify was checked at f64b57d6565; the five relevant runner, pipeline, and event-entity files are identical to fetched main d8c9889616e.
Checks covered final reasoning/text/usage, preserved tool arguments, reasoning-before-answer event order, the node selector, exactly one final reasoning marker, and leaving the answer state untouched during reasoning emission.

The event ownership follows langgenius/graphon#180 and langgenius/dify#37460: plugin output remains <think> compatible, Graphon separates reasoning and emits dedicated events, and Dify forwards reasoning_chunk for accumulation by node.
Live event deltas and normalized final reasoning have intentionally different whitespace handling.

These are controlled HTTP and in-process consumer checks.
They do not establish live LiteLLM/vLLM deployment behavior, daemon deployment compatibility, or browser rendering.
A model endpoint is not required to reproduce or verify the historical wire sequences.

Remaining work

• Merge and release fix(model): support reasoning field in streamed responses #295 and fix(model): preserve reasoning across empty deltas #367.
• Publish OpenAI API Compatible and Stepfun packages containing SDK 0.10.2.
• Verify the published override/custom-handler behavior using historical reasoning sequences.
• Identify fix(model): preserve reasoning streams and final results #382's remaining delivery gap and verify current SDK behavior with both provider sources.
• Verify reasoning accumulation and event emission through the Graphon and Dify consumer methods.
• Publish SDK 0.11.0 through #413, including the merged reasoning/SSE fixes and the reviewed fix(model): validate responses and stabilize token estimates #412 follow-up.
  Release preparation chore(release): prepare v0.11.0 #391 is merged; PyPI and GitHub still expose 0.10.2 as latest.
• Update the OpenAI-compatible dependency/lock to that published release, rebuild the plugin, and verify the actual Marketplace artifact.
  langgenius/dify-official-plugins#4034 contains the upgrade candidate.
  Reconcile it with #4156, which adapts response hooks to fix(model)!: validate completion input and correct fallback usage #411, and refresh the final published lockfiles.
• Record live LiteLLM/vLLM acceptance when a deployment is available; retain this as unverified rather than requiring credentials for the completed offline work.

The verified reasoning path requires no additional daemon, Graphon, or Dify schema change.
Historical image serialization and completion-mode modality handling remain tracked separately in #387.