Skip to content

A streamed tool call without an index starts a new call when it brings an id - #1190

Merged
WaylandYang merged 2 commits into
deeplethe:devfrom
gedaiu:fix/parallel-tool-calls-without-index
Oct 11, 2026
Merged

WaylandYang merged 2 commits into
deeplethe:devfrom
gedaiu:fix/parallel-tool-calls-without-index

Conversation

@gedaiu

@gedaiu gedaiu commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

What happened

With #1183 in place, a question against Gemini (gemini-3.8-flash, OpenAI-compatible endpoint) still failed on its first turn. The log line from #1189 says why:

UnknownToolCall: model attempted to call unknown or disallowed tool `find_entitiessearch_chunks`

Gemini had made two parallel calls. Its stream sends them one per chunk, with no index on either and an id on each:

data: {"choices":[{"delta":{"tool_calls":[{"function":{"arguments":"{\"name\":\"lore\"}","name":"find_entities"},"id":"call_266956","type":"function"}]},"index":0}]}
data: {"choices":[{"delta":{"tool_calls":[{"function":{"arguments":"{\"query\":\"…\"}","name":"search_chunks"},"id":"call_266959","type":"function"}]},"index":0}]}

The streaming parser keyed each delta by index, reading a missing one as 0, so both calls merged into slot 0 and the names concatenated.

What changed

tool_call_slot decides where a delta goes: a numbered one where it says; an unnumbered one with an id starts a new call; an unnumbered one without an id continues the last call, which is the only shape an unnumbered continuation can have (OpenAI's continuation deltas carry argument fragments and no id). Numbered streams behave exactly as before.

Tests: the Gemini two-chunk shape yields two calls with their own ids, names and arguments; an unnumbered, id-less fragment appends to the previous call.

cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test -p utopia-llm pass (49).

Related: #1135, #1183, #1189.

…s an id

Gemini's OpenAI-compatible endpoint streams parallel tool calls one per
chunk, none numbered: no index, an id each, the arguments whole. Keyed
by a missing index read as 0, two calls merged into one, and the model
was refused for calling `find_entitiessearch_chunks`.

A delta that names an index goes there. One that does not and carries
an id is a new call; one with neither continues the last, which is the
only shape an unnumbered continuation has.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for following the Gemini path to the end. The cause is clear from the two frames you quote, and the rule reads correctly: a numbered delta goes where it says, an unnumbered one with an id starts a call, an unnumbered one without continues the last. Checked here: 53 utopia-llm tests pass, and parallel_tool_calls_without_an_index_stay_separate fails without the change. One note, not blocking: the continuation test passes without the change too, since with a single call both rules pick slot 0; a second call ahead of the fragment would make it bite. Merging now. Thanks again.

@WaylandYang
WaylandYang merged commit da2c45e into deeplethe:dev Oct 11, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants