fix: propagate configured temperature through reflect calls - #3828
fix: propagate configured temperature through reflect calls#3828koriyoshi2041 wants to merge 1 commit into
Conversation
Strix Security ReviewNo security issues found. Updated for Reviewed by Strix |
Sanderhoff-alt
left a comment
There was a problem hiding this comment.
The threading itself is complete (all four reflect call sites plus the pre-existing think path), but the tool-loop call reaches a provider path that never had to handle a temperature before, and the resolved default is not None.
| tools=tools, | ||
| scope="reflect_tool_call", | ||
| tool_choice=iter_tool_choice, | ||
| temperature=get_config().llm_temperature_reflect, |
There was a problem hiding this comment.
agent.py:874 is the only in-repo caller of call_with_tools, so this is the first time a temperature reaches that path — and OpenAICompatibleLLM.call_with_tools does not apply the reasoning-model suppression its own call() does (providers/openai_compatible_llm.py:930: if temperature is not None and not is_reasoning_model, versus :1363 where the guard is absent). OpenAIResponsesLLM has it on both paths (:456, :574).
Concrete break: provider=openai, model=gpt-5 (or o1/o3) routes to OpenAICompatibleLLM; with no config set the reflect tool loop now sends temperature=0.9, which chat/completions rejects with a 400 ('temperature' does not support 0.9 with this model), so every reflect fails after retries. Before this change no temperature was sent and it worked.
Suggested fix in the provider, to keep the two methods symmetric:
if temperature is not None and not self._supports_reasoning_model():
...
call_params["temperature"] = temperature| response_format=DynamicModel, | ||
| scope="reflect_structured", | ||
| strict_schema=get_config().llm_strict_schema_reflect, | ||
| temperature=get_config().llm_temperature_reflect, |
There was a problem hiding this comment.
llm_temperature_reflect resolves to DEFAULT_LLM_TEMPERATURE_REFLECT = 0.9 when unset (config.py:222, _resolve_operation_temperature), not None — so contrary to the risk note in the description, every deployment that never set the env var changes behaviour here, and this particular call is the JSON-schema extraction of the final answer, which previously ran at the provider default. Please either confirm 0.9 is intended for structured extraction or use a deterministic value for this call, and correct the description so reviewers of the release notes are not misled about the blast radius.
Problem
HINDSIGHT_API_LLM_TEMPERATURE_REFLECTis resolved by configuration but reflect requests omit it, so providers apply their defaults. This affects tool turns, final synthesis, structured extraction, and over-budget rewrites.Closes #3825.
Fix
Forward
llm_temperature_reflectthrough every reflect LLM call path. Add deterministic coverage for the tool loop, rewrite, and structured-output paths.Test
uv run pytest tests/test_reflect_agent.py::TestReflectStructuredOutput::test_structured_output_forwards_reflect_temperature tests/test_reflect_agent.py::TestReflectAgentMocked::test_done_tool_answer_respects_max_tokens -q(2 passed)uv run ruff check hindsight_api/engine/reflect/agent.py tests/test_reflect_agent.pyuv run ruff format --check hindsight_api/engine/reflect/agent.py tests/test_reflect_agent.pygit diff --checkThe full reflect-agent file also passed 45 tests; four environment-dependent tests could not start because the local checkout lacks
pg0-embeddedand provider credentials. The repository-wide lint wrapper reached unrelated TypeScript linting and stopped because workspace npm dependencies were not installed.Risk
Low. The provider interfaces already accept optional temperature values, and the default remains
None, preserving existing provider-default behavior when the setting is unset.