fix(llm): force reasoning_effort none for gpt-5.6-terra tool calls and tolerate role-less choices - #1859
Conversation
…d tolerate role-less choices
gpt-5.6-terra rejects function tools combined with any reasoning effort
other than "none", and with reasoning left on it returns a forced
tool-call choice without message.role or finish_reason. The strict
ChatCompletion pydantic model then failed validation ("2 validation
errors for ChatCompletion: choices.0.message.role Field required,
choices.0.finish_reason Field required"), so every LLM-as-judge
evaluation using terra errored out and scored 0%.
Python-side mirror of UiPath/Agents#6020 and UiPath/Agents#5995
(SRE-636507 / SRE-639489):
- UiPathLlmChatService.chat_completions sends reasoning_effort "none"
for gpt-5.6-terra when the request carries tools; tool-less requests
keep the default behavior.
- ChatMessage.role defaults to "assistant" and
ChatCompletionChoice.finish_reason is optional, so a role-less
tool-call choice parses instead of being rejected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the uipath-platform LLM Gateway integration to handle gpt-5.6-terra tool-call quirks that currently break LLM-as-judge evaluations: it forces reasoning_effort: "none" for tool-bearing terra requests and relaxes response parsing to accept tool-call choices missing message.role and finish_reason.
Changes:
- Force
reasoning_effort: "none"only forgpt-5.6-terrarequests that include tools. - Make response models tolerant of terra’s role-less / finish_reason-less tool-call choices (
ChatMessage.roledefault,finish_reasonoptional). - Add integration tests covering the request-body normalization and the relaxed response parsing.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/uipath-platform/tests/services/test_uipath_llm_integration.py | Adds test coverage for terra tool-call reasoning_effort normalization and role/finish_reason tolerant parsing. |
| packages/uipath-platform/src/uipath/platform/chat/llm_gateway.py | Relaxes response schema validation to tolerate role-less messages and missing finish_reason. |
| packages/uipath-platform/src/uipath/platform/chat/_llm_gateway_service.py | Implements conditional reasoning_effort: "none" injection for tool-bearing gpt-5.6-terra requests. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🚨 Heads up:
|
|



Problem
Since the
AvailableLlmModelsdefault flipped togpt-5.6-terra(SRE-636507 in alpha, SRE-639489 in prd), every LLM-as-judge evaluation using terra fails with:Two terra behaviors combine to cause this:
"none"("Function tools with reasoning_effort are not supported for gpt-5.6-terra … set reasoning_effort to 'none'").message.roleand withoutfinish_reason, which the strictChatCompletionpydantic model rejects — so the judge errors out and the eval run scores 0%.The C# backend was already fixed for the same signature in UiPath/Agents#6020 (send
reasoning_effort: nonefor terra tool calls) and UiPath/Agents#5995 (accept role-less tool-call choices). This is the Python SDK mirror for the evaluator path (llm_as_judge_evaluator→UiPathLlmChatService.chat_completions).Changes
UiPathLlmChatService.chat_completions: when the request carries tools and the model isgpt-5.6-terra, sendreasoning_effort: "none"in the normalized request body. Tool-less requests keep the default behavior (mirror of Agents#6020).ChatMessage.roledefaults to"assistant"andChatCompletionChoice.finish_reasonbecomesOptional[str], so a role-less tool-call choice parses instead of being rejected (mirror of Agents#5995).Tests
reasoning_effort: "none"reasoning_effortreasoning_effortassistant, tool call args intactpytest tests/services/test_uipath_llm_integration.py tests/services/test_llm_service.py: all pass excepttest_basic_chat_completions_mocked, which also fails on a cleanorigin/maincheckout in my environment (an EndpointManager discovery call leaks into the mock's call count) — pre-existing, unrelated.🤖 Generated with Claude Code