Skip to content

translator: repair Gemini requests ending with a model turn - #2870

Open
christianfeurer wants to merge 1 commit into
theagentrouter:mainfrom
christianfeurer:fix/gemini-trailing-model-turn
Open

christianfeurer wants to merge 1 commit into
theagentrouter:mainfrom
christianfeurer:fix/gemini-trailing-model-turn

Conversation

@christianfeurer

Copy link
Copy Markdown

Description

Gemini (Vertex AI and AI Studio) rejects requests whose last content has the model role with
400 Requests ending with a model turn are not supported.. OpenAI-compatible clients send such
requests in practice, e.g. OpenCode when it resumes an interrupted session (the last message is an
assistant text reply) or after a response was cut off by a content_filter finish reason in the
middle of a tool call. The gateway forwarded these requests unchanged, so every retry failed with
the same 400 and the client session got stuck.

openAIMessagesToGeminiContents now makes sure the contents do not end with a model turn:

  • A trailing assistant message without content (empty model turn) is dropped. Gemini rejects
    contents without parts anyway, and the message carries no information.
  • If the trailing assistant message has tool calls without tool results, a user turn with one
    functionResponse per call is appended, carrying
    {"error": "tool call was not executed: no tool result was provided"}. A plain text turn after a
    function call turn would not be a valid Gemini history, and the error lets the model decide to
    retry the call.
  • Any other trailing model turn is followed by a user turn with the text Please continue., which
    is LiteLLM's default continue message (DEFAULT_USER_CONTINUE_MESSAGE).

Requests that already end with a user or tool turn are unchanged.

The repair is always on rather than behind an option: Gemini has no valid interpretation of a
request ending with a model turn (there is no assistant prefill), so the only alternative to the
repair is a guaranteed 400. LiteLLM puts its continue message behind modify_params because it
applies the same logic across many providers, some of which accept assistant prefill; here the
change is scoped to the Gemini translator only. The same conversion is used by the Vertex
countTokens path, so the token count of such a request now includes the appended turn, which
matches what generateContent will actually receive.

Related Issues/PRs (if applicable)

N/A

Special notes for reviewers (if applicable)

Tests cover an empty trailing assistant message, a trailing assistant text message, trailing
assistant tool calls without tool results, an unchanged normal conversation, and the request path
in openai_gcpvertexai_test.go. Assistant tool calls without results in the middle of a
conversation (followed by a new user message) are not changed by this PR.

🤖 Generated with Claude Code

Gemini rejects requests whose last content has the model role with
400 "Requests ending with a model turn are not supported.".
OpenAI-compatible clients such as OpenCode send such requests when they
resume an interrupted session or retry after a cut-off response, which
leaves the session stuck.

openAIMessagesToGeminiContents now repairs the trailing model turn:
empty trailing assistant messages are dropped, unanswered tool calls get
an error function response, and any other trailing model turn is
followed by a "Please continue." user turn (LiteLLM's default continue
message).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Christian Feurer <3908180+christianfeurer@users.noreply.github.com>
@christianfeurer
christianfeurer requested a review from a team as a code owner October 16, 2026 06:52
@netlify

netlify Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for theagentrouter canceled.

Name Link
🔨 Latest commit 5b3c8c3
🔍 Latest deploy log https://app.netlify.com/projects/theagentrouter/deploys/6ad1c9bb4c72d9e1f727718a

@aabchoo aabchoo added the area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs) label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants