Repository navigation
translator: repair Gemini requests ending with a model turn - #2870
Open
christianfeurer wants to merge 1 commit into
Open
christianfeurer wants to merge 1 commit into
christianfeurer wants to merge 1 commit into
Conversation
Gemini rejects requests whose last content has the model role with 400 "Requests ending with a model turn are not supported.". OpenAI-compatible clients such as OpenCode send such requests when they resume an interrupted session or retry after a cut-off response, which leaves the session stuck. openAIMessagesToGeminiContents now repairs the trailing model turn: empty trailing assistant messages are dropped, unanswered tool calls get an error function response, and any other trailing model turn is followed by a "Please continue." user turn (LiteLLM's default continue message). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christian Feurer <3908180+christianfeurer@users.noreply.github.com>
✅ Deploy Preview for theagentrouter canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Gemini (Vertex AI and AI Studio) rejects requests whose last content has the
modelrole with400 Requests ending with a model turn are not supported.. OpenAI-compatible clients send suchrequests in practice, e.g. OpenCode when it resumes an interrupted session (the last message is an
assistant text reply) or after a response was cut off by a
content_filterfinish reason in themiddle of a tool call. The gateway forwarded these requests unchanged, so every retry failed with
the same 400 and the client session got stuck.
openAIMessagesToGeminiContentsnow makes sure the contents do not end with a model turn:contents without parts anyway, and the message carries no information.
functionResponseper call is appended, carrying{"error": "tool call was not executed: no tool result was provided"}. A plain text turn after afunction call turn would not be a valid Gemini history, and the error lets the model decide to
retry the call.
Please continue., whichis LiteLLM's default continue message (
DEFAULT_USER_CONTINUE_MESSAGE).Requests that already end with a user or tool turn are unchanged.
The repair is always on rather than behind an option: Gemini has no valid interpretation of a
request ending with a model turn (there is no assistant prefill), so the only alternative to the
repair is a guaranteed 400. LiteLLM puts its continue message behind
modify_paramsbecause itapplies the same logic across many providers, some of which accept assistant prefill; here the
change is scoped to the Gemini translator only. The same conversion is used by the Vertex
countTokenspath, so the token count of such a request now includes the appended turn, whichmatches what
generateContentwill actually receive.Related Issues/PRs (if applicable)
N/A
Special notes for reviewers (if applicable)
Tests cover an empty trailing assistant message, a trailing assistant text message, trailing
assistant tool calls without tool results, an unchanged normal conversation, and the request path
in
openai_gcpvertexai_test.go. Assistant tool calls without results in the middle of aconversation (followed by a new user message) are not changed by this PR.
🤖 Generated with Claude Code