Repository navigation
fix: reasoning opt-in for OpenAI-format gateways + learn-once observability - #13
Merged
Merged
Conversation
…bility
OpenAI-format gateways (OpenRouter, LiteLLM, vLLM) can return reasoning in
shapes the SDK never read: the documented request opt-in (OpenRouter legacy
include_reasoning, equivalent to reasoning: {}) was never sent, and response
reasoning arriving as message/delta.reasoning or the typed
message/delta.reasoning_details array was silently dropped. Only the
LiteLLM-standardized reasoning_content was understood.
- Quirks.IncludeReasoning sends include_reasoning: true on OpenAI-format
requests; off by default (strict endpoints reject unknown parameters).
- Buffered and streaming parsers now accept reasoning_content, reasoning,
and reasoning_details (text/summary folded in order, encrypted skipped),
precedence in that order.
- SetLearnObserver exposes the previously silent learn-once fallbacks
(buffered downgrade, /responses retry, effort pinning, stream_options
drop): one callback per engagement with kind, provider, status, and the
provider message. The permanent stream-to-buffered downgrade is now
observable instead of invisible.
Review findings from the adversarial PR panel: - Gateways in the wild send reasoning/reasoning_content as JSON objects or arrays (e.g. OpenRouter reasoning objects). String-typed fields made the whole response or stream chunk fail to parse and abort the request. Both response-side fields now decode through a tolerant string type: JSON strings parse as before, every other shape (null, object, array, number, bool) is skipped instead of failing the request. This is a strict robustness improvement for shapes the SDK previously rejected outright. - The streaming quirk test now asserts stream:true on the captured body, so a regression that routes CallStream through the buffered builder fails. - Simplified a redundant fold condition (i > 0 && b.Len() > 0). Known follow-up (documented, not in this PR): a gateway that rejects the include_reasoning parameter outright has no learn-down path yet, unlike stream_options. The flag is opt-in per provider, and the 400 surfaces visibly, so the failure mode is loud rather than silent.
Tag-gated (//go:build e2e), OPENROUTER_API_KEY from env or .env, never logged. Covers the PR #13 contract end to end against the live API: - control-buffered: no-quirk call must succeed (reasoning presence is provider-side, logged not asserted) - optin-buffered: quirk on, request succeeds, and ReasoningContent must mirror the documented fold of the raw wire response (wire-mirror assertion via a capturing transport — plaintext, encrypted-only, or object-shaped reasoning are all provider-side shapes) - optin-stream: three attempts; every attempt must complete without a parse abort (the non-string reasoning regression class) with content flowing and ReasoningContent == concatenated reasoning deltas; plaintext reasoning deltas are probed across attempts because OpenRouter toggles them per request (observed: 74-delta run and zero-delta runs of the identical request within minutes)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes reasoning visibility through OpenAI-format gateways (OpenRouter, LiteLLM, vLLM).
What was wrong
message/delta.reasoning(OpenRouter string alias) or the typedmessage/delta.reasoning_detailsarray was silently dropped; onlyreasoning_contentwas parsed.What changed
Quirks.IncludeReasoning— sendsinclude_reasoning: true(OpenRouter-documented legacy form ofreasoning: {}) on OpenAI-format requests. Off by default.reasoning_content,reasoning, andreasoning_details(text/summary folded in order, encrypted entries skipped; precedence in that order).SetLearnObserver(func(LearnEvent))— one callback per engaged learn-once fallback (buffered,responses,none_effort,drop_stream_options) with provider, HTTP status, and provider message. Nil by default; no behavior change.Verification
go vet ./...,go test -race -count=1 .,golangci-lint run ./...all clean