Top-10 improvements: thinking/signature replay, Gemini fixes, retry policy, request controls, embeddings

[jkyberneees]

Fixes the bugs and fills the feature gaps from the top-10 audit, plus everything found by four independent adversarial reviewers. Every behavior change was test-first: the test failed, then the fix landed. One commit per area so it can be reviewed in order.

Numbers: unit-suite coverage went from 92.2% to 99.0%. The -race suite runs in about 35s, down from about 108s, because ListModels no longer retries parse errors with real backoff. golangci-lint reports 0 issues, and go vet -tags e2e plus the Windows and darwin builds are clean. I have not run the live e2e suite.

Bugs fixed (top-10 audit)

1. Anthropic thinking requests would 400.
   • max_tokens is now always larger than budget_tokens. An unset MaxTokens becomes budget + 8192. A preset larger than half an explicit MaxTokens is clamped to half of it, never below 1024. An explicit budget that cannot fit returns a ConfigError.
   • temperature and top_p are omitted while thinking is on.
2. Gemini thoughtSignature was dropped. It is now kept per function call (ToolCall.Signature) and on text parts, and sent back on the same parts. Gemini 3 rejects tool turns that lack it.
3. Anthropic multi-block and redacted thinking was lost.
   • The new ChatResult.ThinkingBlocks / Message.ThinkingBlocks keep every thinking and redacted_thinking block in order, each with its own signature.
   • Streamed signatures are no longer concatenated across blocks.
   • An empty-text signed block still sends the "thinking" key.
4. Gemini finish reasons were wrong.
   • STOP plus function calls now maps to tool_calls.
   • A stream that ends without any completion signal is now a premature close on every format; Gemini is no longer exempt.
   • A prompt blocked by promptFeedback finishes with content_filter instead of being retried 8 times.
5. Delta.ToolIndex was inconsistent. It is now the call's position in ChatResult.ToolCalls on every format. Before, Anthropic used the content-block index and Responses used output_index.
6. No overall budget on buffered calls, and no retry configuration.
   • WithRetryPolicy(RetryPolicy{MaxAttempts, MaxBackoff}) configures the ladder.
   • The request timeout is now the budget for the whole call, retries included, on every entry point.
   • The three copied retry loops (chat, TTS, STT) are now one shared helper, withRetry.
7. Global mutable settings. WithStreamIdleTimeout and WithLearnObserver set these per SDK. SetStreamIdleTimeout is now atomic, so calling it while streams run is race-free.
8. Usage meant different things per provider.
   • Gemini cached tokens are now reported, and Gemini thinking tokens count in CompletionTokens.
   • A DeepSeek cache miss stays in PromptTokens; it is no longer counted as a cache write.
   • A gateway that reports both Anthropic and OpenAI cache fields is not double-counted.
   • Moonshot's top-level cached_tokens is understood.
   • New Usage.InputTokens() and TotalTokens().

• Also fixed:
  • The Gemini model id is path-escaped in the URL.
  • A Responses incomplete turn with reason content_filter maps to content_filter.
  • The Responses stream error event is handled.
  • A request that cannot be built is no longer retried 8 times.

Features added

• Request controls:
  • ToolChoice (auto / none / required / a named tool), ParallelToolCalls, Seed.
  • ResponseFormat for json_object / json_schema. OpenAI uses response_format or text.format; Gemini uses responseMimeType + responseJsonSchema; Anthropic uses a forced tool whose input is folded back into Content, both buffered and streamed.
  • Everything is validated before any network call.
• Passthrough: WithHeaders, applied last on every request (an empty value removes a header), and ChatRequest.Extra for top-level body fields. The keys the SDK validates are reserved.
• SDK.Embed: OpenAI-compatible /embeddings and Gemini batchEmbedContents. Results come back in input order, with batching and count/index checks.
• Tool results: Message.IsError, and image Parts on tool results, mapped per format.
• Replay helper: ChatResult.AssistantMessage() builds the assistant turn with every replay field.
• ListModels:
  • Concurrent cache misses share one upstream fetch, and a panic in that fetch can never block later calls.
  • A malformed body is not retried.
  • A listing longer than 100 pages returns ErrModelListTruncated instead of being silently truncated.

Security hardening (from review)

• Redirects to another host are no longer followed. Go strips only Authorization across hosts, so x-api-key and custom headers would otherwise leak.
• Provider and ProviderConfig now hide the API key and header values under %v, %+v and %#v.
• CR/LF in STT Filename/MIMEType is rejected, to stop header injection.

Behavior changes to review

• Buffered timeout: buffered Call, Speak, Transcribe and Embed now stop at the request timeout in total (default 120s). Before, each of up to 8 attempts got the full timeout.
• DeepSeek usage: a cache miss is now in PromptTokens, not CacheCreationTokens. This is intentional but differs from the old odek-parity test, which I updated.
• Gemini streams: a stream with no finishReason is now an error, as on every other format.
• DeepSeek JSON: Quirks.NoJSONSchema is set for DeepSeek, so a json_schema response format fails fast. That DeepSeek supports only json_object comes from provider docs and has not been checked against the live API.
• New struct fields were added; Thinking on the internal Anthropic block is now a pointer. No exported field was removed.

Review process

Four adversarial reviewers each worked in an isolated worktree: three Sonnet (streaming and thinking, retry and concurrency, request controls and wire formats) and one Haiku (invariants and security). Every finding they proved was fixed test-first in 4b60ce2 or bd73bba.

Not acted on, and documented instead:

• Signatures are provider-specific, so don't replay them across providers.
• Gemini 2.x rejects JSON mode combined with tools.
• Extra merges only top-level keys (shallow).
• The TTS audio/mpeg MIME fallback predates this PR; left as is.

The new surface and invariants 8–10 are documented in README.md and AGENTS.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N76SRxxjJ4FgZGWmv4sGPS

---

Generated by Claude Code

[jkyberneees]

Superseded by #15: same commits, with the branch renamed to fix/top-10-improvements to follow the repo's fix/ convention.

---

Generated by Claude Code