Skip to content

fix(reflect): cap done tool answers - #2757

Merged
nicoloboschi merged 1 commit into
vectorize-io:mainfrom
Sanjays2402:fix/done-tool-max-tokens
Jul 20, 2026
Merged

fix(reflect): cap done tool answers#2757
nicoloboschi merged 1 commit into
vectorize-io:mainfrom
Sanjays2402:fix/done-tool-max-tokens

Conversation

@Sanjays2402

Copy link
Copy Markdown
Contributor

Fixes #2756.

The normal done tool path now applies the same capped rewrite used by direct-text completions when an answer exceeds max_tokens. The rewrite's token usage and trace entry are included in the final result.

Regression coverage confirms the done-tool answer is uncapped before this change and rewritten with the configured budget afterward.

Apply the configured max_tokens budget when the reflect agent finishes through the done tool. Add a regression test covering the previously uncapped completion path.
@benfrank241

Copy link
Copy Markdown
Member

Thanks @Sanjays2402 — and nice root-cause report on #2756.

I verified this locally: the fix faithfully mirrors the existing short-circuit rewrite path (same prompt, scope, and max_completion_tokens=max_tokens), plumbs max_tokens into _process_done_tool correctly, and threads final_usage + the final_rewrite trace entry through cleanly. test_reflect_agent.py passes 57/57 deterministic tests including the new test_done_tool_answer_respects_max_tokens (the 3 errors are just LLM API key required on the real-LLM classes, which run in the core-LLM CI job).

Routing to @nicoloboschi for the merge call, because this is a behavior decision as much as a bug fix. Today two of the three final-answer paths (forced-final, direct-text short-circuit) enforce the cap; the done path — the common one — does not. Applying the cap here means an extra LLM round-trip + a lossy summarization rewrite on the hot path, turning max_tokens into a hard, rewrite-enforced contract rather than a soft hint. It is scoped to mental-model refresh (the only place max_tokens is populated, per #1076), so it only fires where a user opted into a budget — but whether that budget should be hard-enforced via a second call vs. best-effort is a product call for you.

Fix looks correct and ready if you want it; flagging the design tradeoff so the decision is yours. cc @nicoloboschi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] mental model max_tokens not enforced when reflect agent finishes via the done tool (the common path)

3 participants