Skip to content

fix(responses): collect compaction from closed items and answer Codex WebSocket prewarms - #536

Open
pStrikeZ wants to merge 2 commits into
Menci:mainfrom
pStrikeZ:fix/codex-ws-compaction-prewarm
Open

pStrikeZ wants to merge 2 commits into
Menci:mainfrom
pStrikeZ:fix/codex-ws-compaction-prewarm

Conversation

@pStrikeZ

Copy link
Copy Markdown
Contributor

Codex remote compaction through a Codex upstream still failed after #530 with OpenAI Responses compact decryption: native compaction returned no compaction output item. The upstream answers the compaction_trigger turn with response.created and response.in_progress carrying an empty output, the compaction item in response.output_item.added, one or more response.compaction.compacting progress events, the finished item in response.output_item.done, and then response.completed whose output is still []. reassembleOpenAIResponsesEvents returned that terminal snapshot verbatim, so decryptNativeCompaction iterated an empty output and never saw the item. #530 widened the item check to compaction_summary, but the upstream sends compaction, and its tests fed the shim a terminal that already listed the items, so the stream shape the upstream actually sends was never exercised.

The reassembler now applies the rule the client-facing egress already follows in wrapOpenAIResponsesObservedOutput: the spec makes the item lifecycle the authority, so once any item has closed, the closed items in output_index order replace the terminal's output, and a turn that closed nothing keeps its terminal snapshot. Codex CLI reads items the same way — its response.completed parser has no output field at all, and remote compaction takes the item from OutputItemDone — so an item stated only in a terminal was never visible to it. This also covers the other Codex terminal the shim already knew about, one that omits an assistant message it just closed, so collectSummaryTurn drops its own copy of that collection and reads the reassembled output. Every caller of collectOpenAIResponsesProtocolEventsToResult — the compact action, the compaction shim, compaction persistence, non-streaming responses, and the request inspector — now sees the items a turn closed rather than what its terminal restated. One Responses Lite fixture from #511 listed a reasoning item only in its terminal; it now streams that item's lifecycle like the call beside it, and still checks that restoration leaves the item untouched.

response.compaction.compacting is now a typed stream event, and the compaction item types the metadata and internal_chat_message_metadata_passthrough fields the upstream sends with it.

With decryption working, the next failure in the same session was a 400 at the start of every new Codex session: {"detail":"Unsupported parameter: generate"}. Codex opens each session with a WebSocket prewarm, a response.create carrying the session's instructions and tools with generate: false; it waits only for response.completed and then continues from that response's id, sending just the items the prewarm did not carry. The gateway forwarded the field to the Codex HTTP backend, which rejects it. Codex only logs the failed prewarm and carries on with a full request, so sessions kept working, but each one recorded a failed turn, and a translated target would have dropped the field and run a full, billed generation instead.

The gateway now answers the prewarm itself, as the outermost OpenAI Responses interceptor, in the way it already answers Claude Code's one-token probes. By then serve preparation has resolved the model, expanded any previous_response_id, and staged the request's input, so the empty completed response commits a snapshot of exactly that input and the next turn's continuation replays the prewarmed instructions ahead of its own items. A model no upstream serves still fails before the interceptor runs. The turn records zero usage and no latency sample, and generate is now a typed request field. Only generate: false is answered; any other value passes through untouched.

Tests cover reassembly from closed items when the terminal is empty or omits a closed item, the compaction shim against a stream shaped like the upstream's, the prewarm interceptor, and a WebSocket session whose prewarm never reaches the upstream while the following turn sends the prewarmed instructions without generate. Both fixes were also confirmed against a live Codex upstream: a WebSocket compaction now completes through both upstream calls, and a new session's prewarm returns 200.

… snapshot

When Codex CLI triggers remote compaction over WebSocket, Floway failed
during native compaction decryption with `OpenAI Responses compact decryption:
native compaction returned no compaction output item`.

The Codex upstream streams `response.created` (output []) →
`response.in_progress` (output []) → `response.output_item.added` (item
`{type: "compaction", ...}`, output_index 0) → two
`response.compaction.compacting` events → `response.output_item.done` (same
item) → `response.completed` with `response.output: []`.

`reassembleOpenAIResponsesEvents` returned the terminal event's response
verbatim and ignored `response.output_item.done`, so
`collectOpenAIResponsesProtocolEventsToResult` yielded an empty output array.
Upstream Menci#530 (7a7b2c6) widened the type guard to
`compaction | compaction_summary`, but its tests fed a pre-filled terminal
`output`, leaving this stream shape uncovered.

The reassembler now follows the rule the client-facing egress already
applies: the spec makes the item lifecycle the authority, so once any item
closes, the closed items in `output_index` order replace the terminal
`output`; a turn that closed nothing keeps the terminal snapshot. That also
covers a Codex terminal that omits an assistant message it just closed, so
`collectSummaryTurn` in `compact-shim.ts` drops its own copy of that
collection. `OpenAIResponsesCompactionItem` types the wire metadata fields,
and `response.compaction.compacting` is recognized as a stream event.

A Responses Lite fixture that listed a reasoning item only in the terminal
now streams its lifecycle like the other items, as Codex CLI itself reads
items only from `response.output_item.done`.
Codex opens every session with a WebSocket prewarm: a `response.create`
carrying the session's instructions and tools with `generate: false`. It
waits only for the terminal `response.completed`, then continues from that
response's id and sends just the items the prewarm did not carry.

Floway forwarded the field verbatim. The Codex HTTP backend rejects it with
`{"detail":"Unsupported parameter: generate"}`, so every new Codex session
recorded a failed 400 turn and lost its prewarm; a translated target would
instead drop the field and run a full, billed generation.

The gateway now answers the prewarm itself as the outermost OpenAI Responses
interceptor, after serve preparation has resolved the model, expanded any
`previous_response_id`, and staged the request's input. The empty completed
response commits a snapshot of exactly that input, so the next turn's
continuation replays the prewarmed instructions ahead of its own items. As
with the Claude Code probe answer, the turn records zero usage and no latency
sample. `generate` is now a typed request field.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant