[feature] Bound upstream response bodies and SSE events

[cpakkamisaac-sae]

Problem

Switchyard bounds inbound request bodies at 32 MiB, but the upstream response paths do not have corresponding limits:

• buffered success responses are collected in full before JSON decoding;
• non-success response bodies are collected in full for the returned error;
• streamed provider responses can accumulate an arbitrarily large SSE frame before JSON decoding.

A misconfigured, faulty, or hostile upstream can therefore make one logical request retain a large wire body plus decoded JSON and normalized response data. Concurrent requests multiply the memory exposure.

Reproduction

Verified against the current runtime code with a loopback upstream:

Case	Upstream payload	Current result
Buffered HTTP 200 JSON response	33 MiB text payload	Fully buffered, decoded, and returned
Buffered HTTP 500 response	33 MiB error body	Fully buffered and retained in the upstream error
Streaming HTTP 200 response	One 33 MiB SSE event	Fully accumulated, decoded, and collected

All three probes completed successfully, confirming there is no response-side or per-event ceiling. The existing 32 MiB inbound request limit remains an effective control case.

Proposed behavior

Add configurable per-client safety limits for:

1. the total buffered successful response body;
2. the diagnostic body retained from a non-success response;
3. one decoded SSE frame/event.

The streaming limit should apply per event, not to total stream bytes, so a long well-formed stream remains valid.

Enforce limits while bytes are being accumulated, before allocating or parsing the complete oversized payload. The error path should retain only a UTF-8-safe prefix and indicate truncation. Oversized success bodies and SSE events should surface a typed, redaction-safe client error that the HTTP server maps consistently.

Compatibility and design points

• Keep the lower-level translation API usable without deployment configuration.
• Decide whether the client limits have safe defaults or preserve unlimited behavior unless configured.
• Keep the limit configurable for providers that legitimately return large media or tool payloads.
• Count decompressed bytes.
• Do not trust Content-Length as the only enforcement mechanism.
• Do not log response contents.
• Preserve streaming cancellation, timeout, and terminal-event behavior.

Acceptance criteria

• A buffered success response over the configured limit fails without collecting the full body.
• A non-success response retains at most the configured diagnostic-body limit and reports truncation.
• A single SSE event over the configured limit yields a typed stream error without accumulating the full event.
• Total stream length is not capped when every individual event is within the limit.
• Exact-boundary payloads succeed; one-byte-over payloads fail.
• Split network chunks and CRLF-delimited SSE frames are covered.
• Invalid zero-valued limits fail configuration validation.
• Configuration, error mapping, metrics/logging behavior, and defaults are documented.
• Existing request translation, response translation, retry, timeout, and streaming tests remain green.

Scope

Primary ownership is in libsy-llm-client, with the SSE frame boundary enforced by switchyard-translation. Runner configuration, protocol error typing, server error mapping, and documentation need small integrations.