[Bug] Gateway exits with code 1 in an auto-compaction retry loop: the explicit thinking opt-out is dropped and the summary output budget is too small

[lijia838-source]

[Bug] Gateway exits with code 1 in an auto-compaction retry loop: the explicit thinking opt-out is dropped and the summary output budget is too small

Summary

Two coupled defects in the context compactor cause a self-sustaining compaction retry loop
that ends in an unhandled rejection and Gateway exited (exit code=1).

1. resolveThinkingPlan() silently discards the compactor's explicit
   thinking: { enabled: false } for every model configured with thinking.state: 'default',
   so on endpoints that think by default the model spends the entire summary output budget
   on reasoning and returns finish_reason: "length" with empty content.
2. COMPACT_MAX_OUTPUT_TOKENS = 4_000 is the effective output budget of the normal compaction
   path and is too small to express a summary of a large transcript, even with thinking off.

Either defect alone produces Summary model output was truncated at the token limit. The failure
path then re-triggers compaction immediately while keeping the whole transcript, so the context
grows without bound until the gateway dies.

Environment

Item	Value
Host	PilotDeck desktop, Windows 11, packaged runtime (resources/runtime/dist, ESM modules)
Provider	deepseek — protocol: openai, https://api.deepseek.com/v1
Model	deepseek-flash
Model thinking config	thinking: { state: default, efforts: [], format: provider }
Context budget	maxContextTokens: 95232, warningRatio: 0.8, blockingRatio: 0.9
Telemetry	telemetry.enabled: true (during diagnosis it had been false, see "Silent failure")

Impact (measured from one day of runtime.log, 3,545+ lines)

Metric	Value
Summary model output was truncated at the token limit	46
full_compaction_started / full_compaction_no_summary	67 / 59 (88% of attempts produced no summary)
Gateway exited code=1	14 (8 crashes inside the compaction window + 6 immediate ~60 ms respawn failures)
Peak preTokens	665,361 — 7.0× the 95,232 cap
Peak context tokens in a budget snapshot	799,094 — 8.4× the cap
Peak compaction attempts per minute	3

The user-visible symptom is the gateway restarting repeatedly during long sessions; the
compaction itself is invisible because it only logs to the runtime log.

Root cause

Defect 1 — an explicit per-request opt-out is dropped

CompactionEngine.summarize() deliberately asks for thinking to be off, so that the entire
output budget is available for the summary text (dist/src/context/compaction/CompactionEngine.js):

const request = {
    ...
    maxOutputTokens: maxOutputTokens ?? this.options.maxOutputTokens ?? COMPACT_MAX_OUTPUT_TOKENS,
    stream: true,
    thinking: { enabled: false },
    ...
};

For a model whose configured state is default, resolveThinkingPlan() returns early and the
request-level preference is lost (dist/src/model/thinking/registry.js):

export function resolveThinkingPlan(requestThinking, provider, model) {
    const settings = model.thinking;
    const state = settings?.state ?? 'default';
    // Model state always wins over stale conversation/agent preferences.
    if (state === 'default')
        return { mode: 'default', enabled: false };      // <-- no bodyPatch, no effort
    ...

The returned plan carries neither bodyPatch nor effort, so
dist/src/model/providers/openai/request.js:44 adds nothing to the request body:

if (thinkingPlan.useOpenAIReasoning && thinkingPlan.effort) {
    body.reasoning_effort = thinkingPlan.effort;
}
else if (thinkingPlan.bodyPatch) {
    Object.assign(body, thinkingPlan.bodyPatch);
}

The endpoint then applies its server-side default, which for DeepSeek is thinking on.

Measured against the real endpoint with the compactor's own system prompt and a 24,000-character
high-entropy transcript (probe_reasoning_off_field.py, max_tokens=600):

Request body	finish_reason	reasoning_tokens	summary chars
(no thinking field — what the current code sends)	length	600 / 600	0
reasoning_effort: "none"	length	none	2013
thinking: {"type":"disabled"}	length	none	2069
both	length	none	1986

The control row is the bug: 100% of the output budget is consumed by reasoning and the
summary content is empty. The last three rows confirm both parameter spellings are honoured
by this endpoint, and that with thinking suppressed the model produces real content
(finish_reason: length there is only because that probe used max_tokens=600).

Defect 2 — the summary output budget cannot hold a real summary

The normal compaction path passes no maxOutputTokens budget:

• dist/src/context/DefaultContextRuntime.js:381 (trigger: "auto") omits it;
• the only construction site, dist/src/cli/createLocalGateway.js:1035, does not set
  options.maxOutputTokens;
• therefore summarize() falls back to COMPACT_MAX_OUTPUT_TOKENS = 4_000, whose own comment
  explains it was sized to "keep room for the summary input inside small configured contexts":

// Keep room for the summary input inside small configured contexts such as
// agent.maxContextTokens: 20_000.
export const COMPACT_MAX_OUTPUT_TOKENS = 4_000;

That reasoning ignores how long a summary of a large, high-entropy transcript actually is.
Budget sweep on a real 85k-token prompt, 3 trials per configuration:

Configuration	Trials	Verdict
thinking on, max_tokens=4000 (control)	1/3 truncated	unreliable
thinking off, max_tokens=6000	1/3 truncated	unreliable
thinking off, max_tokens=12000	0/3	reliable
thinking off, max_tokens=16000	0/3	reliable
thinking off, max_tokens=24000	0/3	reliable

Two things worth noting: the fix needs both halves — raising the budget alone is not enough,
because with thinking on the model simply fills the larger budget (max_tokens=16000 produced
reasoning=16000); and a larger budget does not make summaries longer, because at ≥12000 the
model stops naturally (completion 667–1869 tokens).

Why it escalates into a crash loop

• The failure path still appends the summary boundary marker while keeping every message, so
  the context keeps growing and compaction re-triggers as soon as the next turn is over.
• COMPACT_SUMMARY_FAILURE_COOLDOWN_MS = 60_000 rate-limits the retries but does not stop them.
• The resulting rejection is never caught, so dist/src/cli/pilotdeck.js:372
  (unhandledRejection) calls shutdownAndExit(1) → the reported exit code=1.

Silent failure (aggravating factor)

With telemetry.enabled: false, collector.js:20-22 returns before recording anything, so
trackError is a no-op and the crashing condition leaves no trace at all. Diagnosing this
required re-enabling telemetry.

Proposed fix

Attached: pilotdeck-fix-thinking-and-budget.patch (4 hunks, 2 files).

Note: the patch is written against the compiled dist/ JavaScript that ships inside the
installer, because that is what could be inspected and validated on the affected machine.
The equivalent change should be made in the TypeScript sources
(thinking/registry.ts, context/compaction/CompactionEngine.ts).

Part 1 — honour an explicit per-request opt-out when the model state is default, rendered
per endpoint parameter format. Endpoints whose format cannot express "disabled" keep the previous
no-op behaviour (no regression), and always-thinking models do not receive a fake switch so that
throwIfUnsupportedThinkingPlan() cannot throw:

Measured plan rendering after the patch:

Provider	Format	Emitted on the wire
deepseek	thinking-type	thinking: {"type":"disabled"} (verified honoured)
openai	openai	reasoning_effort: "none" (verified honoured)
anthropic	anthropic	thinkingType: "disabled"
openrouter	openrouter	reasoning: {"enabled": false}
dashscope	qwen-cloud	enable_thinking: false
google	google	(no emit — no-op, as before)
ollama	server-default	(no emit — no-op, as before)
moonshot kimi-k3*, zhipu glm-5.3*	thinking-type	(no emit — always-thinking)

Part 2 — give the summary a workable budget while preserving the original intent for small
contexts:

export const COMPACT_MAX_OUTPUT_TOKENS = 16_000;
export const COMPACT_OUTPUT_RESERVE_CONTEXT_RATIO = 0.4;

const requestedSummaryOutputReserve = positiveTokenCount(input.maxOutputTokens)
    ?? positiveTokenCount(this.options.maxOutputTokens)
    ?? COMPACT_MAX_OUTPUT_TOKENS;
const summaryContextCap = positiveTokenCount(input.effectiveContextTokens);
const summaryOutputReserve = summaryContextCap === undefined
    ? requestedSummaryOutputReserve
    : Math.min(requestedSummaryOutputReserve, Math.floor(summaryContextCap * COMPACT_OUTPUT_RESERVE_CONTEXT_RATIO));

and pass summaryOutputReserve (instead of the always-undefined input.maxOutputTokens) to
summarize(), so the reserve that trims the input and the budget sent to the model agree.

Verification performed

Check	Result
verify_openai_wire.mjs — real resolveThinkingPlan + buildOpenAIRequest chain	5/5
↳ compactor body	{"max_tokens":16000,"thinking":{"type":"disabled"}}
↳ ordinary agent turn	{"max_tokens":32768} — no thinking field, reasoning preserved
verify_compaction_budget.mjs — real run() with an injected model	7/7
↳ realistic context (95232)	16000
↳ small context (20000)	8000 (clamped)
↳ emergency path (explicit 1536)	1536 preserved
↳ unknown context	16000
Provider matrix (9 providers)	no regression outside the intended opt-out
Patch applies cleanly and reproduces the patched files byte-for-byte	yes

Other observations (separate, lower severity)

• No busy_timeout on the memory SQLite database. memory/workspaces/*/control.sqlite runs
  in WAL mode but the runtime never sets a busy timeout (default 0 ms, i.e. no retry), and both
  the gateway and the WebUI server process open the same database file. enqueueMaintenanceTask
  only serialises within a single process. Cross-process write contention therefore fails
  immediately instead of waiting. Adding PRAGMA busy_timeout would help.
• calibrationActualInputTokens outliers. Values in the tens of millions appear where a
  per-request input count is expected (up to ~689× the plausible value), which looks like
  cumulative usage being recorded as a single request's input.

patch (4 hunks, 2 files — against the shipped dist/ JavaScript)

# Fix: gateway exit(1) caused by the auto-compaction thinking/budget bug
# Apply from the repository root with: git apply pilotdeck-fix-thinking-and-budget.patch
#
# Part 1: honour an explicit per-request thinking opt-out (the compactor's
#         thinking:{enabled:false} was silently dropped for models configured
#         with thinking.state=default, so reasoning consumed the whole budget).
# Part 2: give the summary enough output budget, clamped to the effective
#         context so small contexts still fit their input.

--- a/dist/src/model/thinking/registry.js
+++ b/dist/src/model/thinking/registry.js
@@ -21,12 +21,47 @@
     };
     return preset[provider.id] ?? 'openai';
 }
+/**
+ * Per-request opt-out for models whose configured thinking state is `default`, which
+ * normally emits no thinking parameter at all. The context compactor sets
+ * thinking.enabled = false so its entire output budget is available for the summary
+ * text; without an explicit switch, an endpoint that thinks by default (e.g. DeepSeek)
+ * spends that whole budget on reasoning and the summary comes back truncated or empty.
+ * Returns undefined when the endpoint's parameter format cannot express "disabled", so
+ * models that cannot disable thinking keep the previous no-op behavior.
+ */
+function thinkingDisablePlan(requestThinking, provider, settings, model) {
+    if (requestThinking?.enabled !== false)
+        return undefined;
+    const format = resolveFormat(provider, settings?.format ?? 'provider');
+    const plan = { mode: 'off', enabled: false };
+    if (format === 'openai')
+        return { ...plan, effort: 'none', useOpenAIReasoning: true };
+    if (format === 'anthropic')
+        return { ...plan, thinkingType: 'disabled' };
+    if (format === 'qwen-cloud')
+        return { ...plan, bodyPatch: { enable_thinking: false } };
+    if (format === 'qwen-local')
+        return { ...plan, bodyPatch: { chat_template_kwargs: { enable_thinking: false } } };
+    if (format === 'openrouter')
+        return { ...plan, bodyPatch: { reasoning: { enabled: false } } };
+    if (format === 'server-default' || format === 'google')
+        return undefined;
+    // These official endpoint versions are always-thinking. Do not emit a fake switch.
+    if (settings?.format === 'provider' &&
+        ((provider.id === 'moonshot' && /^kimi-k3/.test(model.id)) ||
+            (provider.id === 'zhipu' && /^glm-?5\.3/.test(model.id))))
+        return undefined;
+    return { ...plan, bodyPatch: { thinking: { type: 'disabled' } } };
+}
 export function resolveThinkingPlan(requestThinking, provider, model) {
     const settings = model.thinking;
     const state = settings?.state ?? 'default';
     // Model state always wins over stale conversation/agent preferences.
-    if (state === 'default')
-        return { mode: 'default', enabled: false };
+    if (state === 'default') {
+        // Exception: an explicit per-request opt-out must still reach the provider.
+        return thinkingDisablePlan(requestThinking, provider, settings, model) ?? { mode: 'default', enabled: false };
+    }
     const off = state === 'disabled';
     const mode = off ? 'off' : normalizeThinkingMode(requestThinking);
     const plan = { mode, enabled: !off };

--- a/dist/src/context/compaction/CompactionEngine.js
+++ b/dist/src/context/compaction/CompactionEngine.js
@@ -8,8 +8,18 @@
     "the early conversation history, so it MUST preserve all information the agent " +
     "needs to continue working without repeating past steps.";
 // Keep room for the summary input inside small configured contexts such as
-// agent.maxContextTokens: 20_000.
-export const COMPACT_MAX_OUTPUT_TOKENS = 4_000;
+// agent.maxContextTokens: 20_000, but do not starve the summary itself: a
+// summarizer that runs with thinking enabled, or that simply writes a long
+// summary, can spend the entire output budget and get truncated mid-response.
+// That truncation is reported as a compaction failure and, because the failure
+// path keeps every message while still appending a boundary marker, it
+// re-triggers compaction in a loop until the gateway dies. 4_000 was too small
+// for real high-entropy transcripts; 16_000 measured 0/3 truncations where
+// 4_000 measured 1/3. The reserve is additionally clamped to
+// COMPACT_OUTPUT_RESERVE_CONTEXT_RATIO of the effective context so small
+// contexts still leave room for the summary input.
+export const COMPACT_MAX_OUTPUT_TOKENS = 16_000;
+export const COMPACT_OUTPUT_RESERVE_CONTEXT_RATIO = 0.4;
 const SUMMARY_MARKDOWN_HEADINGS = [
     "Objective",
     "Current State",
@@ -69,9 +79,13 @@
         const cacheReset = input.cacheReset ?? checkpointMerged;
         const stablePrefix = [];
         const planningMessages = checkpoint.liveMessages;
-        const summaryOutputReserve = positiveTokenCount(input.maxOutputTokens)
+        const requestedSummaryOutputReserve = positiveTokenCount(input.maxOutputTokens)
             ?? positiveTokenCount(this.options.maxOutputTokens)
             ?? COMPACT_MAX_OUTPUT_TOKENS;
+        const summaryContextCap = positiveTokenCount(input.effectiveContextTokens);
+        const summaryOutputReserve = summaryContextCap === undefined
+            ? requestedSummaryOutputReserve
+            : Math.min(requestedSummaryOutputReserve, Math.floor(summaryContextCap * COMPACT_OUTPUT_RESERVE_CONTEXT_RATIO));
         const targetPostTokens = positiveTokenCount(input.targetPostTokens);
         const tailRatio = clamp(input.keepTailRatio ?? DEFAULT_KEEP_TAIL_RATIO, 0, 1);
         const tailTokenBudget = targetPostTokens !== undefined
@@ -125,7 +139,7 @@
             }
             else {
                 try {
-                    const result = await this.summarize(summaryInput, input.userInstruction, input.signal, summaryAnchors, input.maxOutputTokens, checkpoint.previousSummaries);
+                    const result = await this.summarize(summaryInput, input.userInstruction, input.signal, summaryAnchors, summaryOutputReserve, checkpoint.previousSummaries);
                     summaryMessage = wrapSummaryMessage(result.message);
                     summaryUsage = result.usage;
                     this.summaryFailureCooldownUntil = 0;

pilotdeck-compaction-crash-evidence.zip