Should exponential backoff be... exponential?

[radubahmata]

The Claude usage window is (currently) 5 hours. As the window nears its end, the backoff grows, so the agent sleeps longest right when the new window is about to refresh.

Since max_retry_wait already caps things for the weekly-limit case, why is the shape exponential at all? A fixed interval (let's say 15/30 min) bounds the wake-up lag to a small fraction of the window and keeps behavior predictable.

Am I missing a reason for the exponential curve?

[moodmosaic]

No, you're not missing a reason — the exponential curve exists because it's the right default for unknown-duration failures. It's the wrong tool for the five-hour limit, which is a known-duration failure the server explicitly tells us about. A fixed interval is strictly an improvement for the rate-limit case and a regression for transient errors; honouring resetsAt is strictly better than both for the rate-limit case and costs nothing for transient errors.

Long answer to your question

I'll lay out exactly what the code does today and walk through why neither the current exponential curve nor a flat 15/30 min interval is actually ideal.

What the code does today

From lib/harness.sh:352-362:

            _backoff=30; _total_waited=0
            while [ "$_total_waited" -lt "$MAX_RETRY_WAIT" ]; do
                hlog_err "retriable error: ${FATAL_MSG}"
                hlog "retrying in ${_backoff}s (waited ${_total_waited}/${MAX_RETRY_WAIT}s)"
                printf '%s/%s\n' "$_total_waited" "$MAX_RETRY_WAIT" > "$RETRY_FILE"
                sleep "$_backoff"
                _total_waited=$((_total_waited + _backoff))
                _backoff=$((_backoff * 2))
                if [ "$_backoff" -gt 1800 ]; then
                    _backoff=1800
                fi

Concrete sleep schedule (start 30 s, double, cap at 1800 s = 30 min):

Attempt	Sleep	Cumulative
1	30 s	30 s
2	60 s	1 m 30
3	120 s	3 m 30
4	240 s	7 m 30
5	480 s	15 m 30
6	960 s	31 m 30
7	1800 s	61 m 30
8+	1800 s	+30 min

Worst-case wake-up lag once the backoff saturates: up to 30 min past the true reset time. With max_retry_wait = 25200 (7 h) inside a 5 h window, after a few failed probes you spend ~30 min asleep per iteration.

Why exponential is the default

Classical reasons — which I'll argue don't fit the 5-hour case:

1. Unknown-duration outage. The remote could recover in 1 s or 1 h. Doubling means the first retry is fast if the blip was short, and you asymptotically avoid hammering a service that's actually down.
2. Load shedding. If many clients are all retrying, exponential backoff (+ jitter) desynchronises them and keeps the thundering-herd bounded.
3. Server doesn't tell you when it's back. So "probe efficiently" is the best you can do.

All three apply squarely to transient errors: overloaded_error, api_error / 500, internal_error, DNS blips, zero-token network failures. That's exactly what the current agent_is_retriable classifies alongside rate limits.

Why exponential is wrong for the 5-hour rate limit specifically

Reason 3 above is false here — the API does tell you when it's back. Every rate_limit_event in stream-json carries:

{"type":"rate_limit_event","rate_limit_info":{"status":"rejected","resetsAt":1775505600,"rateLimitType":"five_hour"}}

(I saw the same in the live Opus 4.7 probe earlier — "resetsAt":1776394800,"rateLimitType":"five_hour".) So you know to the second when the window refreshes, yet agent_is_retriable only returns a yes/no — the retry loop never reads resetsAt or rateLimitType.

That's why your observation is sharp: the loop treats a hard-deadline wait (5 h) the same as an unbounded outage, and the doubling curve is increasingly wasteful as you approach the real reset. Two distinct kinds of waste:

• Probe waste. Every retry that wakes before resetsAt is a doomed request that hits the server, gets another rate_limit_event, and goes back to sleep longer. With a 30 s/60 s/... ramp, the first 6 probes happen inside the first ~30 min — all guaranteed-to-fail on a five-hour limit.
• Oversleep waste. Once _backoff saturates at 1800 s, a limit that lifts 1 min after you fell asleep costs 29 min of idle agent time (and × N for a swarm).

Why a fixed 15-30 min interval isn't ideal either

You're right that it caps the oversleep to ≤ interval, but it has the mirror problem:

• Worse than exponential for transient errors. A 30 s blip means waiting 15-30 min for the first retry. Exponential wins here.
• Still polls blindly. On a 4 h 59 m-remaining five-hour limit, 15 min fixed = 20 doomed probes; 30 min fixed = 10 probes. Less wasteful than exponential's early ramp, more wasteful than resetsAt.
• Doesn't use information the API gives you for free.

The fix that actually matches the problem

Branch on what agent_is_retriable already distinguishes internally:

• Transient (overloaded, api_error / 500, network): keep the current 30 s → double → 30 min cap exponential. It's correct here.
• Rate-limited (rate_limit_event with a resetsAt): read the timestamp, sleep max(resetsAt - now, small_min) + small_jitter, all capped by MAX_RETRY_WAIT. One sleep, one retry, no probe waste, no oversleep beyond the jitter.
• Rate-limited with no resetsAt (older Claude Code, other drivers): fall back to exponential.

rateLimitType is also useful: "five_hour" and "weekly" both carry a resetsAt and should use it; anything else can fall through.

Concretely: change agent_is_retriable to also emit a suggested wait (or emit the parsed resetsAt) and let the loop do sleep_until when it's available. The cap at MAX_RETRY_WAIT stays as the safety valve for the weekly case, as you noted.

[radubahmata]

Fantastic! Didn't know about the rate_limit_event payload. Branching/pattern matching sounds like the best way to go here. Do you plan to refactor this soon? Want me to open an issue for it?

[moodmosaic]

@BowTiedRadone see #74. Thank you for catching this!

[moodmosaic]

Closing this as per #74.