Skip to content

Bound user-defined function call depth instead of overflowing the stack - #46

Open
arcusbuilds wants to merge 1 commit into
DataDog:masterfrom
arcusbuilds:fix/call-depth-limit
Open

Bound user-defined function call depth instead of overflowing the stack#46
arcusbuilds wants to merge 1 commit into
DataDog:masterfrom
arcusbuilds:fix/call-depth-limit

Conversation

@arcusbuilds

Copy link
Copy Markdown
Contributor

Fixes #30.

execContext.depth was already incremented for every user-defined function call in
bindCallContexts but never read back, so this adds the missing comparison.
bindCallContexts is the single choke point — execCall (value mode) and
collectAssignPaths (path mode) both route through it — so one check covers every
call shape.

Per your answers on the issue: the limit reuses maxRecurseDepth (10,000), there is no
new sentinel error (deferred to #37), and the README wording plus fuzz seeds are included
here for you to call.

Verified terminating, each returning a normal error:

Query Time
def f: f; f 20 ms
def f: .|f; f 50 ms
def f(x): f(x); f(.) 4 ms
def f($x): f($x); f(.) 12 ms
def f: {a:f}; f 24 ms
def f: .a = f; f 7 ms
def f: path(f); f 5 ms

[recurse(.)] from the issue list needs nothing — it already returns errRecurseCycle.

The limit bounds call depth, not total work — please read this part.

The depth error is catchable by try, matching the existing errRecurseDepthLimit
precedent. That means a recursive function whose error handler also recurses absorbs the
limit and evaluates its body twice per level — E(d) = 2·E(d+1) — so it runs unboundedly
instead of returning:

Query On this branch On master
def f: try f catch f; f no result in 20 s fatal error: stack overflow, rc=2
def f: f? // f; f no result in 20 s fatal error: stack overflow, rc=2
def f: (f?, f?); f no result in 20 s fatal error: stack overflow, rc=2

I confirmed the doubling is exponential rather than merely slow by scaling maxCallDepth
locally: limits of 12/14/16/18 gave 2 ms / 9 ms / 33 ms / 144 ms — ×2 per level.

So this is not a regression — master dies on all three, and this branch fixes every
shape the issue actually lists. But the failure mode changes from an instant crash to an
unkillable hang, which for the in-process remote-config case in the issue may be the worse
of the two. I've worded the README and CHANGELOG to say the limit bounds call depth and
not total work, and to tell callers to bound untrusted queries with a timeout, rather than
implying the guard alone makes them safe.

The real cure is making the error uncatchable — a sentinel registered in
isControlFlowError, so try cannot absorb it, matching jq where the recursion error is
uncatchable. That is exactly the error-shape work you deferred to #37, and I did not want
to pre-empt that call. But your "this can wait until #37" was made before either of us knew
catchability creates an exponential hang, so it seemed worth putting back in front of you.
Happy to add it here if you'd rather not ship the gap — it's a small diff on top of this one.

One more, lower stakes: def f: [f]; f terminates but takes ~2.4 s and builds a
230 KB error string, because execArrayConstruct wraps with "in array construction: "
at each of the 10,000 levels — O(n²) string copying. That wrapping predates this change;
the guard only makes it reachable instead of fatal. Same "bounds depth, not work" theme.
It is also why that query is not in the fuzz seeds.

Benchmarks: no movement. Small_Def 680 B/op 16 allocs/op, Small_Bind 432 B/op
8 allocs/op, Complex_TolerantMap 2936 B/op 124 allocs/op — identical to master when run
back to back on the same machine; the guard is one integer comparison and allocates
nothing. I did not regenerate docs/BENCHMARKS.md since no number changed — happy to if
you'd prefer it run regardless.

Tests: rebased onto current master (through #44). go test . -count=1 and
go vet ./... pass. ./jqtest already fails on master for reasons unrelated to this
change, and both the count and the exact set drift run to run — 18–19 failures observed,
with the two reduce range(10001) deep-nesting cases at jq.test lines 2602/2616
appearing intermittently on master and on this branch alike. Across repeated runs of each,
every failure on this branch also occurs on master; there are no new ones.

A recursive user-defined function (def f: f; f) recursed without limit
until the goroutine stack overflowed. Go treats that as a fatal error
rather than a panic, so recover() cannot catch it and the calling process
dies -- with no mitigation available to the caller, since a compiled
*Program cannot be inspected for recursion.

execContext.depth was already incremented for every user-defined call but
never read back. bindCallContexts is the single choke point for both value-
and path-mode calls, so one comparison there covers every call shape.
The limit reuses the existing maxRecurseDepth value (10000), roughly 13 MB
of stack at ~1.3 KB per level.

The depth error is an ordinary error value, so try catches it, matching the
existing errRecurseDepthLimit precedent. The README and CHANGELOG say the
limit bounds call depth and not total work, and that untrusted queries still
need a timeout.

Fixes DataDog#30

Signed-off-by: Srijan Keshri <srijankeshri007@gmail.com>
@arcusbuilds
arcusbuilds requested a review from a team as a code owner August 18, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recursive user-defined function causes a fatal stack overflow that recover() cannot catch

1 participant