The DuckLake catalog answers a call that exits as an error, so retention, GC and the compactor keep their state (T-464) - #329
Merged
Conversation
chasers
added this pull request to stack #332
September 11, 2026 17:07
…ion, GC and the compactor keep their state (T-464)
Every Catalog callback already promised {:error, term()}, but every
statement the DuckLake implementation ran was a bare GenServer.call: a
read that timed out behind a busy connection, or found the connection
gone, exited its caller. The compactor had grown its own catch for that
(T-460); Retention.run/1 and GC.run/1 still crashed, and GC's crash reset
its grace-period candidates, so a catalog engine timing out each sweep
meant GC never deleted anything.
Engine.try_transaction/3 joins try_query/4, and DuckLake's query/4 and its
five transaction sites go through the try layer, so an exit comes back as
{:error, %CallExited{}} to every caller. The compactor's listing and read
wrappers go; its swap tags the error {:swap_failed, ...} so the log names
the phase, and one catch stays for an exit the catalog never sees — a
store put whose HTTP pool died mid-upload.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014K63Kxq4pf7eD9K1416Eju
…ention stops at the first exit, the swap's exit keeps its bare shape, and the engine's two try functions share one catch
With the catalog answering exits as errors, PgCatalog.listed_tables/1
collapsed any error to [] and rebuilt pg_namespace and pg_class empty, so a
psql client asking \dt during a wedge saw a database with no tables and
the empty build was cached for the TTL. A refresh that cannot read the
catalog now answers {:error, {:pg_catalog_unavailable, _}} and leaves the
generated tables and the timestamp as they were, which is what main's
crash-and-restart gave the client.
Retention.run/1 mapped every table and turned each exit into a per-table
failure, so a wedged connection cost one timeout per table instead of one;
it now stops at the first exit and reports the rest as deferred, like the
compactor. The compactor's replaced/4 existed only to re-tag the swap's
exit as :swap_failed, which nothing branched on; the bare struct is the
one shape. Engine.try_query/4 and try_transaction/3 share catching_exit/1.
Smolquery.Test.ExitingCatalog wraps a real catalog and answers chosen
calls with %CallExited{}, since a real wedge cannot be staged
deterministically.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014K63Kxq4pf7eD9K1416Eju
chasers
force-pushed
the
t-464-catalog-try-layer
branch
from
September 11, 2026 21:15
145d22a to
4fada2c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracker: T-464. Stacked on #328 (T-462).
Every
Smolquery.Catalogcallback already promised{:error, term()}, but every statement the DuckLake implementation ran was a bareGenServer.call: a read that timed out behind a busy connection, or found the connection gone, exited its caller. The compactor had grown its own catch for that (T-460);Retention.run/1andGC.run/1still crashed, and GC's crash reset its grace-period candidates, so a catalog engine timing out each sweep meant GC never deleted anything.Engine.try_transaction/3joinstry_query/4.query/4and its five transaction sites go through the try layer, so an exit comes back as{:error, %CallExited{}}to every caller.{:swap_failed, ...}so the log names the phase, and one catch stays for an exit the catalog never sees (a store put whose HTTP pool died).Tests:
try_transactionon a missing and a busy connection; the catalog answering{:error, %CallExited{}}from a read and a commit; a retention sweep failing instead of crashing on a dead catalog connection; the compactor's existing exit tests re-shaped.🤖 Generated with Claude Code
https://claude.ai/code/session_014K63Kxq4pf7eD9K1416Eju