[shadaj]

Make the disk-backed daemon durable against being killed at any point, and
stop rewriting whole history files on every completion.

Conversation log (threads/{thread_id}.jsonl)

• Each thread's conversation is now an append-only JSONL log (persist::ThreadLog).
  The first line is a header carrying the format version and thread id; every
  later line is a message (with the dedup id it was appended under) or a
  compaction_summary. Appends are a single write_all + fdatasync, so a
  crash can tear at most the final line.
• Loading recovers the longest valid prefix: a torn trailing line is
  dropped (with a warning) and the file truncated to the last complete line
  before anything new is appended. A complete-but-unparseable line, or a header
  from a newer version, is a loud error rather than a silent reset.
• Legacy whole-snapshot {thread_id}.json files are migrated into the log on
  first load (the log is written atomically, so a crash mid-migration just
  retries); the legacy file is left in place and ignored thereafter.

Versioned, atomically written documents

• {thread_id}.meta.json, {thread_id}.views.json, state/{thread_id}.state.json
  and sessions.json now carry a version field and are written via
  temp-file + fsync + rename (persist::write_atomic), so readers only ever see
  the old or the new contents. Unversioned files from older daemons read as
  version 0 and are upgraded eagerly on first load; files from a newer daemon
  are refused instead of being overwritten.
• Load/save errors are no longer swallowed: trait methods propagate them, and
  the synchronous getters log them and skip any save.

Processed ids are looked up in history, not stored separately

• StateStore::get_processed_ids / add_processed_message_ids are removed.
  ConversationStore gains is_message_processed(thread_id, message_id), a point
  lookup against the persisted history (each row/line already stores its dedup
  id). This eliminates the partial-write hazard of a message landing in one
  store but its dedup id not landing in the other, and the DSQL backend no
  longer needs DynamoDB for this (it runs a SELECT EXISTS with a new
  (session_id, message_id) index). The daemon's .state.json drops the
  processed_message_ids field.
• InMemoryConversationStore keeps a per-thread HashSet of dedup ids
  alongside the entries (same lock, same mutation paths) so the lookup is O(1).
• HistoryManager::handle_input is the new async entry point for incoming
  inputs: it checks ids handled in this process, then asks the store; the in-
  memory set is no longer preloaded. handle_content stays for in-process
  synthetic content with fresh ids.

Incorporated PR #170

Interrupts are recorded as a first-class InfinityMessage::Interrupt history
entry (rendered to the model at request time) instead of rewriting user text.
This means a persisted history that ends on an unanswered tool call without
an interrupt entry was cut off by a crash — the signal a future auto-resume
can use.

Breaking changes

BREAKING CHANGE: StateStore::get_processed_ids and add_processed_message_ids
are removed; ConversationStore::is_message_processed is a new required method.
BREAKING CHANGE: stores::ThreadState no longer has processed_message_ids.
BREAKING CHANGE: daemons older than this cannot read the new .jsonl logs or
histories containing interrupt entries.

Not covered (by design, for now)

Inputs the model has not yet validated are still held only in memory and are
lost on a crash (the existing context-overflow protection); a durable inbox is
the next step toward auto-resume.

[cloudflare-workers-and-pages]

Deploying infinity with    Cloudflare Pages

Latest commit:	a2cba45
Status:	✅  Deploy successful!
Preview URL:	https://458f9f09.infinity-dc7.pages.dev
Branch Preview URL:	https://sandbox-85094202-d192-4d34-8.infinity-dc7.pages.dev

View logs