Skip to content

Reuse the migrated local device in the legacy mirror; stop UpsertDevice FK scans - #217

Open
MaxGhenis wants to merge 8 commits into
mainfrom
claude/interesting-payne-d53972
Open

MaxGhenis wants to merge 8 commits into
mainfrom
claude/interesting-payne-d53972

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #215 (SQL plan audit), items 1 and 2.

What was wrong

1. The legacy mirror could not run on a migrated store. v2wire.ensureLocalDevice upserted local-primary:<account> as a current local_installation device. The migration already gives every account a current local device under a derived id (9bcc1343… for Google on the live store), and devices_current_local_uq allows one current local device per account. So every call through the mirror failed:

upsert device "local-primary:google-primary": ... UNIQUE constraint failed: devices.account_id (2067)

This reproduces on origin/main with a store built by migration.Transform, and on a copy of the live v2 store (all three accounts).

2. UpsertDevice scanned read cursors on every conflict. Its DO UPDATE SET assigned account_id = excluded.account_id. account_id is part of the parent key that read_cursors(account_id, device_id) references, so SQLite compiled a scan of the device's cursors into every upsert, even when the value did not change. The bytecode confirmed it: the old statement opens read_cursors, its autoindex, and outbox_read_receipts, with FkCounter/FkIfZero bookkeeping. The new statement opens none of them. On the live copy (797 Google cursors), a same-account conflict upsert in autocommit dropped from 892 µs to 138 µs.

Two writes that ran around the device step. Fixing the device step would let them run further, so both are fixed here:

  • MirrorConversation called UpsertAccount before the device step, as its own autocommit. Even the failing calls above rewrote the account row: google_messages→google, whatsmeow→whatsapp, signal_cli→signal, plus display name, enabled, mode and config (observed on the migrated fixture and on the live copy). internal/v2read derives each conversation's platform from bridge_key and passes unknown keys through, so Google threads would read as google instead of sms. The live store's keys are still clean.
  • Its UpsertConversation upserts on the natural key. On a migrated store, that would rewrite the hash-keyed v2 row with the same (account_id, remote_conversation_id) (title, kind, archive, mute, favorite, recency, metadata) and only then fail the "natural key belongs to v2 conversation" check. The failing device step was all that stopped this.

Changes

Where Change
sqlite.Store.EnsureLocalInstallationDevice (new) One INSERT … SELECT … WHERE NOT EXISTS (local_installation for this account), then returns GetLocalInstallationDevice. It never adds a local device next to an existing one, whatever that one's id.
v2wire.ensureLocalDevice Uses it and returns the device id. MirrorReadCursor writes the cursor on that device, the same one ingest's advanceReadCursor uses.
sqlite.Store.UpsertDevice account_id is out of the SET. WHERE devices.account_id = excluded.account_id plus a zero-rows-affected check now return ErrCrossAccountDevice (and ErrConstraintViolation). Before, a device with no cursors moved accounts without any error.
sqlite.Store.EnsureAccount (new) ON CONFLICT(account_id) DO NOTHING, returns the stored row. The mirror uses it, so existing accounts keep their metadata.
v2wire.accountBootstrap When the mirror creates an account, it uses the same bridge key and display name as the live adapters and the migration (google_messages/whatsmeow/signal_cli). A differential test pins this against the migration's output.
sqlite.Store.UpsertOwnedConversation (new) The natural-key upsert plus WHERE conversations.conversation_id = excluded.conversation_id. It writes nothing and returns ErrConversationIdentityConflict when another row owns the key. UpsertConversation is unchanged: the migration and ingest (worker.go, idspace.go, backfill_participants.go) rely on its merge.
v2wire.mirrorNaturalKey (new) Before writing, the mirror refuses if the legacy thread already has a v2 row under another id, in either place one can be. First, under the routed account at the remote id normalized as the migration normalizes it (v2keys.NormalizeRemoteConversationID), so signal: +1650… meets a migrated signal:+1650…. Second, at any placement the migration could have given this legacy id. The migration routed by the platform stored at migration time and the mirror routes by id prefix, and the legacy store can change a thread's stored platform afterwards; so a signal: thread once stored as sms may sit under Google. Those placements are under the migration's own accounts and keyed by the legacy id itself, so an unrelated thread that shares an opaque Google thread id under another account is not consulted. The key written is the normalized form. The one exception is a row an earlier mirror wrote under the same account with the raw legacy id as its key, which keeps that key. A legacy-id row with any other key is refused, not adopted; ingest rewrites a stale Google binding's key to displaced:….
migration.ConversationPlacement, ConversationPlacements (new) ConversationPlacement(platform, id) is the account and normalized remote id Transform files a legacy conversation under, composed from the migration's own accountForPlatform and normalization. ConversationPlacements(id) lists that for every platform the migration handles, so a caller need not trust a stored platform that may have changed since. A differential test checks both against rows the real migration wrote.
/api/v1/outbox errors ErrConversationIdentityConflict → 409 with the message, which names the owning v2 conversation. It was previously a 500.
/api/mark-read Runs the legacy mirror only when !V2Primary, matching the /api/v1 send split.
docs/agent-runbook.md One bullet in the cutover section: what the mirror now does on a migrated store, the old error text, and how to check the bridge keys.

Callers, by mode

ensureLocalDevice is reached only through mirrorConversation:

Caller Live install (v2-primary, v2 send) Legacy-primary + OPENMESSAGES_V2_SEND=1 on a migrated store (e.g. rollback) Legacy-primary + v2 send on a never-migrated store
POST /api/v1/outbox/messages|media → SubmitText/SubmitMedia Not reached: native SubmitTextV2/MediaV2 Before: every send 500s on the UNIQUE error, after the account rewrite. After: threads created after migration send; threads the v2 store already holds under another id get a 409, and the v2 row is left unchanged Worked before (minted local-primary:*), but each call rewrote the bridge key to google. After: same, with no rewrite
MCP send_message, send_group_message, send_to_conversation, send_media_to_conversation → tools/v2.go submitText/submitMedia (in the daemon; transportless stdio clients post to its /api/v1/outbox via internal/localapi) Not reached (V2Primary → native) Same as the row above (the tool returns the error text) Same as the row above
Reply sends → mirrorSubmitReplyTarget → MirrorReplyTarget → MirrorConversation Not reached Same as the row above Same as the row above
POST /api/mark-read → mirrorV2ReadCursor → MirrorReadCursor (best effort, logged, always 200) Before: v2 ids fail at the legacy lookup (no write, warning). A legacy-form id rewrote the account row, then failed on the UNIQUE error. After: the mirror is skipped Before: warning plus account rewrite. After: cursor on the migrated device for post-migration threads; refusal for migrated ones, with the v2 row left unchanged Worked before. After: same, no rewrite

On the live install the mirror never succeeded. Nothing consumes v2 read cursors for unread state yet (v2read sets UnreadCount: 0). Skipping the mirror on v2-primary therefore loses no working behavior; native v2 read-cursor writes are a follow-up.

Invariants (property-tested)

  • At most one current local installation device per account (schema), and the mirror never attempts a second one.
  • The mirror adds a device only for an account with no local installation device. Otherwise device rows stay byte-identical.
  • Mirror cursors land on GetLocalInstallationDevice(account), the device ingest uses.
  • The mirror never modifies an existing account row, or a conversation row whose id differs from the legacy id.
  • The mirror never hides a row that already holds this legacy conversation, whatever its stored platform is now. When a row exists at the routed key or at any migration placement before the call, v2read returns the same history for the legacy id before and after (TestMirrorNeverChangesLegacyIDReads, TestMirrorRefusesMigratedThreadAfterLegacyPlatformChanges, both differential against internal/v2read). A row another writer adds under a different account during the call is not detected; that gap predates this PR.
  • A refusal writes nothing when the conflicting row exists before the call. If a concurrent writer creates it mid-call, the account and device bootstraps may already have run; the guarded upsert still refuses a conflict on the routed account's key.
  • An unrelated thread under another account never blocks a send or mark-read, even when it shares the remote id (TestMirrorConversationIgnoresUnrelatedThreadInAnotherAccount).
  • UpsertDevice never changes a device's account. The compiled statement has no FK child scans; this is pinned from the bytecode, with child tables read from pragma_foreign_key_list.
  • EnsureLocalInstallationDevice is idempotent, and on a taken id it fails without writing.

Tests: TestEnsureLocalInstallationDeviceProperties (200 random device tables) and TestMirrorReadCursorDeviceProperties (40 random account/device shapes × up to 6 mirror calls). Differential: TestAccountBootstrapMatchesMigrationAccounts. Migrated-shape suites build the v2 store with the real migration.Transform. Five mutants (account_id back in the SET, the device WHERE dropped, NOT EXISTS disabled, the conversation guard dropped, EnsureAccount turned into a clobbering upsert) are each killed. The two new /api/mark-read tests fail on origin/main.

Verification

  • go test ./... passes (34 packages), and go vet is clean. A local go test -race ./... run is noted in a comment.
  • Live store: copied v2/store.sqlite3 with its -wal/-shm (integrity_check ok, foreign_key_check clean); the live files were never opened. Before/after output, probes, bytecode notes and mutation results: ~/reviews/openmessage-sqlite-plan-audit-2026-10-09/device-fix/EVIDENCE.md.

Review

Four rounds by GPT-6.1 Sol on Subfleet, each REQUEST_CHANGES; every finding is fixed. Reports: ~/reviews/openmessage-sqlite-plan-audit-2026-10-09/device-fix/review/.

Round 1 (20261009-111730-om217-review, at 96c10f7), fixed in 2665680:

  • Major: the Signal natural key was not normalized, so a spaced legacy id minted a duplicate beside the migrated row.
  • Minor: "rows the mirror did not create are never rewritten" overstated a check that compares ids.
  • Minor: a refusal could bootstrap a device first.
  • Nit: a stale localDeviceID comment.

Round 2 (20261010-073548-om217-review-r2, at 2665680), fixed in d6eb9c6:

  • Major: my round-1 fallback adopted whatever key the legacy-id row held, including a key ingest had displaced, and dispatch then sent to it. It now accepts only the exact earlier-mirror shape and refuses the rest.
  • Major: prefix routing let the mirror create a row under a different account than the migration used, shadowing migrated history for legacy-id reads. This divergence predates the PR.
  • Minor: the write-free claim did not hold under a concurrent writer. The claim is now scoped (see Invariants).
  • Nit: a cross-account device upsert never orphaned read cursors; with cursors the foreign key failed the write.

Round 3 (20261010-091006-om217-review-r3, at d6eb9c6), fixed in adfd636:

  • Major: my round-2 fix scanned every account and so refused a legitimate send when an unrelated thread under another account shared the remote id. It now checks only the routed key and the migration's placement of this conversation.
  • Major (pre-existing, not worsened): a row added under another account during the call is not detected. The invariant is now stated for the serial case only.
  • Nit: a comment wrongly said ingest uses the same bridge-key mapping. The mirror no longer has that mapping.

Round 4 (20261010-095905-om217-review-r4, at b55936d), fixed in 14916d1:

  • Major (made worse by this PR): the placement check recomputed where the migration filed a thread from the thread's current stored platform. The legacy Signal receiver can correct a signal: thread from sms to signal after migration, so the check looked under Signal and missed the migrated row under Google. On main the device collision happened to refuse this; with the device fixed, the mirror wrote a shadowing row (legacy-id history 1 → 0, executed). The mirror now checks every placement the migration could have used (migration.ConversationPlacements).
  • Major (pre-existing): a cross-account owner inserted mid-call is still not detected; the invariant is scoped to rows that exist before the call.

The branch then merged main twice (through #243), with the touched packages (v2wire, sqlite, web, migration) re-run on the merged tree.

Each round's regression tests fail on the head they were raised against. Mutants for every fix are killed, including dropping the placement check.

Round 3 confirmed the round-2 fixes by replaying its own reproductions: the displaced binding is refused before and after another row takes the key, and DispatchDue sends nothing; the spaced Signal thread stored as sms is refused with its history intact; the corrected device comment matches executed behavior on the base. It ran the full v2wire suite, the focused sqlite suite and a v2wire race run, all passing.

Not changed

  • The mirror still refuses threads the migration keyed by hash. In legacy-primary mode on a migrated store those sends still fail, now with a 409 instead of a 500. Adopting the hash row would mean mirrored reply targets could duplicate migrated messages, which is a design call. It is filed as a follow-up.

🤖 Generated with Claude Code

…ce FK scans

On a migrated v2 store every account already has a current
local_installation device under a derived id. The legacy mirror
(v2wire.ensureLocalDevice) tried to add "local-primary:<account>" next to
it, which devices_current_local_uq rejects, so every MirrorConversation /
MirrorReadCursor / SubmitText call through the mirror failed with
"UNIQUE constraint failed: devices.account_id". Reproduced on a store built
by migration.Transform and on a copy of the live store.

- sqlite: EnsureLocalInstallationDevice inserts a local device only when the
  account has none (one INSERT ... WHERE NOT EXISTS) and returns the device
  GetLocalInstallationDevice resolves, the one ingest advances receipts on.
- v2wire: the mirror uses it and writes read cursors on that device.
- sqlite: UpsertDevice drops account_id from the DO UPDATE SET. Assigning it
  made SQLite scan the device's read_cursors children on every conflict
  (read_cursors references devices(account_id, device_id)). A WHERE keeps a
  device from moving accounts; that now fails with ErrCrossAccountDevice
  instead of silently succeeding when the device had no cursors.

Fixing the device step un-masks two writes that ran around it:

- MirrorConversation's UpsertAccount ran before the device step and rewrote
  existing accounts (bridge_key google_messages -> google, display name,
  enabled, config) even when the call then failed; v2 reads derive platform
  from bridge_key. It now uses EnsureAccount (insert if absent), with the
  bootstrap values the live adapters and the migration use.
- Its UpsertConversation would, on a migrated store, rewrite the hash-keyed
  v2 row holding the same natural key (title, kind, archive, mute,
  favorite, recency, metadata) before erroring. It now uses
  UpsertOwnedConversation, which writes nothing and returns
  ErrConversationIdentityConflict (409 on /api/v1/outbox) in that case.
  UpsertConversation keeps its merge semantics for the migration.

/api/mark-read no longer runs the legacy mirror on a v2-primary daemon,
matching the /api/v1 send split: there the UI sends v2 ids the mirror
cannot resolve, and a resolvable legacy id would add a legacy-keyed row to
the primary store.

Evidence: ~/reviews/openmessage-sqlite-plan-audit-2026-10-09/device-fix/

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Owner Author

Local verification at 96c10f7: GOWORK=off go test -race ./... -count=1 passed (34 packages ok, exit 0), and go vet ./internal/... ./cmd/... is clean. Independent hard-tier review is dispatched (Subfleet run 20261009-111730-om217-review).

Review of #217 (Subfleet 20261009-111730-om217-review, REQUEST_CHANGES):

- Major: the mirror upserted under the raw legacy id, but the migration
  stores Signal threads under v2keys.NormalizeRemoteConversationID, so
  "signal:  +1650..." missed the migrated row and minted a duplicate that
  v2read's alias lookup would then prefer. The natural key is now the
  migration's normalized form. A row an earlier mirror created under the
  legacy id keeps the remote id it was created with, so those stores do not
  hit their own primary key.
- The ownership check now runs before the account and device bootstraps,
  so a refused call writes nothing. Before, an account with no local device
  gained one first.
- Comments now say what the guard checks (ID equality, not provenance) and
  what the account-scoped device id used to protect against.

Tests: migrated fixtures gain spaced Signal threads (migrated and
post-cutover), plus an earlier-mirror-row case and a refusal-writes-nothing
case. Each of the three changes is killed by its mutant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Owner Author

Local verification at 2665680: go vet is clean, go test ./... passed (34 packages ok), and go test -race ./... passed (34 packages ok). Round-2 independent review is queued (Subfleet 20261010-073548-om217-review-r2).

MaxGhenis and others added 6 commits October 10, 2026 09:09
Round 2 review of #217 (Subfleet 20261010-073548-om217-review-r2,
REQUEST_CHANGES at 2665680):

- Major: the "earlier mirror row keeps its key" fallback returned whatever
  key the legacy-ID row holds now. Ingest rewrites a stale Google binding's
  key to "displaced:...", so the mirror adopted that key and dispatch sent
  to it. The fallback now accepts only the exact earlier-mirror shape
  (same account, key equal to the raw legacy ID), and a legacy-ID row with
  any other key or account is refused. It also no longer refreshes a raw-key
  row while another row holds the normalized key.
- Major: the mirror routes by ID prefix, the migration by stored platform,
  so a "signal:" thread stored as sms was migrated under Google and the
  mirror created a second row under Signal that shadowed the migrated
  history for legacy-ID reads. mirrorNaturalKey now looks the legacy ID up
  the way internal/v2read resolves one: under every account, normalized for
  that account's platform. Any match under another conversation ID is a
  refusal.
- Minor: a refusal is write-free only when the conflicting row exists
  before the call. If a concurrent writer creates it mid-call, the account
  and device bootstraps may already have run; the guarded upsert still
  leaves the other conversation unchanged. Comments say so.
- Nit: a cross-account device upsert never orphaned read cursors. With
  cursors the foreign key failed the write; without them the device moved.

Tests: a misrouted migrated thread in the migration-built fixture, a
differential check that v2read returns the same history for a legacy ID
before and after the mirror runs, a displaced-binding case through
SubmitText, and a raw-key row beside a normalized owner. All four fail on
2665680; the two new branches are each killed by a mutant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 3 review of #217 (Subfleet 20261010-091006-om217-review-r3,
REQUEST_CHANGES at d6eb9c6):

- Major (introduced by d6eb9c6): the all-accounts scan refused a legitimate
  send when an unrelated thread under another account shared the remote ID
  (opaque Google thread IDs can repeat across accounts). v2read resolves a
  direct ID match first, so that thread was never this conversation.
  mirrorNaturalKey now checks two places only: the routed account's
  normalized key, and where the migration files this legacy conversation.
  The second comes from the new migration.ConversationPlacement, which
  composes the migration's own accountForPlatform and remote-ID
  normalization, so the mirror does not carry a copy of that table.
- Major (pre-existing, not worsened): a row another writer adds under a
  different account during the call is not detected. The comments now say
  what is guaranteed: a refusal writes nothing when the conflicting row
  exists before the call, and the guarded upsert covers only the routed
  account's key.
- Nit: the mirror no longer has its own bridge-key-to-platform mapping, so
  the comment that wrongly called it identical to ingest's is gone.

Tests: an unrelated thread under a second Google account, in both orders,
through MirrorConversation, MirrorReadCursor and SubmitText (fails on
d6eb9c6); a differential check that ConversationPlacement names the row the
real migration wrote for every fixture thread. Dropping the placement check
is killed by the misrouted-thread and v2read differential tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-3 reviewer's failing paths included reply-target reuse, so the
unrelated-thread test now sends two replies to the same legacy message;
the second reuses the mirrored reply target.

The runbook said a refused mirror call writes nothing. That holds only when
the conflicting row exists before the call, so it now states what is always
true: the migrated row is left unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 4 review of #217 (Subfleet 20261010-095905-om217-review-r4,
REQUEST_CHANGES at b55936d):

- Major (made worse by this PR): the mirror recomputed the migration's
  placement from the conversation's current stored platform. The legacy
  Signal receiver can correct a "signal:" thread from sms to signal after
  migration, so the check looked under Signal, missed the migrated row
  under Google, and the mirror (no longer blocked by the device collision
  that stopped it on main) created a legacy-ID row that hid the history.
  migration.ConversationPlacements now lists every placement Transform
  could have given a legacy ID, one per platform it migrates, and the
  mirror checks them all. They sit under the migration's own accounts and
  are keyed by the legacy ID itself, so an unrelated thread under another
  account (round 3) is still not consulted.
- Major (pre-existing): a cross-account owner inserted mid-call is still
  not detected; the comments already scope the guarantee to rows that exist
  before the call.

Tests: a second "signal:" thread stored as sms in the migration-built
fixture, and TestMirrorRefusesMigratedThreadAfterLegacyPlatformChanges,
which flips the stored platform after migration and checks the mirror
refuses and legacy-ID history is unchanged (fails on b55936d). The
placement differential now also checks every migrated row is among
ConversationPlacements and that it names only the migration's accounts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant