Repository navigation
Conversation
…ing them
A legacy-primary daemon with OPENMESSAGES_V2_SEND=1 on a store built by
`openmessage migrate` (for example after rolling back from v2-primary)
could send only to threads created after the migration: the mirror refused
every thread the migration keyed by v2keys.DeriveID hash with
ErrConversationIdentityConflict (409).
The mirror now adopts the v2 row that holds the thread's natural key
(account_id, remote_conversation_id = legacy id) and returns its id without
writing it. It keys a row by the legacy id only for a thread v2 has never
seen, and adopts a row v2 ingest creates between its lookup and its upsert.
Two consumers needed fixing so adoption is safe:
- The legacy visibility projector wrote each confirmed send into the legacy
store under the v2 conversation_id. The legacy upsert sets conversation_id
on a message-id conflict, so a hash id would also move an already-ingested
echo out of its thread. It now writes into the thread named by the v2
conversation's remote_conversation_id and publishes that id.
- Signal reply targets. The mirror stored them under the full legacy id
("signal:<ts>"), while the migration and the v2 decoder store the bare
timestamp, so a reply in a migrated thread would have added a second copy
of the quoted message. The mirror now uses the bare id (and reuses copies
it stored under the full id before). signallive signalQuoteArgs resolves a
bare id by restoring the "signal:" prefix when it is not itself a legacy
message id; every id that resolved before resolves identically. This also
lets v2-native Signal replies (SubmitTextV2), which carry bare ids, quote
messages the legacy store holds; before, every one failed with "signal
reply target not found".
A Google reply target the migration keyed by a source_id other than its
message id (no live Google writer sets source_id) is refused with
ErrReplyTargetUnavailable rather than duplicated, since the transport quotes
the message id.
Tests: migrated-shape suites built with the real migration.Transform
(adoption without rewrites, sends and replies to migrated threads, a
differential check of reply remote ids against the migration, the projector
on an adopted thread with and without an ingested echo, the ingest race via
a test seam), a property test over random mirror call sequences on a
migrated store with ingest-keyed threads, a differential property test of
the Signal quote fallback against exact lookup, an HTTP mark-read test, and
an end-to-end rollback test in cmd that runs the real `openmessage migrate`,
a legacy-primary v2 stack and scripted adapters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #217 (base is its branch; I'll retarget to
mainonce it merges). This is the follow-up #217 filed under "Not changed".What was wrong
After #217, the legacy→v2 mirror refuses any thread whose natural key
(account_id, remote_conversation_id)already belongs to a v2 row under another id. Every pre-migration thread on a store built byopenmessage migrateis such a thread, since the migration keys conversations byv2keys.DeriveID. So a legacy-primary daemon withOPENMESSAGES_V2_SEND=1on a migrated store (for example after rolling back from v2-primary) could only send to threads created after the migration. Sends, reply sends and mark-read mirroring to migrated threads got a 409.The three questions
1. What do the adapters consume?
RemoteConversationID, never the v2conversation_id. The dispatcher loads the conversation and passesbridge.ConversationRef{RemoteID: conversation.RemoteConversationID}for text, media, reactions and read receipts (internal/messaging/dispatch.go:206, :377, :530, :680). Replies carry the quoted message'sRemoteMessageID(internal/messaging/service.go:176). The Google, WhatsApp and Signal adapters read onlyreq.Conversation.RemoteIDandreq.ReplyTo.RemoteID. The mirror's doc comment ("byte-for-byte equal to the legacy ID consumed by the live adapters") was therefore wrong about who consumes it.What does treat the v2
conversation_idas a legacy id is the legacy visibility projector, which runs only on legacy-primary daemons (cmd/v2stack.go:377).legacyProjectionwrote each confirmed send into the legacy store withConversationID: row.ConversationID, andPublishMessages(row.ConversationID)announced it under that id. With a hash id, a send would land in a legacy thread no one lists, and the legacy UI (primary in this mode) would never show it. The legacy upsert also setsconversation_id = excluded.conversation_idon a message-id conflict (internal/db/messages.go:59), so an echo the legacy store already had would be moved out of its thread. Adoption is only safe with the projector fixed.2. What would adopting break: reply targets. I compared each platform's remote message id across the mirror (
replyRemoteID), the migration (deriveRemoteMessageID:source_id, else the legacy id with its platform prefix stripped) and the v2 decoders:message_id/source_id(writers)internal/client/events.go:160) / empty (no live writer sets it)message_idmessage_idGetMessageID()(googledecoder.go:387)whatsapp:<id>/<id>source_idsource_idsignal:<ts>/<ts>(every Signal writer, including the Signal Desktop importer, storesmessage_id = "signal:" + source_id)signal:<ts><ts><ts>(signaldecoder.go:386)v2 dedupes messages by
(account, conversation, remote_message_id), keeping the first message id. The mirror'slegacy-reply:row undersignal:<ts>would therefore sit beside the migrated or ingested<ts>row as a second copy of the quoted message. That copy would show in reads if the same store became primary again by re-settingOPENMESSAGES_V2_PRIMARY, whichresolveV2RuntimeModeallows. A re-cutover throughopenmessage migratebuilds a fresh store and carries only pending outbox intents, so it would not inherit the copy. The same duplicate already happens today in post-cutover threads when v2 ingest runs beside the legacy-primary daemon.The mirror kept the full id only because the Signal transport (
signalQuoteArgs) resolves quotes with legacyGetMessageByID. A probe confirmed it:"signal:1700000000002"resolves, while"1700000000002"fails withsignal reply target not found. The same applies on v2-primary:SubmitTextV2forwards the v2 message's bare remote id, so every v2-native Signal quote-reply failed at dispatch.3. Adoption is safe with those two fixes, so this PR implements it.
Changes
v2wire.mirrorConversationUpsertOwnedConversationas before. If v2 ingest takes the key between the lookup and the upsert, the guarded upsert writes nothing and the mirror adopts the ingested row. The lookup comes first so that adopting takes no SQLite write lock.v2wire.Projectorremote_conversation_idand publishes that id. For conversations the mirror created, that equals the old value byte for byte.v2wire.MirrorReplyTargetsignal:<ts>is reused. Google refuses (ErrReplyTargetUnavailable, naming the id) when v2 holds the message under asource_idother than its message id: the migration keys bysource_id, but the transport quotes the message id. No live Google writer setssource_id; the R5 fixture does.signallive.signalQuoteArgs"signal:" + id, and only a Signal row is accepted. Every id that resolved before resolves identically (differential property test).web/apiv1.godocs/agent-runbook.mdInvariants (tested)
remote_conversation_idis the legacy id. The mirror keys a row by the legacy id only for a thread v2 has no row for.read_cursors.go:81), so an older mark-read leaves the migrated cursor alone."signal:" + id.Tests:
TestMirrorOnMigratedStoreProperties: 30 random seeds × up to 10 calls on a store built bymigration.Transform, with post-cutover threads randomly pre-keyed by "v2 ingest", with and without the target message.TestSignalQuoteArgsFallbackOnlyAddsResolutions: differential property against exact lookup, 60 random stores.TestMirrorReplyTargetOnMigratedStoreReusesMigratedMessages: differential check of the mirror's remote ids against what the real migration wrote.TestR5RollbackLegacyPrimarySendsIntoMigratedThreads: end to end incmd. It runs the realopenmessage migrate, a legacy-primary v2 stack with scripted adapters, and replies on Google, WhatsApp and Signal. For each it checks that the transport saw the legacy thread and the migrated quoted id, that the v2 thread gained only the sent message, that the projector wrote the send into the legacy thread with the rightReplyToID, and that nothing landed under the hash. It also covers the refused Google case and mark-read.beforeOwnedConversationUpsert), the projector on an adopted thread with and without an ingested echo, reuse of an earlier full-id Signal copy, and HTTP/api/mark-readon an adopted thread.Mutation check (each mutant applied alone, then reverted):
row.ConversationIDrow.ConversationIDsource_idcheck removedsource_idguard removedVerification
GOWORK=off go vetis clean forv2wire,signallive,webandcmd.go test -count=1passes forv2wire,signallive,bridgeadapters/...,web,tools,messaging,cutoverandmigration, and forcmd -run 'TestR5RollbackLegacyPrimarySendsIntoMigratedThreads|TestR5BackendIntegrationMigratedData'../...sweep, because the machine was short on disk. CI runs the full suite and the race job.Residuals and behavior changes to know
projectorReplayWindow). After a rollback withV2_SEND=1, sends confirmed under v2-primary are now written into their legacy threads. Before, they were written under the hash id, which moved any WhatsApp or Signal echo the legacy store already had out of its thread. For Google, the projected row is keyed by the TmpID, so if the legacy store already received the permanent echo, a leftover "sending" row can appear. That is the TmpID race the projector already documents (projector.go, "does not persist a correlation from a permanent message ID back to TmpID").last_read_message_id = NULL(as it always has) when it advances a migrated cursor that had a message id. Nothing reads v2 cursors for unread state yet.🤖 Generated with Claude Code